The Great WU Spam Debacle—Some Further Clarification

Thanks to Paul for giving a heads-up about the unfortunate loss of some archived comments. I thought I'd add some words as the resident WU tech-support guy to try to paint a slightly fuller picture of what happened, and hopefully allay some concerns.

First of all, the recent site redesign had nothing to do with the loss of some of the archived comments. The two are totally separate issues.

The story really begins during my two-year hiatus from active participation in the site, during which I failed to pay the close attention I should have to the functioning of the site, and so I didn't notice a massive, tumor-like mass of spam growing on the site. The problem wasn't the spam that appeared in the comments. That was only the tip of the iceberg. The real problem was the member list and the member profile pages. Many of you might not even be aware these existed... but the spammers were very aware of them. And thousands of them were registering as members every day so they could get a member profile page on which they could post spammy links in their "bios". By the time I noticed the problem, a few weeks ago, we had a quarter-million spammers registered as members.

Having that many spam members was having serious, negative consequences for the site. First, having up to 10,000 spammers registering a day was putting a big strain on the server and was slowing down the entire site. Second, the google bot was busily trying to index all these spam member pages, but was getting increasingly confused about what was the real content on the site, and was frantically indexing and re-indexing, provoking our webhost, at some point, to block the google bot entirely from the site.

So we were kind of hosed. Blocked from google and slowly being drowned in spam.

To fix the problem, the first thing I did was to block new member registration. Then I made the member profile pages non-public. Finally, I decided that all the spam members had to be deleted. The problem was how to delete the spammers while preserving the legitimate members and their comments.

I figured that the one unifying feature of all the spam members was that they hadn't left comments on the site. Instead, they had been posting the spam on the member profile pages. So I instructed the server to delete all members who had left no comments. The server churned and churned for a looooonnngg time. Finally, when it was all done, there were a couple hundred legitimate members left. And that, I thought, was the end of the matter. We were free to move forward with a clean slate.

Until Dumbfounded pointed out that a lot of old comments left by unregistered members had disappeared.

It was never my intention to delete any comments. In fact, I thought I had told the server specifically to preserve all the comments. But I'm guessing I wasn't specific enough in my instructions. I told the server to delete members who had left no comments, but I didn't say anything about NON-MEMBERS. So maybe the server interpreted my command to mean that it should delete everything EXCEPT members with comments. So all the non-registered guests got purged. That's my best guess as to what happened. Though I'm really not certain of this. The lesson is that whenever you give the server a broad, system-wide command to delete stuff, strange things can happen!

So there you have it. The Great WU Spam Debacle of 2012. At least the spam problem is far more under control than it was. So we've been left a little bruised and battered by the whole experience, but we're still standing. My apologies for the screw-up. Hopefully we can accumulate lots of great new comments to replace the old ones that are gone.
     Posted By: Alex - Fri Mar 16, 2012
     Category: Weird Universe

It's worse than that! It usually involves some fake platitudes about the site or some inane comment about the subject(?) then a link to some commercial site selling boots, clothes, or religious junk.
Posted by Expat47 in Athens, Greece on 03/20/12 at 10:05 AM
I knew about the profile pages (just follow the memberlist link, top-right, and click on anyone's name), but wasn't aware of the 1000's of dummy accounts the spammers were creating.

The 30-40 spam comments are a minor nuisance by comparison, but I hope the spammers attention doesn't switch back to the threads now their preferred outlet is closed to them.

It's a big loss for me, a lot of my posts were done non-member as I'd often drop a quick line between calls or over a lunch break. Ah well, sic transit gloria wuvie and all that.
Posted by Dumbfounded on 03/20/12 at 11:06 AM
Some sick transvestite posing as a WUvie? My Latin is a bit rough.
Posted by Expat47 in Athens, Greece on 03/20/12 at 11:40 AM
:lol: :lol: :lol: :lol: :lol: :lol: Expat!
Posted by Patty in Ohio, USA on 03/20/12 at 12:54 PM
Expat, with the spam deleted so fast, I don't get the Latin reference. Are you sure it wasn't a sweet transvestite?
Posted by jswolf19 in Japan on 03/20/12 at 07:19 PM
In order to stop spam 100%, we'd have to switch to moderating all comments by non-members. That is, every comment would have to be approved before it appeared on the site. The thing is, that's actually more work, because instead of just clicking 'delete' for the spam, we'd have to click 'approve' or 'delete' for every comment. Plus, it means that it would take longer for comments to appear on the site, which decreases the sense of spontaneity. But if the spam gets bad enough, we can go that route. I did over at the museum of hoaxes.
Posted by Alex on 03/20/12 at 07:34 PM
I hope we do not have to do that Alex.

'sweet transvestite'
from transsexual Transylvania! :lol:
Posted by Patty in Ohio, USA on 03/20/12 at 08:34 PM
No SPAM Jwolf, DF posted the Latin in just before mine. I was only trying to translate it.
Posted by Expat47 in Athens, Greece on 03/21/12 at 01:05 AM
Ah, I didn't get DF's post in email form, I don't think... ^_^
Posted by jswolf19 in Japan on 03/21/12 at 09:16 AM
Page 2 of 2 pages  < 1 2
Commenting is not available in this channel entry.