I would definitely train your moderators to “delete as spammer” any spam accounts immediately, there’s a bunch of stuff that just can’t happen if that doesn’t happen:
none of the IPs are blacklisted
none of the emails are blacklisted (or tested for levenshtein distance)
none of the post content is sent to Akismet for bayesian spam inference
Also, your idea that “hey just look for similar emails at signup, it’s easy!” doesn’t actually help in the cases I’ve seen; it’s quite rare for spammers to use nearly-identical emails, they usually generate 12-16 character random emails. I also simply don’t agree that we should check other new accounts, all of this is predicated on spammers being deleted and correctly tagged as spammers first.
(It is interesting @sam that these bamwar-esque koreans did not fall afoul of the fast new user typist check though. I also think Stonehearth might be missing the “korean characters are an immediate warning flag” check? They also clearly came from a variety of different IP addresses, otherwise they would have hit the no-more-than-3-new-users-from-the-same-IP check.)
Stated another way, here’s all the checks these spammers passed:
Akismet explicitly said “not spam” on each and every post they made
They did not come from the same IP address, or they would have run into our default “no more than 3 new users from the same IP” block
They did not hit the “super fast typist” new user check
Failing to delete them as spammers also means:
we did not add them to the IP block list
we did not add their email to the email block list
we did not test their emails against recent spam block emails
Akismet did not get the spam content to add to its bayesian inference engine as a spam signal
Other than the korean character check, not sure there’s a whole lot else we could really do here. It is critical that staff understands to Delete as spammer as quickly as possible though. Until the user is unequivocally marked as “YES THIS IS A SPAMMER” our hands are a bit tied. I am not down with randomly comparing emails of new users, until those users have been confirmed as spammers.
Thanks for the detailed look into what happened. I will inform staff of the importance of using the “delete as spammer” option as quickly as possible. To check, does deleting a user from the Admin user page (and blocking IP/email) do the same thing? Looking through the logs it seems each moderator deleted people differently.
Understanding all the checks they managed to pass through, I am curious why you are not down with checking emails on registration? Checking emails against the last 100 registered emails shouldn’t be a performance issue, doesn’t seem to be a privacy issue (the check is by the system, not a human looking at an email), and won’t effect 99% of people. I guess what I am asking is what harm would it do to check emails before spammers are confirmed?
Right, I see the option to delete an block in all 3 spots. I just wanted to verify that it didn’t matter where the blocking was done from, as the delete and block from the spam flag queue explicitly says “Delete Spammer”.
We figured out the bamwar folks were using edits in the grace period to get around some of the spam checks. @sam added a new site setting as a temporary fix to prevent TL0 users from editing their posts, but we need a deeper fix so edits in the grace period go through the normal validation paths…
Thanks for the update @codinghorror. We appreciate you continuing to investigate the barrage of spam we received. The deeper fix you mentioned, are you planning on adding that to the 1.7 task list or do you expect it to be in a later release?
As to the feature this topic is about, I’m still questioning your stance regarding proactively checking emails against recent registrations. You’ve mentioned multiple times in this topic that it is critical to delete spammers as soon as possible. I have no issue with that, and have informed the rest of our moderators and staff about this. You’ve stated that doing so kicks in even more spam protections (IP & email blocks and the levenshtein check, for example).
My concern is what to do when the spam attack first happens and there is no staff online. I think it is unrealistic to assume that every spam post will be caught by a filter of some kind, and similarly unrealistic that there will be staff online 24/7. Not checking for similar emails until staff intervention makes it easier for a spammer to keep spamming if they’ve gotten around the filters.
It is a catch 22, until the email has been definitively marked as a spammer it cannot be checked for anything. So step zero is to delete the spammers as spammers, then the emails can be vetted.
This is what I am trying to get an answer to: why is this the case? The email is there in the database, so I don’t think it is a technical issue - but I could be wrong…
Any update on these “deeper fixes”? We’ve enabled “must approve first 2 posts” back when the attack happened, and while we haven’t seen any spam get caught, we’re hesitant to disable it until we know this fix is in.
Yes. Specifically it is the grace period edits that are the problem, not editing per se, but this was the easier fast fix. Setting grace period for edits to zero would probably also work, but significantly impact all users…
