Discourse Akismet Anti-Spam

Fight spam with Akismet, an algorithm used by millions of sites to combat spam automatically.

What does it do?

Akismet helps keep your site free of spam by automatically scanning all posts from new users. Scanned posts that Akismet flags as spam are immediately removed from the site and added to a queue for review. Site staff can then review the posts to confirm spam, or restore if the posts are not spam. Akismet learns as staff confirm or restore spam posts, improving its spam detection and decreasing false positives.

Spam sucks, Fight it with Akismet

Akismet is a well known service that trains a Bayesian filter for detecting spam specific to your domain. Akismet is NOT free for commerical use, but can be for personal use. To use this plugin you will need an Akismet API key, you can get one here.

How it works

The plugin works by collecting info about a new post’s HTTP request. Every 10 minutes, a background job runs that looks for new posts. All new posts are sent to Akismet to determine if they are spam or not. If a post is deemed spam, it is deleted and placed in a moderator queue where staff can take action against it.

How to use it

After enabling the plugin, you can find the moderator queue by visiting /admin/plugins/akismet or from the Hamburger menu.

Screenshot%20from%202019-02-18%2010-46-31


Action Result
Confirm Spam Confirms the post as spam, leaving it deleted, and tells Akismet that it was spam.
Not Spam Akismet thought something was spam but it actually wasn’t. This undeletes the post and tells Akismet that it wasn’t spam. Akismet gets smarter so it hopefully won’t make the same mistake twice.
Dismiss Confirms the post as spam, leaving it deleted, but doesn’t notify Akismet.
Delete user It will delete the user, their posts, topics and block their email and IP address.

What data is sent to Akismet?

Field Name Discourse Value
Author User’s Name
Author Email User’s verified email (can be disabled with the akismet_transmit_email site setting)
Comment Type “forum-post”
Content Post’s raw column (including post’s topic title if first post)
Permalink Link to topic
User IP IP address of request
User Agent User agent of request
Referrer HTTP referrer of request

Installation

Follow our Install a Plugin howto, using
git clone https://github.com/discourse/discourse-akismet.git as the plugin command.

Once you’ve installed, add your akismet key under site settings by searching for akismet.
Alternatively, you can also add it using a DISCOURSE_AKISMET_API_KEY environment variable.

Also make sure to enable the akismet_enabled site setting.

Development Setup

cd plugins
git clone https://github.com/discourse/discourse-akismet.git

Once Discourse starts up make sure you enter your akismet_api_key under site settings.

Testing

Once you have the plugin installed, let’s do a quick test to make sure everything is working. Login as a non admin user and create a new topic and post. Use the following info:

title: Spam test - Will this plugin do what it says!
post: love vashikaran, love vashikaran specialist,919828891153 love vashikaran special black magic specialist hurry hurry love now

Now, go to /sidekiq/scheduler and find the CheckForSpamPosts jobs and trigger it. Now, as a staff member, view the moderator queue by going to admin/plugins/akismet or by using the Hamburger menu. You should see the post with some additional info about it.

15 Likes

Two minor issues here:

  • When the reviewable_api_queue is enabled and in use, this plugin still creates a PostCustomField AKISMET_STATE when checking for spam, e.g. ‘new’ or ‘needs_review’. However, if the post is confirmed (or not) as spam, this state is not updated (e.g. to ‘confirmed_spam’).

  • The OP needs updating. If the reviewable_api_queue is enabled admin/plugins/akismet is not registered as a route.

    Now, as a staff member, view the moderator queue by going to admin/plugins/akismet or by using the Hamburger menu. You should see the post with some additional info about it.

1 Like

Plugin is working well, one improvement that could really speed up spam cleaning would be if posts from deleted users are removed from the Akismet review queue and don’t need to be processed.

Example: Spammer makes 50 spammy posts which are in the Akismet review queue. The ‘Confirm Spam & Delete User’ is used on one of the posts. The remaining 49 posts will stay in the queue from ‘Deleted User’ and essentially have to be processed manually. The amount of extra time and action needed to clean up spam really adds up.

20

4 Likes

Interesting, have you considered the above @eviltrout?

2 Likes

I guess our Moderator team is quick, but I think Akismet should Silence a member way before they make 50 posts. Perhaps not at a single suspect post, but at least three or so?

Deleting a user is supposed to agree with any flags on their posts, so I’m curious as to why this is happening. If you refresh the page do the posts go away? I’m wondering if the bug is that the UI is not updated but they are handled.

@eviltrout I can confirm that refreshing doesn’t remove them from the Akismet review queue and the review list count/items stays consistent between different browsers/devices. I’m using the default Discourse theme. To clarify, I’m using the ‘Confirm Spam & Delete User’ option from the Akismet review queue.

One other very minor issue I noticed, is that infinite scrolling stops working after taking action on any posts in the list.

@Mittineague I’m not sure if Akismet automatic detection prevents the user from submitting more posts? I think it might just hide their posts while waiting for human review.

Handled about 1000 of them over the last 24hrs, most of which were Akismet flags for posts by deleted users.

:checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag::checkered_flag:
45

3 Likes

Aha. I see that Confirm Spam & Delete User doesn’t delete all their posts.

I think that if we are confirming a post is spam and deleting a user, we can assume that their other posts are no good too right? What do you think @codinghorror - should we go ahead and delete all their previous posts (and any flags on them).

7 Likes

Probably, because at the point you’re saying Confirm Spam & Delete User that is really clear intent.

6 Likes

Ahh… yeah would be really great if the posts are deleted too. There is a confirmation prompt when using that option also:

“Are you sure you want to delete this user? This will remove all of their posts and block their email and ip address.”

In my opinion, it’d be best if the post deletion is queued up and done in the background (unlike the delete all posts on the user admin page currently, which needs to be completed before deleting the user is an option).

It can be quite a slow process with a lot of multi-tasking and tab toggling if the admin needs to wait for all of the users posts to be deleted. It can also slow down the site if too many are done at the same time, depending on server resources.

Also, if this doesn’t exist already, a cron that periodically searches for and deletes orphan posts by deleted users could be a good idea? I believe something like this already exists for deleting orphaned images.

2 Likes

@Roman_Rizzi can you look at this at some point? Not super high priority but it should be fixed to be consistent with the text.

6 Likes

Anyone know how to fully remove akismet + data?

I uninstalled it, but the review queue is still the same length and I get a ‘Error code: 500 Internal Server Error’ if I try open the review queue.

I’ve reinstalled it and disabled it in the settings for the time being to resolve this.

Hi @eviltrout,

Same error for us as @markersocial , accessing to https://somesite.io/review, we get the following 500 message:
image

Having a look to the log, it says:

# info
NoMethodError (undefined method `button_class=' for #<Reviewable::Actions::Action:0x00007faee0123d88>)
/var/www/discourse/plugins/discourse-akismet/models/reviewable_akismet_post.rb:72:in `block in build_action'

# backtrace

/var/www/discourse/plugins/discourse-akismet/models/reviewable_akismet_post.rb:72:in `block in build_action'
/var/www/discourse/lib/reviewable/actions.rb:52:in `add'
/var/www/discourse/plugins/discourse-akismet/models/reviewable_akismet_post.rb:68:in `build_action'
/var/www/discourse/plugins/discourse-akismet/models/reviewable_akismet_post.rb:9:in `build_actions'
/var/www/discourse/app/models/reviewable.rb:241:in `block in actions_for'
/var/www/discourse/app/models/reviewable.rb:240:in `tap'
/var/www/discourse/app/models/reviewable.rb:240:in `actions_for'
/var/www/discourse/app/serializers/reviewable_serializer.rb:39:in `bundled_actions'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer/associations.rb:71:in `associated_object'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer/associations.rb:139:in `serialize_ids'
/var/www/discourse/lib/freedom_patches/ams_include_without_root.rb:49:in `include!'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:368:in `block in include_associations!'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:367:in `each_key'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:367:in `include_associations!'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:362:in `serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:347:in `as_json'
/var/www/discourse/app/controllers/reviewables_controller.rb:49:in `block in index'
/var/www/discourse/app/controllers/reviewables_controller.rb:42:in `map!'
/var/www/discourse/app/controllers/reviewables_controller.rb:42:in `index'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/abstract_controller/base.rb:194:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal/rendering.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/abstract_controller/callbacks.rb:41:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal/rescue.rb:22:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.3/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/abstract_controller/base.rb:134:in `process'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionview-5.2.3/lib/action_view/rendering.rb:32:in `process'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-mini-profiler-1.1.0/lib/mini_profiler/profiling_methods.rb:78:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal.rb:191:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_controller/metal.rb:252:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/route_set.rb:34:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/journey/router.rb:52:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/journey/router.rb:35:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/journey/router.rb:35:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/routing/route_set.rb:840:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.0.5/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:32:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/tempfile_reaper.rb:15:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/conditional_get.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/head.rb:12:in `call'
/var/www/discourse/lib/content_security_policy/middleware.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:220:in `call'
/var/www/discourse/config/initializers/008-rack-cors.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/session/abstract/id.rb:232:in `context'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/session/abstract/id.rb:226:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/cookies.rb:670:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:98:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/logster-2.3.2/lib/logster/middleware/reporter.rb:43:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/rack/logger.rb:28:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:18:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/var/www/discourse/lib/middleware/enforce_hostname.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/method_override.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-5.2.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/sendfile.rb:111:in `call'
/var/www/discourse/plugins/discourse-prometheus/lib/middleware/metrics.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-mini-profiler-1.1.0/lib/mini_profiler/profiler.rb:184:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-2.2.0/lib/message_bus/rack/middleware.rb:57:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:163:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:524:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/railtie.rb:190:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/railtie.rb:190:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/urlmap.rb:68:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/urlmap.rb:53:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/urlmap.rb:53:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:605:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:700:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:548:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:144:in `start'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/bin/unicorn:128:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `<main>'

Did not get too much into details, but could it be related to this commit? FIX: Add button_class arguments to the build_action functions · discourse/discourse-akismet@d079c2e · GitHub

We are using stable branch for deployment.

Cheers,
Ismael

2 Likes

@Roman_Rizzi can you take a look at this one? ^

3 Likes

Oops, looks like we forgot to backport this commit:

I just cherry-picked it into stable.

4 Likes

I can confirm it works for stable branch. Many thanks for your quick action :clap: !

Cheers,
Ismael

3 Likes