Discourse AI - Spam detection

Documentation Site Management
, , ,
21

And he’s an example of the same spammers getting caught here: https://meta.discourse.org/t/full-list-of-quickbooks-desktop-support-contact-numbers-a-complete-call-center-in-the-usa/380776 (it’s already hidden).

These guys are definitely working hard.

1 Like
22

Great work on this feature. :raising_hands: This is exactly how I like to see AI used.

Quick question: when a new TL0 user submits a reply or topic, is there a time delay while the content is scanned?

I see a short delay in the built-in tester (screenshot below), but when I post from a test account, there’s no similar pause. Is the live scan asynchronous after publishing, with the post hidden only if it trips a rule? (context: I’m using the OpenAI ChatGPT 5 API.)

For what it’s worth, AI > Spam & Stats increment as expected with the test account, so the post IS being scanned; it just isn’t introducing the same delay as the Test button does.

image

Thanks.

23

Ok so this works pretty well but what happens when it flags dozens of topics/users. I’m not seeing a way to bulk ban/delete these users/posts.

24

Thanks for the detailed thread. We have Discourse AI spam detection enabled on our instance, and one of the things that we’re seeing is the auto-silencing default when the first post made by an account is flagged.

I understand this is for silencing one-shot spammers; however, this is causing issues where approving/accepting a flag means that the user will remain silenced in cases where we want to approve the flag but not silence the user. It would be good to have:

  1. an “agree and keep silenced” button, and
  2. a separate “agree and lift silencing.” button.
2 Likes
25

This is a tricky one, we don’t want to paralyze people with choice here, but I totally get that at scale this can be a problem.

Let me check with the enterprise xp team maybe there is a small customization we can make for your forum.

1 Like
26

I published a test spam topic in my local development environment, but it did not automatically enter the review queue.

The AI detection result is indeed a spam post.

And it also meets the other conditions for entering the review queue.

  1. User Trust Level:

    • Scan posts from users with a trust level of 1 or lower.

    • Exclude posts from users with a higher trust level.

  2. Post Type:

    • Public posts (excluding private messages).

    • Include reply posts and first topic posts based on other thresholds.

  3. Post Edits:

    • Scan posts with significant edits (e.g., changes exceeding 10 characters).

    • Enforce a 10-minute delay between scans of the same post.

  4. Post Frequency:

    • Prioritize cases where new users have posted a total of fewer than 4 posts in public topics.

    • Exclude posts from users exceeding this threshold.

However, the final result is that it did not enter the review queue.

Where should I check to find the problem?

27

Hi @singi2016cn.

Do you meant that you verified this with the testing tool?

You can access that tool by following these instructions:

  1. Log into an account on your forum that has administrator privileges.
  2. Navigate to this page on your forum: /admin/plugins/discourse-ai/ai-spam
  3. Click the “Test…” button on that page.
    The “Test spam detection” dialog will open.
  4. Enter the URL or post ID of your test spam post into the “Post URL or ID” field in the dialog.
  5. Click the “Run test” button.
2 Likes
28

Yes, the testing tool clearly returned Spam, but when I posted the exact same content, it didn’t enter the moderation queue.

29

Who posted this? Did you use a new user you created for testing or did you for example use an account with moderator permissions?

30

Regular user, trust_level_1 trust level, not an administrator or a moderator.