Frustrations on AI spam detector

Product Support
,

AI-generated summary

The discussion revolves around setting up the AI spam detector in a Discourse instance. lilydjwg seeks clarification on customizing the spam detector persona and the role of the tokenizer. NateDhaliwal explains that the tokenizer converts text into tokens for processing, while nat clarifies that the ‘spam detector’ label is used to identify the persona in the admin interface. lilydjwg notes that custom instructions are appended to the system prompt, potentially causing mixed languages. nat confirms the setup works with a custom persona. lilydjwg concludes that the tokenizer is used for token counting and statistical analysis, affecting performance.

1

I’m trying to setup the AI spam detector for my own Discourse instance, but have several questions:

The system prompt of spam detector persona can’t be edited. I can create a new persona but there is no field to make Discourse know that it is going to be used as a spam detector?

I want to use Qwen3 as the LLM for the task. There is no preset for Qwen3 so I’m going to create a custom one. But there isn’t a Qwen3Tokenizer either. What does the tokenizer do here? Does QwenTokenizer work for Qwen3?

Or, is there an easier option that I can setup a webhook-like service and Discourse calls it on new topics / posts?

2

Hi there, to answer your question:

From the docs:

3

Thanks, but I know what a tokenizer is. What I don’t know is what Discourse uses it for, and what would happen if I picked the wrong tokenizer? I don’t need a tokenizer to call an LLM anyway.

4

Can you try visiting this https://<your-site>/admin/plugins/discourse-ai/ai-spam - it should allow you to

  • select your persona
  • insert custom instructions and test against a post

Yup you can go ahead and do that, then run the “Test”

5

IIUC this instruction is concatenated to the system prompt instead of replacing it, making the prompt even longer and have mixed languages…

But passing the test doesn’t mean there is no problem. (FYI I haven’t fully setup the LLM yet; I tried the free Gemini service but got 429, so I’m trying to figure out how to integrate llama.cpp into Discourse now.)

6

I just did a quick setup and it works fine selecting the custom persona for spam.

Screenshot 2025-11-19 at 1.54.11 PM
Screenshot 2025-11-19 at 1.54.11 PM2254×1828 297 KB

Screenshot 2025-11-19 at 1.51.51 PM
Screenshot 2025-11-19 at 1.51.51 PM2282×1976 378 KB

Let us know again if you are not able to do so via /discourse-ai/ai-spam

7

Thanks, so that “spam detector” label only affects the list filering of personas?
I’ll setup a test environment and try it.

8

It seems to work, thank you!

9

I checked the source code. If I didn’t make mistakes, the tokenizer is used for two things: count tokens for statistics and price estimation, and truncate posts to the set limit. So it shouldn’t affect me a lot if a wrong one is used.