I’m trying to setup the AI spam detector for my own Discourse instance, but have several questions: The system prompt of spam detector persona can’t be edited. I can create a new persona but there is no field to make Discourse know that it is going to be used as a spam detector? I want to use Qwen3 as the LLM for the task. There is no preset for Qwen3 so I’m going to create a custom one. But there isn’t a Qwen3Tokenizer either. What does the tokenizer do here? Does QwenTokenizer work for Qwen3? Or, is there an easier option that I can setup a webhook-like service and Discourse calls it on new topics / posts?

I just did a quick setup and it works fine selecting the custom persona for spam. [Screenshot 2025-11-19 at 1.54.11 PM] [Screenshot 2025-11-19 at 1.51.51 PM] Let us know again if you are not able to do so via /discourse-ai/ai-spam

Hi there, to answer your question: [图片] lilydjwg: What does the tokenizer do here? From the docs: [图片]Discourse AI - Large Language Model (LLM) settings page What is tokenizer? The tokenizer translates strings into tokens, which is what a model uses to understand the input.

Thanks, but I know what a tokenizer is. What I don’t know is what Discourse uses it for, and what would happen if I picked the wrong tokenizer? I don’t need a tokenizer to call an LLM anyway.

[图片] lilydjwg: The system prompt of spam detector persona can’t be edited. Can you try visiting this https:// /admin/plugins/discourse-ai/ai-spam - it should allow you to select your persona insert custom instructions and test against a post [图片] lilydjwg: Does QwenTokenizer work for Qwen3 Yup you can go ahead and do that, then run the “Test”

[图片] nat: insert custom instructions IIUC this instruction is concatenated to the system prompt instead of replacing it, making the prompt even longer and have mixed languages… [图片] nat: Yup you can go ahead and do that, then run the “Test” But passing the test doesn’t mean there is no problem. (FYI I haven’t fully setup the LLM yet; I tried the free Gemini service but got 429, so I’m trying to figure out how to integrate llama.cpp into Discourse now.)

Thanks, so that “spam detector” label only affects the list filering of personas? I’ll setup a test environment and try it.

I checked the source code. If I didn’t make mistakes, the tokenizer is used for two things: count tokens for statistics and price estimation, and truncate posts to the set limit. So it shouldn’t affect me a lot if a wrong one is used.

关于AI垃圾邮件检测器的挫败感

产品支持

NateDhaliwal (Nate Dhaliwal) 2025 年11 月 19 日 03:59 2

您好，为了回答您的问题：

根据文档：

话题		回复	浏览量
AI powered Spam detection Announcements ai , spam	11	1412	2025 年1 月 11 日
Random discussion about Discourse LLM setup Product Support	21	308	2026 年8 月 20 日
Introducing Discourse AI Blog	26	4570	2023 年5 月 4 日
What's the cheapest/best AI to use for AI Spam? Product Support ai , spam	6	591	2025 年3 月 18 日
Discourse AI spam detection replaces Akismet plugin Announcements akismet , ai , spam	13	1602	2025 年11 月 4 日

关于AI垃圾邮件检测器的挫败感

相关话题