I’m setting up Discourse AI on my forum and have an AI Bot/Agent working with Gemini 2.5 Flash via LiteLLM and Google Vertex AI. The bot is working, but I have a problem with its scope. For example, my forum is a business-focused community. If I ask the bot: “Who owns Uber?” it answers the question using the LLM’s general knowledge, even though the question has nothing to do with my forum. This creates two problems: I want the bot to function as a forum-specific knowledge assistant, not as a general-purpose Gemini/ChatGPT interface. I don’t want users to be able to ask arbitrary questions and consume my LLM credits. What I would ideally like is: User asks question ↓ Is it relevant to this forum? ↓ NO → politely refuse ↓ YES ↓ Search forum knowledge ↓ Generate answer based on forum content [image]

Yeah, after a conversation gets deep and you chain it, you can get it to keep on going. Not a worry for us as we host the models ourselves, so we don’t pay per token anyway. We also analyze all conversations on Ask daily and can ban any bad actors. The forum moderation flow is still the same. If you want even more control than what the system prompt gives you, you can try some approaches: Workflows Our new workflow plugin would allow you to do a 2-pass on every message, first with an agent evaluating if a message should be replied to, then another conditionally replying. Fine-tuned models The most surefire way to control is to deploy a model that is actually trained with the behavior you want. This is easy with Discourse AI, as we allow you to pick and choose LLMs for each agent.

Have you taken a look at Agents? Perhaps the Forum Researcher or Forum Helper Agents would be a better fit, as they are tuned to work with the forum specifically. Or is that what you are already using?

Forum Helper also has this problem: [image]

Hmm… then perhaps you might want to make your own agent with a customized system prompt to not respond to queries outside of your forum’s scope.

Here’s what I see today if I try this on ask.discourse.com Thanks for the question! Since you asked on a Discourse support channel, I’ll answer in the context of how language models power Discourse’s AI features — but let me know if you actually meant a general technical explanation of how LLMs work, and I can point you elsewhere for that. Someone round here must know how that catch was made.

The “Creative” agent you used in the test is our “no system prompt” agent, that just passes the request to the underlying LLM. You will want to craft a new agent, with a system prompt that specifies how and when to answer on your own forum. That’s what we did on ask.discourse.com as mentioned by @Ed_S.

Although prompts can influence an agent’s behavior, they aren’t always reliable. I was able to trick the agent at ask.discourse.com into answering a few off-topic questions. [image]

How can I restrict an AI Agent to answer only questions related to my forum?

Product Support

Ed_S (Ed S) August 11, 2026, 4:24pm 8

(This is probably one of those situations where 100% enforcement would cause more inconvenience than benefit. )

Topic		Replies	Views
You can now use Discourse to power your own AI Q&A homepage Announcements ai	3	299	September 24, 2026
Why is my AI forum helper struggling to answer questions? Product Support ai , ai-bot	3	639	September 15, 2025
AI bot - Agents Site Management how-to , ai , ai-bot	26	5171	April 15, 2026
Email my forum? Feature ai	6	364	October 9, 2024
Introducing AI Bot 🤖 Blog	0	1050	October 25, 2023

How can I restrict an AI Agent to answer only questions related to my forum?

Related topics