Random discussion about Discourse LLM setup

[sam (Sam Saffron)]

We improved the UI a lot in the past week, can you try it out again?

[hameedacpa (Abdelhamid)]

When Gemini 2.0 will be supported ?

[sam (Sam Saffron)]

Been supported for quite a while.

[Joe_F (Joe Farrar)]

I seem to have an issue where I cannot Select a LLM even though I have the CDCK hosted ones configured..

is this normal?

[sam (Sam Saffron)]

A lot to unwrap here, which llm are you trying to choose for what?

The CDCK LLMs are only available for very specific features, to see which you need to head to /admin/whats-new on your instance and click “only show experimental features”, you will need to enable them to unlock the CDCK LLM on specific features.

Any LLM you define outside of CDCK LLMs is available to all features.

[AquaL1te]

Is there also a topic that provides a general rundown of the best cost/quality balance? Or even which LLM can be used for free for a small community and basic functionality? I can dive into the details and play around. But I’m a bit short in terms of time.

For example, I only care about spam detection and a profanity filter. I had this for free, but those plugins are deprecated or soon to be. It would be nice if I can retain this functionality without having to pay for an LLM.

[Saif (Saif Murtaza)]

@AquaL1te

Is there also a topic that provides a general rundown of the best cost/quality balance? Or even which LLM can be used for free for a small community and basic functionality? I can dive into the details and play around. But I’m a bit short in terms of time.

We do have this topic, that might be what you are looking for.

What LLM to use for Discourse AI?

It’s important to understand the needs of you as the community admin and your members when choosing a Large Language Model (LLM) to power Discourse AI features. Several factors may influence your decisions: Performance for use-case: Are you looking for the best-performing model? Performance can vary depending on the task (e.g., summarization, search, complex reasoning, spam detection). Assessment is based on the model’s ability to generate correct, relevant, and coherent responses. Context le…

[AquaL1te]

Done! It was indeed pretty easy. But maybe for a non techie it may still be a bit hard to setup. For example, the model name was automatically set in the settings, but wasn’t the correct one. Luckily I recognized the model name in a curl example for Claude on the API page and then it worked

Estimated costs are maybe 30 euro cents per month for spam control (I don’t have a huge forum). So that’s manageable! I’ve set a limit of 5 euros in the API console, just in case.

[Saif (Saif Murtaza)]

@AquaL1te

For example, the model name was automatically set in the settings, but wasn’t the correct one.

Which one did you pick for Claude? What was the incorrect name shown, and what did you correct it to?

[AquaL1te]

I use Claude 3.5, the model ID is by default claude-3-5-haiku, but I had to change it to claude-3-5-haiku-20241022, otherwise I got an error.

[Saif (Saif Murtaza)]

@AquaL1te

I use Claude 3.5, the model ID is by default claude-3-5-haiku, but I had to change it to claude-3-5-haiku-20241022, otherwise I got an error.

Good to note, yeah sometimes there might be a disconnect. The auto-populated info should act as guidance, which tends to work most of the time, but does fall short in certain cases such as yours (given all the different models and provider configs)

I have updated the OP of this guide

[jrgong]

@Discourse

• Gemini Flash 2.0

This model is not listed on 3.4.2 - are those pre-configs only available on 3.5 and I have to add them manually?

Edit: Also what option do I choose for “Tokenizer” when using Grok 3 models?

[Falco]

@jrgong

This model is not listed on 3.4.2 - are those pre-configs only available on 3.5 and I have to add them manually?

Pre-configs are simply templates, you can get the same end result by using the “Manual configuration”.

@jrgong

Edit: Also what option do I choose for “Tokenizer” when using Grok 3 models?

I’ve found that the Gemini tokenizer is pretty close the the Grok one, so try that.

[CraigW (Craig Wright)]

Is there a way to use IBM WatsonX through the current configuration management, or would this require additional development work by the Discourse staff?

[Falco]

Does IBM WatsonX expose an OpenAI compatible API by any chance?

[CraigW (Craig Wright)]

Great question. A quick poke around the docs didn’t tell me much, but the fact that this repository exists suggests that it is not directly compatible: GitHub - aseelert/watsonx-openai-api: Watsonx Openai compatible API · GitHub

[AntiMetaman]

Which of these LLMs are free to use for anti-spam?

Edit: Nm, I’m using Gemini Flash 2.5

[pfaffman (Jay Pfaffman)]

I always wonder too. This seems like the best answer to that question.

@Saif

• Spam Detection / AI Triage (Text): Cost-Effective/Speed models are usually sufficient and cost-efficient (Haiku 3.5, Gemini 2.0 Flash, GPT-4o mini, Llama 3.3).
• AI Triage (NSFW Image Detection): Requires a Vision Capable model (GPT-4o/mini, Sonnet 3.7, Gemini 2.5 Pro/2.0 Flash, Llama 3.2, specific Discourse hosted/supported models).

But also, there is this in the OP from the Spam config topic. I think it’s just a little hard to find in all of the information that’s there.

@sam

LLM selection recommendations

The performance of spam detection depends on the chosen LLM.

Most low-cost LLMs work effectively, such as:

• GPT-4o-mini
• Claude 3.5 Haiku
• Gemini 2.0 Flash

Experiment with different models to find the best fit. Configure your models via /admin/plugins/discourse-ai/ai-llms.

[danhanghai]

Hello, I’m new to Discourse. I have just tried to set up gemini-3.1-flash-litebut it didn’t work. Hope to have your support. Thank you.

1. Model id: gemini-3.1-flash-lite
2. Provider: Google
3. URL of the service hosting the model: https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite

[OS_Corsair]

Okay … I would like a close kin of xAI Grok