How to configure Discourse to use a locally installed LLM?

Support
8

Uncompromising

  • DeepSeek V3 0324
  • Qwen 3 235B A22

Balanced

  • Qwen 3 32B / 30B A3B
  • Llama 3.3 70B
  • Qwen 2.5 70B

Lightweight

Maybe unsloth/gemma-3-4b-it-qat-GGUF · Hugging Face ? It’s hard at this level, much more economical to use a hosted API, like OpenRouter.

4 Likes