Sorry guys I couldn’t figure out from this article if locally installed LLM can be confgirued through the standard settings UI?

I think as long as it exposes a supported API it should be possible. Is there a particular LLM you are planning to (or already have) installed locally?

this topic may help [image] Self-Hosting an OpenSource LLM for DiscourseAI Self-Hosting The Discourse AI plugin has many features that require an LLM to be enabled, like, for example, Summarization, AI Helper, AI Search, AI Bot. While you can use a third party API, like Configure API Keys for OpenAI or Configure API Keys for Anthropic we built Discourse AI since first day to not be locked into those. Running with HuggingFace TGI HuggingFace provides an awesome container image that can get you running quickly. For example: mkdir -p /opt/tgi-cache docker run --rm --gpus all --shm…

[image] mcwumbly: Is there a particular LLM you are planning to (or already have) installed locally? That’s another exercise to choose the right one - I wasn’t certain even after reading your AI-related articles here at Meta. I guess some Open Source LLM Selector tool from the Discourse Team would be very helpful - because you know the internals and what exactly LLM must be capable of doing for it to excel in various types of tasks relevant to Discourse communities. So, the tool/wizard/LLM would ask questions or let me check on/off in a list of 20+ typical tasks I’d like the LLM to do in my community, and then get a recommended Top 3 (uncompromising but heaviest and requires expensive hardware; balanced that requires medium-priced dedicated server; and lightweight for basic tasks in small-to-medium communities that can run on a $20-40 VPS).

[image] meglio: So, the tool/wizard/LLM would ask questions or let me check on/off in a list of 20+ typical tasks I’d like the LLM to do in my community, and then get a recommended Top 3 (uncompromising but heaviest and requires expensive hardware; balanced that requires medium-priced dedicated server; and lightweight for basic tasks in small-to-medium communities that can run on a $20-40 VPS). I think maintaining the correct answer to that would be a full time job. :rofl:

If you are looking for cheap, Gemini Free tier is the way to go: [image] Gemini Developer API pricing | Gemini API | Google AI for Developers Gemini Developer API Pricing Gemini flash 2.0 is a very capable model and the free tier provides enough usage to perform quite a few things on your forum.

Looking for locally hosted because I can’t send any of our content to any service outside our servers. Re cheap options, thanks for the hint!

[image] meglio: a recommended Top 3 (uncompromising but heaviest and requires expensive hardware; balanced that requires medium-priced dedicated server; and lightweight for basic tasks in small-to-medium communities that can run on a $20-40 VPS). Uncompromising DeepSeek V3 0324 Qwen 3 235B A22 Balanced Qwen 3 32B / 30B A3B Llama 3.3 70B Qwen 2.5 70B Lightweight Maybe unsloth/gemma-3-4b-it-qat-GGUF · Hugging Face ? It’s hard at this level, much more economical to use a hosted API, like OpenRouter.

How to configure Discourse to use a locally installed LLM?

Product Support

pfaffman (Jay Pfaffman) June 1, 2025, 3:26pm 5

I think maintaining the correct answer to that would be a full time job.

Topic		Replies	Views
Self-Hosting an OpenSource LLM for DiscourseAI Self-Hosting ai	12	4605	July 6, 2026
What LLM to use for Discourse AI? Site Management how-to , ai	0	1132	January 23, 2025
Inquiry About AI Plugin Options Product Support ai	6	405	November 24, 2025
Random discussion about Discourse LLM setup Product Support	21	308	August 20, 2026
Estimating cost of enabling Discourse AI for related content and search Product Support ai	2	328	October 28, 2025

How to configure Discourse to use a locally installed LLM?

Related topics