How to set up Discourse AI for internal-only usage

[Thefacto]

@Lilly

perhaps this topic may interest you

The Discourse AI plugin has many features that require an LLM to be enabled, like, for example, Summarization, AI Helper, AI Search, AI Bot. While you can use a third party API, like Configure API Keys for OpenAI or Configure API Keys for Anthropic we built Discourse AI since first day to not be locked into those. Running with HuggingFace TGI HuggingFace provides an awesome container image that can get you running quickly. For example: mkdir -p /opt/tgi-cache docker run --rm --gpus all --shm-…

If you mean using a Local LLM, I do not have plans to increase expenses yet. On the server, it requires a lot of processing power for more than 20 concurrent users, so this plan is not being implemented. I would like to focus on using external APIs such as Groq or OpenRouter, which are more cost-effective, and try to control costs in this area.