Random discussion about Discourse LLM setup

[Saif (Saif Murtaza)]

@AquaL1te

Is there also a topic that provides a general rundown of the best cost/quality balance? Or even which LLM can be used for free for a small community and basic functionality? I can dive into the details and play around. But I’m a bit short in terms of time.

We do have this topic, that might be what you are looking for.

What LLM to use for Discourse AI?

It’s important to understand the needs of you as the community admin and your members when choosing a Large Language Model (LLM) to power Discourse AI features. Several factors may influence your decisions: Performance for use-case: Are you looking for the best-performing model? Performance can vary depending on the task (e.g., summarization, search, complex reasoning, spam detection). Assessment is based on the model’s ability to generate correct, relevant, and coherent responses. Context le…