Discourse AI - Large Language Model (LLM) settings page

[Saif (Saif Murtaza)]

@AquaL1te

Is there also a topic that provides a general rundown of the best cost/quality balance? Or even which LLM can be used for free for a small community and basic functionality? I can dive into the details and play around. But I’m a bit short in terms of time.

We do have this topic, that might be what you are looking for.

What LLM to use for Discourse AI?

It’s important to understand the needs of you as the community admin and your members when choosing a Large Language Model (LLM) to power Discourse AI features. Several factors may influence your decisions. Performance for use-case. Are you looking for the best-performing model? Performance can vary depending on task, the assessment is based on the model’s ability to generate correct and relevant responses. Context length. The context window is the amount of text a model can “see” and con…