# Self-Hosting an OpenSource LLM for DiscourseAI

**URL:** https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923
**Category:** Self-Hosting
**Tags:** ai
**Created:** [January 8, 2024, 8:39pm UTC](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923 "2024-01-08T20:39:55Z")
**Posts on this page:** 1
**Showing post:** 13

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [July 6, 2026, 5:12pm UTC](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923/13 "2026-07-06T17:12:01Z")

</div>

> [@mononym](#):
>
> Thanks ! I noticed as well that the URL only works when the AI plugin is enabled 🤦

😅

> [@mononym](#):
>
> 1. Tokenizer

The closest to the model you will pick. For Gemma, you can pick the Gemini one.

> [@mononym](#):
>
> 1. Context window

Here you set the max context of your deployment.

> [@mononym](#):
>
> I was thinking to maybe use [stablelm2:1.6b](https://ollama.com/library/stablelm2:1.6b)

Please don’t. That model is very very very very very old. It’s the worst choice you can make.

> [@mononym](#):
>
> [translategemma:4b](https://ollama.com/library/translategemma:4b)

That model is one generation behind, and it has very strict prompt requirements that aren’t compatible with Discourse. We expect a more standard general use LLM.

At those sizes, I recommend either Qwen 3.5 or Gemma 4.

---

_[View the full topic](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923)._