# Self-Hosting an OpenSource LLM for DiscourseAI

**URL:** https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923
**Category:** Self-Hosting
**Tags:** ai
**Created:** [January 8, 2024, 8:39pm UTC](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923 "2024-01-08T20:39:55Z")
**Posts on this page:** 1
**Showing post:** 4

<div class="post-metadata">

### Author: ![Isambard](https://avatars.discourse-cdn.com/v4/letter/i/858c86/32.png) [@Isambard](https://meta.discourse.org/u/Isambard)
#### Post date: [March 23, 2024, 10:48pm UTC](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923/4 "2024-03-23T22:48:26Z")

</div>

I’m using vLLM too. I also would recommend the openchat v3.5 0106 model, which is a 7B parameter model which performs very well.

I actually am running it in 4bit quantized so that it runs faster.

---

_[View the full topic](https://meta.discourse.org/t/self-hosting-an-opensource-llm-for-discourseai/290923)._