Is there a way to Add an API key from an LLM that is not from those listed? We use this LLM called Deepseek and would like to integrate it. Already have the Api keys etc.

AFAIK, no, as Discourse probably only supports those listed. If I am wrong, anyone, feel free to contradict me.

[صورة] oga: Deepseek https://github.com/deepseek-ai/DeepSeek-LLM It works with VLLM, you can can just configure a VLLM endpoint and it should work out of the box.

WOW !! Thanks so much. I will try this out.

Having support for almost every modern LLM that exist was a big focus for release 3.3, and we finally got to it, just in time when open weights LLMs are catching up to closed ones. Let me know if you need help configuring DeepSeek in your instance, this feature is brand new and we are still updatin…

Yes I do need help. Looked around but don’t see how to add it without VLLM. VLLM is another overhead I would rather avoid due the requirements. Would be great if there’s an integration not only for Deepseek, but for many other Open source LLMs. I am of the opinion that’s where this LLM race is head…

[صورة] oga: From what we see, companies are hesitant handing out their Data to the “cloud”. Open source that allows them to run local instances using tools like GPT4ALL, HuggingFace will end up being the preferred model. Yes, I agree with that, which is why we support running your own LLM fro…

Thanks, from my reading, {may have read it wrong) don’t I need a new Droplet [Digital Ocean] aka, VPS to deploy a VLLM? or can it be deployed on the same Droplet that hosts my Discourse Instance? If yes, on the same instance, would it be on the same directory as Discourse or a separate Directory? …

What exact droplet are you running your current Discourse site? The $6 one ?

The $15 one. I think it is 2 GB ram. Once we finish testing I will scale it up. Thanks

[صورة] oga: The $15 one. Hmm, are you aware that a droplet to run this Deepseek model you want will cost you like $10,000 a month?

هل يمكنني إضافة نماذج لغة كبيرة أخرى؟

الدعم

sam (Sam Saffron) 1 أغسطس 2024، 12:01ص 3

إنه يعمل مع VLLM، يمكنك ببساطة تكوين نقطة نهاية VLLM ويجب أن يعمل بشكل مباشر.

Debugging adding new LLM

Rebuilding Discourse after invalid GitHub repo addition

الموضوع		الردود	مرات العرض
DeepSeek provider support? What to do when model provider isn't in "Provider" list? Support ai	12	1101	3 فبراير 2025
Debugging adding new LLM Support ai	8	486	23 أغسطس 2024
How to configure Discourse to use a locally installed LLM? Support ai	7	362	3 يونيو 2025
Self-Hosting an OpenSource LLM for DiscourseAI Self-Hosting ai	12	3950	6 يوليو 2026
I need help to configure LLM DeepSeek R1 0528 - free for Automatic translations with Discourse AI Support content-localization	33	897	3 ديسمبر 2025

هل يمكنني إضافة نماذج لغة كبيرة أخرى؟

الموضوعات ذات الصلة