Please, I need help with the manual configuration of VLLM in the AI model section admin/plugins/discourse-ai/ai-llms/new?llmTemplate=none.

I’m unsure about these two options and how to utilize them, specifically in relation to the API. URL of the service hosting the model Tokenizer

[afbeelding] Olajuwon: URL of the service hosting the model That is the hostname of the machine where you are running vLLM. It may also work with an IP in the form of http://1.1.1.1:1111 but I have not tested it as such. [afbeelding] Olajuwon: Tokenizer We have a few tokenizers to hel…

OMG, this is too complex for me atm, would go with Sambanova Instead!

Handmatige configuratie voor VLLM

Ondersteuning

Falco (Falco) 11 oktober 2024 om 20:04 2

Sure, what exactly are you struggling with in there?

Topic		Antwoorden	Weergaven
Debugging adding new LLM Support ai	8	571	23 augustus 2024
Self-Hosting an OpenSource LLM for DiscourseAI Self-Hosting ai	12	4259	6 juli 2026
创建可用的llm后也无法设置 AI default LLM model Support ai	3	131	20 maart 2026
Adding Semantic Search feature for our self-hosted discourse site Support ai , ai-search	9	386	19 maart 2025
Can I add other LLM? Support ai	13	470	2 augustus 2024

Handmatige configuratie voor VLLM

Gerelateerde topics