# Manual configuration For VLLM

**URL:** https://meta.discourse.org/t/manual-configuration-for-vllm/330492
**Category:** Support
**Tags:** ai
**Created:** [October 11, 2024, 7:55pm UTC](https://meta.discourse.org/t/manual-configuration-for-vllm/330492 "2024-10-11T19:55:09Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Sidi\_Codet](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sidi_codet/32/455699_2.png) [@Sidi\_Codet](https://meta.discourse.org/u/Sidi_Codet)
#### Post date: [October 11, 2024, 7:55pm UTC](https://meta.discourse.org/t/manual-configuration-for-vllm/330492/1 "2024-10-11T19:55:09Z")

</div>

Please, I need help with the manual configuration of **VLLM** in the AI model section `admin/plugins/discourse-ai/ai-llms/new?llmTemplate=none`.

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [October 11, 2024, 8:04pm UTC](https://meta.discourse.org/t/manual-configuration-for-vllm/330492/2 "2024-10-11T20:04:20Z")

</div>

Sure, what exactly are you struggling with in there?

---

<div class="post-metadata">

### Author: ![Sidi\_Codet](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sidi_codet/32/455699_2.png) [@Sidi\_Codet](https://meta.discourse.org/u/Sidi_Codet)
#### Post date: [October 11, 2024, 8:08pm UTC](https://meta.discourse.org/t/manual-configuration-for-vllm/330492/3 "2024-10-11T20:08:42Z")

</div>

I’m unsure about these two options and how to utilize them, specifically in relation to the `API`.

> URL of the service hosting the model

> Tokenizer

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [October 11, 2024, 8:17pm UTC](https://meta.discourse.org/t/manual-configuration-for-vllm/330492/4 "2024-10-11T20:17:54Z")

</div>

> [@Sidi\_Codet](#):
>
> URL of the service hosting the model

That is the hostname of the machine where you are running vLLM. It may also work with an IP in the form of `http://1.1.1.1:1111` but I have not tested it as such.

> [@Sidi\_Codet](#):
>
> Tokenizer

We have a few tokenizers to helps us limit the size of the prompts before we send it to the LLM. Pick whatever produces closest results to the tokenizer used by the model you are running in vLLM, it doesn’t really need to be a perfect match.

When in doubt leave it as the OpenAI or Lllama3 one.

---

<div class="post-metadata">

### Author: ![Sidi\_Codet](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sidi_codet/32/455699_2.png) [@Sidi\_Codet](https://meta.discourse.org/u/Sidi_Codet)
#### Post date: [October 11, 2024, 8:23pm UTC](https://meta.discourse.org/t/manual-configuration-for-vllm/330492/5 "2024-10-11T20:23:57Z")

</div>

OMG, this is too complex for me atm, would go with **Sambanova** Instead!