Yes I do need help. Looked around but don’t see how to add it without VLLM. VLLM is another overhead I would rather avoid due the requirements.
Would be great if there’s an integration not only for Deepseek, but for many other Open source LLMs. I am of the opinion that’s where this LLM race is heading.
From what we see, companies are hesitant handing out their Data to the “cloud”. Open source that allows them to run local instances using tools like GPT4ALL, HuggingFace will end up being the preferred model.
Thanks