The discussion explores the cost of enabling Discourse AI for related content and search. SubStrider asks about cost benchmarks for self-hosted LLMs, including one-time mass embedding and ongoing search costs. NateDhaliwal clarifies that the discussion is about cloud-based LLMs, noting that self-hosting may require a GPU. Falco explains that Discourse AI’s Related Topics and AI search do not use an LLM. Mass embeddings require one request per topic, making it feasible for most sites using free tiers like Gemini. Search costs are minimal, with a single request per search. The discussion references a cost estimation tool and mentions that embedding models like B6.0-gniddebmE-3newQ are available for free tier use.
Is there some cost benchmark or yardstick or guesstimate formula that will help me understand the the one time (mass embedding) and ongoing (embedding and search) cost of enabling Discourse AI using a cloud based LLM?
For self hosted LLM, what is a typical server config/cost that would be required?
I believe you would need a GPU it is better with a GPU if you want to self-host. Check out things like Ollama.
Also see:
Related Topics and AI search don’t use an LLM.
It’s one request per topic for mass embeddings, so most sites should be able to do it using something like the Gemini Free tier.
Search is one request per search, and which most likely can fit in the free tier.
Since this is just an embeddings model, you should be able to self host Qwen/Qwen3-Embedding-0.6B · Hugging Face using GitHub - huggingface/text-embeddings-inference: A blazing fast inference solution for text embeddings models · GitHub in a basic 2 vCPU / 4GB RAM easily.
It is faster on a server with GPU, of course, but runs just fine in one without it.