I have set up the LLM, embedding, and persona features; however, my prompt is not providing the expected responses from the questions and answers or the available wiki resources within our community.
Self-Hosted LLM: mistral:latest
Self-Hosted Embedding: all-minilm:latest
Persona:
Could anyone help me further with what needs to be done?
What exact Mistral model is this?
Can you share the prompt?
You can add Search as a “Forced Tool” for the first interaction to help ground the model.
@Falco, The below are the details for tailing reply:
What exact Mistral model is this?
LLM Model : mistral:latest
Link:
The model at https://ollama.com/library/mistral:latest is:
Mistral 7B v0.3 — a 7 billion parameter open-source model released by Mistral AI.
Can you share the prompt?
System prompt:
You are a community knowledge assistant designed for this forum called {site_title} and with site URL {site_url}, having engineers as users.
Always search and reference relevant forum posts, wiki articles, and tagged discussions before generating an answer.
Your first priority is to use retrieved forum content (via embeddings search) to craft responses.
Prefer summaries and citations from existing posts.
If multiple related topics are found, combine them clearly.
Only if no relevant content exists, respond using your general knowledge through the LLM.
Include topic titles or URLs when referencing posts.
Never hallucinate or invent answers not supported by forum data.
Be factual, concise, and professional.
When users ask broad questions, prefer summarizing multiple sources rather than guessing.
Always prefer context from categories, tags, and wikis indexed in embeddings.
Updated the forced Tool as suggested
I’m afraid a rehashed 2023 model won’t make the cut here. Also, from Ollama own documentation for this model, it only supports tool calling on the raw API, which is not what we use.
Overall, this is a poor choice for AI Bot today.
Instead, use one of the following