Discourse AI - AI search

[Discourse]

This guide explains how to enable and configure the AI search feature, which is part of the Discourse AI plugin.

Required user level: Administrator

Similar to Related topics, AI search helps you find the most relevant topics using semantic textual similarity that are beyond an exact keyword match used by traditional search. This results in the discovery of topics that are non-exact matches but still relevant to the initial search. If you can’t find what you’re looking for, AI search is here to help!

Features

• Semantic textual similarity: going beyond just a keyword match and using semantic analysis to find textual similarity
• AI quick search: automatically adds AI results in the search menu popup when few regular results are found (enable with ai_embeddings_semantic_quick_search_enabled)
• Toggled on/off for AI search in full-page search
• Optional HyDE (Hypothetical Document Embeddings): uses an LLM to expand queries for better results
• Results indicated by icon
• Applicable to both anonymous and logged-in users

Enabling AI Search

Prerequisites

To use AI Search you will need Embeddings configured. A Large Language Model (LLM) is optionally needed if you enable HyDE (Hypothetical Document Embeddings) for improved search quality.

Embeddings

If you are on our hosting, we will provide a default option. For self-hosters follow the guide at Discourse AI - Embeddings

Large Language Model (LLM) (optional — for HyDE)

An LLM is only required if you enable the ai_embeddings_semantic_search_use_hyde setting, which uses an LLM to create a hypothetical document from the search query before embedding it. This can improve result quality but adds latency and cost.

To get started you can configure them through the Discourse AI - Large Language Model (LLM) settings page.

• OpenAI
• Anthropic
• Azure OpenAI
• AWS Bedrock with Anthropic access
• Self-Hosting an OpenSource LLM for DiscourseAI
• Google Gemini

Configuration

1. Go to Admin → Plugins → Discourse AI → Features → Embeddings to find all AI search settings
2. Enable ai_embeddings_enabled for Embeddings
3. Enable ai_embeddings_semantic_search_enabled to activate AI search on the full-page search
4. Optionally enable ai_embeddings_semantic_quick_search_enabled to add AI results in the search menu popup
5. Optionally enable ai_embeddings_semantic_search_use_hyde to use HyDE for improved results (requires an LLM)

Technical FAQ

Expand for an outline of the AI search logic (with HyDE enabled)

mermaid height=255,auto
sequenceDiagram
    User->>+Discourse: Search "gamification" 
    Discourse->>+LLM: Create an article about "gamification" in a forum about<br>  "Discourse, an open source Internet forum system."
    LLM->>+Discourse: Gamification involves applying game design elements like<br> points, badges, levels, and leaderboards to non-game contexts...
    Discourse->>+EmbeddingsAPI: Generate Embeddings for "Gamification involves applying game design..."
    EmbeddingsAPI->>+Discourse: [0.123, -0.321...]
    Discourse->>+PostgreSQL: Give me the nearest topics for [0.123, -0.321...]
    PostgreSQL->>+Discourse: Topics: [1, 5, 10, 50]
    Discourse->>+User: Topics: [1, 5, 10, 50]

How does AI Search work?

• When HyDE is enabled (ai_embeddings_semantic_search_use_hyde), the search query is run through an LLM which creates a hypothetical topic/post. Embeddings are then generated from that hypothetical post and used to search your site for similar matches. When HyDE is disabled (the default), the search query is directly embedded and used for similarity matching. In both cases, the results are merged with regular search results using Reciprocal Rank Fusion (RRF) to re-rank the top results.

How is topic/post data processed?

• When HyDE is enabled, LLM data is processed by a 3rd party provider; please refer to your specific provider for more details. By default, the Embeddings microservice is ran alongside other servers that host your existing forums. There is no third party involved here, and that specific information never leaves your internal network in our virtual private datacenter.

Where does the data go?

• When HyDE is enabled, a hypothetical topic/post created by the LLM provider is temporarily cached alongside the Embeddings for that document. Embeddings data is stored in the same database where we store your topics, posts and users, It’s another data table in there.

What does the Embeddings “semantic model” look like? How was it “trained”, and is there a way to test that it can accurately apply to the topics on our “specialized” communities?

• By default we use pre-trained open source models, such as this one. We have deployed to many customers, and found that it performs well for both niche and general communities. If the performance isn’t good enough for your use case, we have more complex models ready to go, but in our experience, the default option is a solid choice.

Last edited by @Saif 2025-02-13T19:43:12Z

Last checked by @hugh 2024-08-06T04:44:33Z

Check document

Perform check on document:

[RBoy]

I noticed a minor UI bug for ai embeddings semantic search hyde model. Steps to replicate

1. Install AI Discourse plugin
2. Open settings → Configure Gemini key
3. Enable i embeddings semantic search enabled
4. ai embeddings semantic search hyde model shows Google - gemini-pro (not configured)

The not configured doesn’t go away until after all the configurations are enabled and the page is refreshed thereafter.

[Saif (Saif Murtaza)]

@RBoy

The not configured doesn’t go away until after all the configurations are enabled and the page is refreshed thereafter.

I think this is a limitation of our site settings page so apologies for that and glad you were able to get it sorted out

[RBoy]

A question about semantics. In some AI modules I see a reference to using Gemini while in others I see a reference to Gemini-Pro. Are these referring to different models (Gemini Nano, Pro and Ultra) or do they refer to the same LLM? If so then what does Gemini itself refer to and does it matter if one has a paid or a free subscription to Gemini?

[Saif (Saif Murtaza)]

@RBoy

A question about semantics. In some AI modules I see a reference to using Gemini while in others I see a reference to Gemini-Pro.

There are different Gemini models such as the ones you’ve pointed out. Depending on the one you have (likely to pro since its free right now) you would just plugin the API key in the relevant setting. The setting is for whatever Gemini model you have

@RBoy

does it matter if one has a paid or a free subscription to Gemini?

This would depend on you and how you want to use Gemini, but either should work

More on this here

Configure API Keys for Google Gemini

This guide explains how to configure API keys for Google Gemini to enable Discourse AI features that require 3rd party LLM keys. Required user level: Administrator Google Gemini In this example, we are using Google Gemini to generate the keys. In order to use certain AI feature, users must have an API Key from a 3rd party provider such as OpenAI (ChatGPT), Anthropic (Claude), Stability AI (Stable Diffusion), etc. Please check each AI …

[smithmarkl (Mark L. Smith)]

Is there a way that I can inject something into the initial prompt to the LLM or manipulate the embeddings before they are passed off to the regular search?

I’m trying to boost (or possibly split out) staff responses (more authoritative) from community responses.

[Saif (Saif Murtaza)]

@smithmarkl

Is there a way that I can inject something into the initial prompt to the LLM or manipulate the embeddings before they are passed off to the regular search?

Currently, we don’t support any custom LLM prompt injection within AI Search, though this might be something to add in the future.

I would recommend making a Feature request!

[markschmucker (Mark S)]

Just started playing with AI search, and liking it so far. I can’t find a reason not to always use it. Is there a way to only show AI results (other than using css to hide .fps-result but not .ai-result)?

Also can you please elaborate on where to find quick search?

@Discourse

I see an option for AI from quick search?

• The AI quick search option performs AI search faster by skipping creating the hypothetical post. Sometimes this option is faster and provides more relevant results, other times it falls short.

[Saif (Saif Murtaza)]

@markschmucker

Just started playing with AI search, and liking it so far. I can’t find a reason not to always use it. Is there a way to only show AI results (other than using css to hide .fps-result but not .ai-result)?

Also can you please elaborate on where to find quick search?

Hey that was temporarily available exclusively on Meta as an experiment. I have removed it from the OP - thank you for pointing this out

[James_Oh]

We are using Discourse Algolia Search for autocomplete search in the header. Can we continue with using Discourse AI - AI search?