Enable AI Search

documentation admins
, ,
1

This topic covers the configuration of AI Search feature from the Embeddings module of the Discourse AI plugin.

Overview

Similar to Related Topics, AI Search helps you find the most relevant topics using semantic textual similarity that are beyond an exact keyword match used by traditional search. This results in the discovery of topics that are non-exact matches but still relevant to the initial search. If you can’t find what you’re looking for, AI search is here to help!

The following is an example, note: the current search is about “AI Search”
DiscourseAISearch

Features

  • Semantic textual similarity: going beyond just a keyword match and using semantic analysis to find textual similarity
  • Toggled in full-page search
  • Applicable to both anonymous and logged-in users

Availability

Currently, we are offering this feature for our self-hosted and hosted by Discourse customers on the Enterprise plan.

Enabling AI Search

Prerequisites

In order to use AI Search you will need a provider for Embeddings and a provider for Large Language Model (LLM), both of which are required.

Embeddings

Follow the guide at Enable Related Topics

Large Language Model (LLM)

Currently, both hosted by Discourse and self-hosted customers will need one of the options below:

  • OpenAI
  • Anthropic
  • Azure OpenAI
  • AWS Bedrock with Anthropic access
  • HuggingFace Endpoints with Llama2-like model
  • Run your own OSS Llama2-like model with TGI

Configuration

The following instructions would apply to any Discourse instance (hosted/self-hosted)

  1. Go to Admin settings-> Plugins → search or find discourse-ai and make sure it’s enabled
  2. Enable ai_embeddings_enabled for the Embeddings module needed for AI Search
  3. Enable ai_embeddings_semantic_search_enabled to activate the AI Search feature

Technical FAQ

Architecture Diagram 
sequenceDiagram
    User->>+Discourse: Search "gamification" 
    Discourse->>+LLM: Create an article about "gamification" in a forum about<br>  "Discourse, an open source Internet forum system."
    LLM->>+Discourse: Gamification involves applying game design elements like<br> points, badges, levels, and leaderboards to non-game contexts...
    Discourse->>+EmbeddingsAPI: Generate Embeddings for "Gamification involves applying game design..."
    EmbeddingsAPI->>+Discourse: [0.123, -0.321...]
    Discourse->>+PostgreSQL: Give me the nearest topics for [0.123, -0.321...]
    PostgreSQL->>+Discourse: Topics: [1, 5, 10, 50]
    Discourse->>+User: Topics: [1, 5, 10, 50]
  • How does AI Search work?
    • The initial search query is run through an LLM which creates a hypothetical topic/post. Afterwards, Embeddings is done on that post and then it searches your site for similar matches to the search query.
  • How is topic/post data processed?
    • Hosted by Discourse: LLM data is processed by a 3rd party provider, please refer to your specific provider for more details. By default, the Embeddings microservice is ran alongside other servers that host your existing forums. There is no third party involved here, and that specific information never leaves your internal network in our virtual private datacenter.
  • Where does the data go?
    • Hosted by Discourse/Self-hosted: Hypothetical topic/post created by the LLM provider is temporarily cached alongside the Embeddings for that document. Embeddings data is stored in the same database where we store your topics, posts and users, It’s another data table in there.
  • What does the Embeddings “semantic model” look like? How was it “trained”, and is there a way to test that it can accurately apply to the topics on our “specialized” communities?
    • Hosted by Discourse: By default we use and recommend this model. We have this deployed to many customers, and found that it performs well for both niche and general communities. If the performance isn’t good enough for your use case, we have more complex models ready to go, but in our experience, the default option is a solid choice

Caveats

  • AI Search does not always find topics with 100% relevancy
2 Likes