I’m having a hard time getting the semantic AI search to give me any results so I wanted to check with everyone and see if you all have any ideas on what could be wrong. If I take a simple example and search for “shopify”, you can see I get plenty of normal results, but no AI results. When I compare this to searching for “shopify” in Discourse Meta community, you can see I get many normal results and many AI results. I’ve tried more complex and specific questions, but I still end up with no results. As for our setup, we are using gpt-4o-mini and that is correctly setup under LLMs. “AI embeddings semantic search enabled” is enabled. We are using “text-embedding-ada-002” for the embeddings. [The image shows a search results page for the term "shopify" on a website, displaying various posts and a message indicating no matching topics found, with topics sorted by relevance. (Captioned by AI)] [This image shows a Google search results page for 'hopify', listing various topics and posts about Shopify integration in Discourse, including job postings. (Captioned by AI)]

Hey @tyler.lamparter, On a first glance, one issue I’ve found with your site current config is that while you are using text-embedding-ada-002 you’ve filled the embeddings configs for prompts, which are not supported for that model. I removed the instructions you had set there and regenerated the embeddings on your site. I’m also updating the tooltip on those settings to try to avoid this confusion going forward https://github.com/discourse/discourse-ai/pull/1121 Other than that, I tried doing a search for “shopify integration” and I got the following hypothetical search document which is aligned to what we expect. Can you try searching now and sharing your experience?

Thanks Sam! I wanted to make sure I wasn’t missing something obvious here since it seems to be working better on your own site.

The only thing that is jumping to mind is that possibly we are not done backfilling embeddings on your site, we will have a look

@Falco this seems to be working much better, thank you! I had added the prompts under the embedding configuration in an attempt to improve it, but of course it had no effect.

@Falco maybe I spoke too soon. Whenever I search now, AI always has exactly 40 results found regardless of what I search for. Many of the results are also not that relevant (suggesting the about this category topic for example). [The image shows a search query result for "what javascript version does celigo use" with three topics and a drop-down menu for advanced filters. (Captioned by AI)] [The image shows a search bar with the term "shopify" typed in, indicating that 50+ results have been found and the user has the option to filter the search using advanced options. (Captioned by AI)] [The image shows a search results page with a query for " how to connect to netsuite jdbc?" on a dark-themed interface. (Captioned by AI)]

Let me try with a different embeddings model. Will report back in ~1h.

[image] Tyler Lamparter: [The image shows a search query result for "what javascript version does celigo use" with three topics and a drop-down menu for advanced filters. (Captioned by AI)] What would be the ideal topic result for this query ?

I would expect 0 search results and 0 AI results in that particular case. We support ES6/ECMAScript2015 JS (yes very old), but it’s not in any community post as of yet.

[image] Tyler Lamparter: I would expect 0 search results and 0 AI results in that particular case. Ohhhhh I see. This won’t work in this case. The way our current AI search works it: Takes user input Creates a new post about it taking into account the forum description Returns the most semantically similar topics to it There is no threshold of distance where we cut off the search, as finding out a general threshold that covers all thousands of Discourse instances is non-trivial. This is discussed at Setting a similarity threshold for semantic search. We are looking into releasing a new approach that will do a more standard LLM + RAG search and return a conversational response, where you can tweak the prompt to say “no results found”. This is coming in the next weeks, will ping you here when you can test it.

That would be great. That’s what I was attempting to do with adding the prompts into embeddings section. Since you can’t set a threshold now, is that why we’re always seeing about 40 results?

[image] Tyler Lamparter: Since you can’t set a threshold now, is that why we’re always seeing about 40 results? Yes, exactly. Today, the AI search works like a failover system when the standard search returns no results, acting like an overpowered synonyms dictionary. However, in your case, the search query doesn’t have any possible results in the entire forum, which isn’t the use case it was designed to address. Stay tuned for the next version of the search; it will be closer to what you want.

Getting a lot of no results for semantic search

Product Support

tyler.lamparter (Tyler Lamparter) February 15, 2025, 6:48pm 16

Am I good to set our embedding model back to text-embedding-ada-002?

Topic		Replies	Views
Discourse AI - Semantic Search & Summarization Not Working for our configuration Product Support ai , ai-summarize	5	429	September 17, 2025
AI Embeddings Semantic Search returns 500 error Self-hosting ai , ai-search	5	331	July 24, 2026
Is it possible to make the default search Semantic search on the site? How much do these calls cost? Feature ai , ai-search	6	1538	September 8, 2023
Support for Vanilla Embedding Search Feature ai , ai-search	2	203	June 28, 2025
Discourse AI - AI search Site Management how-to , ai , ai-search	9	3902	July 2, 2025

Getting a lot of no results for semantic search

Related topics