의미론적 AI 검색에서 결과를 얻는 데 어려움을 겪고 있어서, 무엇이 문제일 수 있는지 여러분의 의견을 듣고 싶습니다.
간단한 예시로 "shopify"를 검색해 보면 일반 검색 결과는 충분히 나오지만, AI 검색 결과는 전혀 나오지 않습니다. Discourse Meta 커뮤니티에서 "shopify"를 검색하면 일반 검색 결과와 AI 검색 결과가 모두 많이 나오는데, 우리 사이트에서는 그렇지 않습니다. 더 복잡하고 구체적인 질문도 시도해 봤지만, 여전히 결과가 나오지 않습니다.
우리 환경 설정은 다음과 같습니다. LLM 설정에서 gpt-4o-mini를 올바르게 구성해 두었고, “AI 임베딩 의미론적 검색 활성화” 옵션도 켜져 있습니다. 임베딩에는 "text-embedding-ada-002"를 사용 중입니다.
thanks for reporting, the team will have a look!
Thanks Sam! I wanted to make sure I wasn’t missing something obvious here since it seems to be working better on your own site.
The only thing that is jumping to mind is that possibly we are not done backfilling embeddings on your site, we will have a look
Hey @tyler.lamparter,
On a first glance, one issue I’ve found with your site current config is that while you are using text-embedding-ada-002 you’ve filled the embeddings configs for prompts, which are not supported for that model. I removed the instructions you had set there and regenerated the embeddings on your site.
I’m also updating the tooltip on those settings to try to avoid this confusion going forward
https://github.com/discourse/discourse-ai/pull/1121
Other than that, I tried doing a search for “shopify integration” and I got the following hypothetical search document
which is aligned to what we expect.
Can you try searching now and sharing your experience?
@Falco this seems to be working much better, thank you! I had added the prompts under the embedding configuration in an attempt to improve it, but of course it had no effect.
@Falco maybe I spoke too soon. Whenever I search now, AI always has exactly 40 results found regardless of what I search for. Many of the results are also not that relevant (suggesting the about this category topic for example).
Let me try with a different embeddings model. Will report back in ~1h.
What would be the ideal topic result for this query ?
I would expect 0 search results and 0 AI results in that particular case. We support ES6/ECMAScript2015 JS (yes very old), but it’s not in any community post as of yet.
Ohhhhh I see. This won’t work in this case.
The way our current AI search works it:
There is no threshold of distance where we cut off the search, as finding out a general threshold that covers all thousands of Discourse instances is non-trivial. This is discussed at Setting a similarity threshold for semantic search.
We are looking into releasing a new approach that will do a more standard LLM + RAG search and return a conversational response, where you can tweak the prompt to say “no results found”. This is coming in the next weeks, will ping you here when you can test it.
That would be great. That’s what I was attempting to do with adding the prompts into embeddings section. Since you can’t set a threshold now, is that why we’re always seeing about 40 results?
Yes, exactly.
Today, the AI search works like a failover system when the standard search returns no results, acting like an overpowered synonyms dictionary. However, in your case, the search query doesn’t have any possible results in the entire forum, which isn’t the use case it was designed to address.
Stay tuned for the next version of the search; it will be closer to what you want.
Am I good to set our embedding model back to text-embedding-ada-002?
@tyler.lamparter Conversational AI Search coming to Discourse AI 을 확인해 보세요. 사이트에서 테스트해 보고 싶으시면 알려주세요.
네, 저희 사이트에서 이 기능을 테스트해 보고 싶습니다. 귀사의 사이트에서도 테스트를 해 봤고, 전체 검색 모드에서 기존에 있던 “AI로 발견된 50개 결과 숨기기” 토글도 확인했습니다. 이 토글은 여전히 이전의 HyDE 검색을 사용하나요?
새로운 기능은 토글을 전혀 사용하지 않습니다. 이 기능은 표준 검색 경험 위에 대화형 답변을 표시하며, 기존 키워드 검색 및 시맨틱 검색과 함께 제공됩니다.
제가 생각한 바로 그거예요. 새 기능이 우리에게 잘 작동하면 HyDE 토글을 비활성화할 수 있을 거예요.
이 페르소나에 커스텀 도구를 연결할 수 있을까요? 외부 지식 베이스에서 관련 문서를 가져와서 봇과 상호작용하지 않고도 검색 결과에 표시할 수 있는 방법이라고 생각하고 있습니다.
네, Conversational AI Search coming to Discourse AI 에 명시된 대로 모두 수행할 수 있습니다.
