Oh, I see now, thanks for the explanation, that’s what I was missing.
Btw, I know often there’s reranking in more advances RAG. Is there something like reranking in how Discourse handles is? Do you think adding a reranking step would have any positive effect?
reranking is a work-in-progress.
@Falco / @Roman implemented a basic reranker in discourse:
This is used in semantic search and RAG. However it is quite hidden and not easy to configure.
I think the medium term plan here (which we discussed with @awesomerobot ) was to move from LLM terminology to Models … and maybe do a bit of UI unification so you can define embedding/rerankers and llms in a single interface.
For now we only support a very specific hugging face reranker api.
It certainly improves quality of results.
נהדר!
אז כרגע זה כבוי כברירת מחדל, ולא ניתן להגדרה בקלות בתוכנית אירוח (pro)?
לא בטוח, בואו נחכה ש-@Falco יענה.
היי @Falco
האם יש דרך להפעיל את ה-reranker לבדיקה או אולי ציר זמן?
We just deployed a big improvement to the underlying tech that powers semantic search in Discourse in DEV: Re-introduce PG Vector 0.8.0 upgrade by romanrizzi · Pull Request #35233 · discourse/discourse · GitHub.
Can you retry your use case now, it’s already deployed to your site. My expectation is that this will make RAG better without the need for a reranker.
We still want to make the re-ranker widely available, but we are waiting for some upstream changes to land first.
Thanks Falco!
Will this work only for the semantic search over Discourse topics, or also for RAG documents uploaded to a persona? From my own experience, the AI works quite well with forum topics (i.e. on ask.discourse), but not that well with uploaded docs to a persona (or I haven’t found the best formula yet).
It affects all uses of embeddings in Discourse, including Related Topics, RAG, Search, Composer category and tag suggestions, etc.
Hey @tpetrov, did the results improve with the new embeddings model?
Hey Falco, I’m sorry, I haven’t had time to test extensively, so cannot really say.
I guess there’s no way to sitch between the two to compare?
Not now, the old and new models were available for a couple of months, but we recently retired the old ones in our hosting.