Oh, I see now, thanks for the explanation, that’s what I was missing.
Btw, I know often there’s reranking in more advances RAG. Is there something like reranking in how Discourse handles is? Do you think adding a reranking step would have any positive effect?
reranking is a work-in-progress.
@Falco / @Roman implemented a basic reranker in discourse:
This is used in semantic search and RAG. However it is quite hidden and not easy to configure.
I think the medium term plan here (which we discussed with @awesomerobot ) was to move from LLM terminology to Models … and maybe do a bit of UI unification so you can define embedding/rerankers and llms in a single interface.
For now we only support a very specific hugging face reranker api.
It certainly improves quality of results.
Awesome!
So currently this is off by default, and can’t be configured easily on a hosted (pro) plan?
not sure, lets wait for @Falco to answer.
Hi @Falco
is there any way to enable the reranker for testing or maybe a timeline?
We just deployed a big improvement to the underlying tech that powers semantic search in Discourse in DEV: Re-introduce PG Vector 0.8.0 upgrade by romanrizzi · Pull Request #35233 · discourse/discourse · GitHub.
Can you retry your use case now, it’s already deployed to your site. My expectation is that this will make RAG better without the need for a reranker.
We still want to make the re-ranker widely available, but we are waiting for some upstream changes to land first.