Use Semantic Search for related topics

[fbpbdmin (deepblue)]

a question about AI-enabled related topic list

in Disourse AI plugin, there is a setting
Use Semantic Search for related topics.

when turn this on, on the bottom of each topic, it will have a list of related topics,
a question on this, does the system sends a request to the AI API every time when someone opens a topic to view, to get a list of related topics ?
if it works this way, this seems a lot traffic going out to AI server…

thanks…

saw Falco’s post here, seems it does… ?

Differences in search latency between AI semantic and keyword search

Can you expand on what you mean by latency here? For Related Topics, since every embeddings is pre-calculated there is not extra runtime cost. Much to the contrary, finding related topics SQL query is faster than our old suggested topics query, and we cache related topics for even faster performance. As for AI Search, our current HyDE[1] approach to it incurs in serious latency, which is why it happens async and the user is presented first with the standard search and the option to augment it…

[Falco]

@fbpbdmin

a question on this, does the system sends a request to the AI API every time when someone opens a topic to view, to get a list of related topics ?

No, that is not how it works.

You can read a blurb I wrote about how it works in Discourse AI - Related Topics: Enhancing Content Exploration

[fbpbdmin (deepblue)]

oh, thanks, it is handled on the discourse server side with the database pgvector, no extra burden on the AI server… thanks for the info…