Seriously?
I guess I knew I wanted the postgres 18 upgrade, so I guess that’s on me.
AI-generated summary
Users expressed frustration with Discourse’s AI search feature, noting that it often fails to surface relevant recent announcements. pfaffman initially reported difficulty finding the PostgreSQL 18 upgrade announcement, suggesting the search algorithm might overly favor older topics with high engagement metrics rather than recent ones. merefield described the experience as “over-engineered” and suggested that the AI view should not be the default or should display longer result lists.
In response, nat updated the AI search prompts to better value recent announcements and noted that users can toggle off the AI view if preferred. RGJ shared a similar experience where the AI frequently returned outdated or irrelevant results, though occasionally provided excellent answers. nat acknowledged the need for further fine-tuning, such as prioritizing specific categories or high-trust users, while also mentioning ongoing work to improve speed and accuracy over traditional keyword search.
Following these adjustments, users reported improved reliability. merefield confirmed that the AI search now correctly identifies newer upgrades, and pfaffman noted a significant improvement in their ability to find needed information.
10 Likes
Yes, it’s become imho an uncomfortable experience and feels … over-engineered on that first view with so few results.
2 Likes
Pick your poison I suppose, the keyword search for the same query:
Can you elaborate? Over-engineered in that you just want broader results with AI search?
1 Like
Yes, in the AI view I think it might be much more comfortable to have a longer list.
Or maybe the AI view simply shouldn’t be the default?
4 Likes
And none of those relates to the current postgres upgrade, right? So that’s two ways not to find out about the current PG18 upgrade.
This is just what happened today. My experience, which is not to say my careful evaluation of a careful set of tests, or even careful thought about actual searches that I’ve done, is that search never finds anything useful. I’m sure that can’t be true, but that’s how it feels. If I were to hazard a guess, the problem is that search favors results with more likes, reads, and such, which means that the recent announcement that I want won’t show up for a couple more years. Maybe search should favor recent announcements or something like that? Maybe it should somehow have a metric that favors recent stuff like likes per week or recent likes or something?
But since I (perhaps irrationally believe) can’t find stuff here, what I am encouraged to do is to just always go to ask.discourse.org first.
4 Likes
Thanks for sharing your feedback. It’s useful to hear these things to help improve the experience here.
The good thing about “AI search” is that it’s actually much easier to improve results than keyword search, as @awesomerobot demonstrated above.
I’ve updated our prompts (carefully) so that they value more recent announcements more.
If we see any such tweaks needed for meta, please continue to let us know.
At the bottom of the screenshot is a toggle to turn off the AI view, so that’s something you can consider.
Interesting comment.
In an earlier iteration of the Ask AI feature, we’d considered making the number of results a user option instead of a site setting.
10 Likes
Nice! I moved this to General since I don’t think whining isn’t really a Support issue.
Maybe yours is a solution, though since this affects more than me and more than this search, it’s hard to say just yet. This does look promising though!
3 Likes
I had a similar experience earlier today. I was looking for good documentation for post custom fields to give to a client. I just retried it and 80% of the times I get this:
Which is very specific and not a very good answer.
The left topic goes to Insert, Read and Update Post custom fields from plugin which is from 2016 and does not provide an actual response.
that is probably the oldest topic around.
The funny thing is, when I toggle back and forth between “all topics” and “AI” (sometimes getting “Ask AI took too long to complete this search. Try again or use Search”), one of of approximately five times AI gives me an excellent response
and refers to this topic which is probably one of the best possible answers.
3 Likes
Super.
I think it’s on me to update meta’s “search synthesizer” agent, or add some new ability to fine-tune results that don’t require more prompt evals.
FWIW, we did quite a bit of work to make results fast (and ideally better than keyword search).
At the moment, it feels like “prioritise results from <category>” or “higher TL users typically have better answers” could be a good addition.
7 Likes
Typesense is really great; I hope it comes back for community battle testing.
3 Likes
Yeah. So recency might not be such a good suggestion. 
Maybe searching is hard. But it does seem worse in the past 3-12 months
We’ve since made some changes to search on Meta, how’s it been going? better, worse, same?
6 Likes
AI search seems more reliable 
Eg @pfaffman 's query is correctly picking out the newest upgrade.
6 Likes
It seems like I searched for something today and found it! I can’t remember what it was, but the fact that I can’t remember is really, really a good sign!
8 Likes