Search “ost.io” and you won’t find this thread:
https://meta.discourse.org/t/can-discourse-do-what-ost-io-does/11737
Which has ost.io in both the title and the post body.
AI-generated summary
The term “ost.io” is causing trouble for search on Discourse. erlend_sh noticed that searching for “ost.io” doesn’t yield a specific thread despite the term being in the title and post body. Hector reported similar issues with other URLs ending in “.io”. erlend_sh found that searching for “ost.io” does return some results, but not the expected ones.
Hector discovered that the issue only occurs when the term is part of a link, and sam attributed the problem to the snowball stemming algorithm and default English tokenization used by Discourse.
codinghorror suggested that quoted exact term search would be beneficial, and AmiAthena echoed this sentiment, sharing their own struggles with searching for individual words.
erlend_sh added another use case where this issue is detrimental, particularly in technical forums where searching for specific versions is common.
sam fixed the bug and added rudimentary support for exact term matching. The fix is now live on meta.discourse.org, as demonstrated by sam with two example searches.
1 Like
It seems it can’t find anything ended in io, eg: astronomers.io, popcorntime.io. Also others like moter8.pw. And sometimes even part of the url: zenhub or bughubapp.
1 Like
Well the odd thing is that when I search for “ost.io” it’ll turn up results for this topic and another one where the term is used, but not that older topic where it was first mentioned.
If it’s not a link it works. Notice that all the terms I entered can be found in this topic. In the other use of Ost.io that it finds (the one with the capital O) the link does not include the dot (https://changelog.com/ ostio-add-a-forum-to-your-github-project/
2 Likes
I really think snowball and the default english tokanization pg uses creates lots of issues.
Is there a place we could forward this issue to then?
One thing we are direly missing is quoted exact term search e.g. “Manifold carburetor” should match only that string in that order.
6 Likes
In case that functionality isn’t included in what you said: I’d also like to have that for individual words. I tried to search for “top” and got soooo many hits for “topic,” and while it’s often quite helpful to get results for permutations of the word, in this case it rendered the search pretty much useless.
6 Likes
Dropping by to add another use case that’s hurt by this:
Having forgotten this bug, I just dropped by talk.jekyllrb.com and searched “3.0 eta”, looking for a topic I knew already existed. This search garnered no results of course, while “eta” got me exactly what I wanted.
The fact that you actually lose search results by including an invalid search term can be pretty detrimental on technical forums where searching for specific versions is not uncommon.
4 Likes
A bug indeed,
I just fixed it:
https://github.com/discourse/discourse/commit/139314c2133fb74a9e94ffbdc92f3551de279bea
Also added some rudimentary support for exact term matching.
Will take a bit till it is live here, cause rubygems is having a minor freakout at the moment.
5 Likes