For the upcoming 2.3 release, we’ve added a new feature and made several fixes to the way we index posts for search that will make search results better.
1. Search Priorities for Categories
Each individual category can be configured with a search priority which you can find under the
Settings tab when creating/editing a category. Five levels for prioritization have been added: ignore, very low, low, high and very high. The levels work by multiplying a pre-configured weight to the search ranking for each result and can be configured via hidden site settings in your console.
As an example, configuring a category’s search priority to very_high will boost its search ranking by 40% while a search priority of very_low will reduce its search ranking by 40%. Setting a category’s search priority to ignore will remove it from the search results. However, you can still search for posts within the ignored category by scoping the search to the category via advanced search. Do take note that search priority is not inherited which means that a sub-category will still be searchable even if the parent category has been configured to be ignored.
2. Improvements and Fixes to Search Results
-
Search relevance has been updated to consider the document length in the ranking. Previously, search ranking was ranked by the most number of matches for the given search term. This was problematic for search results as we noticed that longer posts were more likely to be ranked higher because of the tendency to produce a higher number of matches. As a result, we’ve made the switch to consider the document length when ranking search results.
https://github.com/discourse/discourse/commit/e87ca594014733d171df0112149f49180d898678 -
Improve the quality of the raw data that we use to generate the search index. https://github.com/discourse/discourse/commit/cfd507822f9330967f3ed9f970505e7f4896b523
- URLs in posts were sometimes incorrectly tokenized and indexed twice leading to terms within links having a higher ranking.
- Content for lightboxes in posts were polluting search results with the image’s metadata.
-
Empty posts such as moderator actions or small post actions (closing, assigning) are no longer part of the search index. This change leads to a smaller index and less noise in search results. https://github.com/discourse/discourse/commit/daeda80ada1d914d52dcc53a2cda087e5f62330c
-
Smaller search index by deleting posts of trashed topics from the index. https://github.com/discourse/discourse/commit/d151425353cb4a99ee9cc543ed75925d9c3edc17
-
When a post is moved to a different topic, the search data for the post was not updated and would lead to the post incorrectly appearing in searches that matched the category of the old topic. https://github.com/discourse/discourse/commit/d808f36fc407a201b79755d9e48e7682e8407edd
-
Smaller payload for search results by excluding the cooked version of the posts. https://github.com/discourse/discourse/commit/03c6b22c626c6c91e009020e40817751d1421324
-
The blurb for posts in search results was broken when searching for an exact phrase and lead to missing search term highlights on the client side. https://github.com/discourse/discourse/commit/dae0bb4c67c9f7f1eb475fb60f375118f93cdfd9
Do let us know if you’re search results have improved after these changes. We would also like to know if you think your search results have become worse so that we can continue to refine it. Thank you all!