Search query times out for big sites

Dev
1

Hey guys,

We’re started seeing timeouts regularly on our /search query, we’ve benchmarked some search terms and some queries might take up to 2mins to complete. We found this post but were wondering if they are any additional recommendations on how to deal with this. Some relevant information related to our particular case:

  • Our biggest site (the one that’s timing out more often) has about 1,700,000 posts.
  • We are using some default postgresql autovacuum settings. We gathered some stats around the pertinent tables: categories , posts , topics , post_search_data , users and they seemed to get vacuumed about once a day so we do not think this has to do with dead rows slowing down our search.
  • We’ve manually run the Jobs::ReindexSearch.new({}) but did not see any considerable improvement

We’re considering running the rake search:reindex on that site which we know already will take quite a while but we are wondering if you have any other suggestion?

Thank you in advance

1 Like
2

One quick and dirty way to get faster search is using the settings search prefer recent posts and search recent posts size. That works very well by prioritizing recent posts first to fill search results.

Other than that you would need to do some analysis of the database performance. Check plans, costs, etc. With larger forums you will need more RAM to keep up performance (and tweak PostgreSQL settings accordingly).

If everything else fails you can also try Add Algolia search to your Discourse

9 Likes
3

Hey Falco,

Thanks for your suggestion, the search prefer recent posts trick definitely helped out and searches are working fine for the most part. We suspect that some of our indexes might be corrupted so we’re starting to look into that too but this helps tons to stop the bleeding for now.

5 Likes