We’ve had a recent influx of users and traffic surge, and with that, more searches performed simultaneously. We’re starting to see search timeouts across about 1.2 million posts. What are you larger forums doing Search-wise? Have you moved outside of core Discourse search feature or is it still p…

What are your server specs and what resources seem most constrained? I’m looking into potentially moving a board with about 20 million posts (public posts + private conversations) so I’m definitely interested in learning about potential performance issues with large boards.

It’s a long way from 1M posts to 20M posts, so there’s little advice that will apply to both of you. FWIW, I’ve got a site with 9M posts that has acceptable performance on a i7-7500U @ 2.7Ghz with 16GB of ram. There are a few other low volume sites on the same server. A site using AWS database wit…

[image] Search query times out for big sites Development One quick and dirty way to get faster search is using the settings search prefer recent posts and search recent posts size. That works very well by prioritizing recent posts first to fill search resu…

[image] pfaffman: what are your server specs? We’re on AWS. Forums are running on 4 instances, I believe t3.larges, so 4 vcpus and 16 GB ram. We’re running Kubernetes on a custom docker image. Our DB is on it’s own instance, m4.large, so 2 vcpus and 8 GB ram. The ram on the db instances c…

[image] Jumanji: Our DB is on it’s own instance, m4.large, so 2 vcpus and 8 GB ram Os your total database size less than 8GB? You will want to fit as much possible of your data in RAM. [image] Jumanji: Sort Method: external merge Disk: 77632kB Oh this will be slow. And a lot worse if …

@Jumanji Could you please share some high traffic forums sites running on discourse, links please? Traffic over millions.

I’m not part of the Discourse team, but I found this using search. [image] Listing of all discourse forums Community Building I’m sure, I suspect there are way more than reported by either of those sites. Maybe as much as 10x more perhaps.

[image] Falco: Os your total database size less than 8GB? We’re at 24 GB. Definitely larger than the RAM available to the instance.

Thanks for that. Yup, we ran than and are on RDS.

Large forums with 1+ MM posts, how's your Search performing?

Support Self-hosting

sam (Sam Saffron) May 12, 2020, 8:04am 10

In some cases after massive moves running vacuum analyze can improve performance.

Also if you are on AWS are you using RDS? If not… why not?

Topic		Replies	Views
Improving Instance Performance (Megatopics, Database Size and Extreme Load) Self-hosting	60	6552	October 13, 2020
Discourse installation has been getting slower and slower and slower Self-hosting server-resources	36	2139	April 15, 2023
Search query times out for big sites Development	2	1419	January 13, 2020
Slow Profile Loads with 100GB+ database Feature	42	3537	December 29, 2020
Are there any commands that will speed up the site? Support	19	2558	December 2, 2020

Large forums with 1+ MM posts, how's your Search performing?

Related topics