I’ve been getting reports of this warning message from my members. Doesn’t come up much but it is being noticed. Can I assume there is nothing to do here and it’s not as serious as it appears? “Extreme load” sounds like a huge problem. But it does go away almost immediately and search works again.
I’d appreciate any guidance on troubleshooting steps. Thanks! 
site is under extreme load, search is disabled, try again later
I’m not so sure it’s something that needs troubleshooting. But maybe it could be handled differently?
You should be concerned, I would look at your web crawler report in the admin dashboard.
That will be triggered when your web requests are coming faster than your server can handle.
For example, if you have 4 unicorns and, on average, need 200ms to fully process a request, your capacity is 20 requests per second.
If, for a moment, you get 40 requests per second, the extra requests will sit in a queue waiting, and that message will be show. Also, you can’t go blindly setting up more unicorns, as the process to fully answer a web request involves a lot of moving parts that should be carefully evaluated.
Discourse gained a lot of those graceful degradation features recently as we work to scale out and handle some crazy level of traffic for the largest Discourse instances on the internet.
Looks like we did get an increase in web crawler page views in recent days. But I’m not sure what if anything there is I can or should do about it. Personally, I am ok with a notice to members from time to time when the server is under higher than normal load, preventing searching temporarily. But maybe the “extreme load” messaging could be moderated a bit, to say “server is busy”.
Blacklist the rogue crawler user-agent in site settings. There are a lot of really bad crawlers on the internet these days.
Hmm, only 1800 pageviews in 24 hours by a crawler should not put that much extra load on your server though. It might be worth opening your access log and just watch the requests come in when this happens, it could point to the real issue. It could be someone trying to hack you, or hammering your API with requests.
I Just got a report of another similar looking notification. We are doing an AMA and have a few more people than usual reading and posting in the forum, but I am not convinced that this is extreme load. Maybe a plugin like Discourse Who's Online Plugin is causing additional load?
Thanks! I will look into this now.
Is there a way to disable these in an admin setting and just let people wait if it is getting slow? I think our site is still not especially busy.
If performance degrades to the point of timeouts it won’t feel slow, it will just feel broken.
Ignoring these warnings is definitely not a good idea!
I’m getting this extreme load notification a lot recently.
I have a sports forum and it’s usually when there are people commenting on the one topic about a particular match at the same time. Is there any guide or help on what I can check to look at logs or the server to see if it’s up to speed. It wasn’t a problem until the last few weeks.
Do you think we could have regressed on logged-in user posting performance @sam?
I am seeing an N+1 on meta for sure that we need to fix ASAP
Will log some items to get this sorted. That said, once site is overloaded admins have the ability to tweak 2 things.
You can give the instance more resources, (more memory for PG or more unicorn workers)
You can adjust the anon threshold per:
DISCOURSE_FORCE_ANONYMOUS_MIN_QUEUE_SECONDS and DISCOURSE_FORCE_ANONYMOUS_MIN_PER_10_SECONDS
I did fix one N+1 in the discourse-voting plugin here:
https://github.com/discourse/discourse-voting/commit/9b4f233b9d33a946f626597aaf3863d2305b8809
Will see if anything else yells out
Found another N+2 here:
https://github.com/discourse/discourse/commit/accbbded15974846523c3aeca85a59454e3f2f05
Looking to see if anything more pops up.
Q: What is an “N+1” ? Although i’m a dev (java), i don’t think i ever came upon this term before.
Here is a (more or less Java-related) article that explains this issue:
