“Due to extreme load, this is temporarily being shown to everyone as a logged out user would see it” I run a website that features live threads during basketball games and have had issues with the server getting overloaded and showing this message during games (peak activity). Is there any way to identify the best way to address the issue? Is it storage speed? Memory? CPU? Is there something I can do other than beefing up my server, or if I do during these times… what should I beef up?

[image] dylanb: Is there something I can do other than beefing up my server, Have fewer people use the forum? Better still would be to have them spread out their use rather than all doing it during the game. But none of that is helpful. Since you know when the games are, you could beef up your server just during the game, or run multiple servers during the game behind a load balancer.

You need more and faster CPU cores and more Ram, so you can increase the number of web processes (Unicorn Workers) in order to deal with the peak traffic.

It is an area that @sam is actively working on. It does seem that newer versions of Discourse have regressed in this area, performance-wise. That being said, the real solution is to use a live chat tool if you need live real time interactions for hundreds of people at the same time – though I continue to question how and why chat is “useful” when it’s scrolling so fast nobody can really see any of it, ala Twitch and YouTube streams. :man_shrugging:

This thread here describes the issue quite well. [image] Real-time updating of topics freezes under high activity Support Perhaps so, but we would like to be slightly less protective over the server as the naturally occuring activity spikes are very short, and generally stabilize within a minute or so. So adjusting the thresolds just a little bit higher might improve the UX, while waiting for the move. This was my experience with a in match football forum before issues kicked in: Digital Ocean 1 CPU 1 GB = 30 -40 users in chat like situation 2 CPUs and 2 GB= 70-80 users in chat like situation 4 CPUs and 8 GB = fine for 120 users and 1000 posts in 2 hours. Didn’t reach limit. With Hetzner (Mirroring site) My experience was: 3 CPU (CPX 21 AMD chip) and 4Gb = struggling with 20 users 2 CPU (Intel) and 8gb = no issue with 20 users. I never got around to testing with more users. The key thing is to improve the CPU and Ram. AND ALSO edit the app.yml file. Add more unicrons in here, and also alter db_shared_buffers .

[image] codinghorror: That being said, the real solution is to use a live chat tool if you need live real time interactions for hundreds of people at the same time – though I continue to question how and why chat is “useful” when it’s scrolling so fast nobody can really see any of it, ala Twitch and YouTube streams. For sport forums it becomes closer to live text updates for games. People are not so much chatting (although it does occur to an extent, especially at half time), they are instead getting a live text commentary from fellow fans. Many people will log onto the forum, just to read the live text. Seeing the thoughts and opinions of key match events by key people. Ranging from funny reactions to serious analysis. If you missed the game, people log on, simply to read the event by event commentary. It great for people at work :wink: Its more personal/biased than the text commentary from Newspapers or TV channels.

[image] Alec: The key thing is to improve the CPU and Ram. AND ALSO edit the app.yml file. That’s right. discourse-setup will do that if you run it on the upgraded server.

I hear you, it is something @sam @eviltrout and I have been looking at closely .. the “hundreds of users logged in and browsing the same topic at the same time” situation has been coming up a fair bit recently as Discourse becomes more popular. We may need to invite a new mode of behavior when this happens, “fast lane” signs should probably start appearing in the topic UI in some form… :warning: [image]

An important thing to keep in mind here is that “chat” implementations generally broadcast the actual content to the subscribers. In Discourse we have quite a complex pipeline which makes the naive implementation complex that leads to large amounts of traffic. User posts a reply All the users looking at the topic discover via broadcast that there is new content All users ask the server for post content (100 viewers = 100 requests) We pull images / optimize images All the users looking at the topic discover via broadcast that there is new content All users ask the server for post content (100 viewers = 100 requests) (we have various optimisations, rate limits, retries and so on, but that is the gist) These requests all have to run via our security pipeline to ensure the user has the rights to see the post and so on. If the content was short-ish and we could somehow figure out how to do security in a more light-weight way for the “fast lane” then we could distribute the chatty messages via broadcast. This would lead to significantly better performance, we could probably handle 10000 users on a single low 2gb digital ocean droplet with that design. Security is very complex. Caching is also complex cause of cache invalidation issues. So, yes, we are absolutely thinking about this problem. But as it stands … Lots of logged in viewers on one topic + lots of new content on one topic = expensive server bills.

Thats fantastic! In all honesty I only have enough knowledge to be dangerous. I am the type of bloke who learns by playing (and breaking). Fully appreciate the fantastic effort in creating this platform. When I created my first forum 12 months back, I created 12 different versions :laughing: Vanilla, MyBB, SMF, Flarum etc. Discourse was easily the best. One biggest pluses is the active development. Its great to see how you guys listen to the community and continually improve.

[image] Alec: The key thing is to improve the CPU and Ram. AND ALSO edit the app.yml file. Add more unicrons in here, and also alter db_shared_buffers . Do you have recommended settings for this?

Extreme load error

Product Support Self-hosting

Alec (Alec) August 12, 2023, 12:05am 14

I have just done it by hand, should I have run discourse setup ?

Topic		Replies	Views
Real-time updating of topics freezes under high activity Product Support	89	9043	February 26, 2022
Discourse under heavy loads Product Support	4	1595	June 26, 2015
Improving Instance Performance (Megatopics, Database Size and Extreme Load) Self-hosting	60	6830	October 13, 2020
Best configurations for speeding up standalone discourse Self-hosting server-resources	25	4748	February 14, 2024
Our discourse overload because of a custom plugin. Please help me Self-hosting hosting	46	2746	August 22, 2022

Extreme load error

Related topics