Note: I am not certain if this is a bug in Discourse. I have tried to collect necessary evidence, and so far I have not found anything pointing to our infra/setup. Our configuration is as vanilla as as possible at Tappara.co
- Chat-like rapid discussion topics stop updating automatically. After a delay of 30 - 180 secs the updating usually resumes, revealing the posts that were made during the freeze.
What we know so far
- We did not see this during the previous season, the last game was played in March.
- We run stable branch and did the latest major update in August.
- The issue was immediately reported in the first exhibition games, with moderate traffic/activity.
- This impacts iOS and Android Chrome, but is far less frequent on Chromebook.
- As I write this, I am seeing freezes on my Android phone, while the discussion flows as expected on my Chromebook. Two different devices in the same network.
- The experience varies per user/client. Different users report the freezes at different times. Overall we just recorded roughly 300 messages in about 30 minutes and users reported dozens of freezes. Mostly the freezes seem to correlate with events in the game (goals, penalties).
Things I have tried to rule out
- CloudFlare – we did one game without CF caching, and the issue persisted.
- CPU overload – CPU usage is well within limits, usually hovering around 20-30%.
- Disk exhaustion – Disk I/O seems to be well within limits. We have UpCloud’s MaxIOPS SSD’s.
- I had the Chrome inspector running during the game and some 429’s were recorded, but for me they did not correlate with freezing.
- The end users are not receiving the notifications regarding 429’s (slow down) or extreme load. The updating just freezes, and then just resumes. Has the rate limiter changed recently - I am under the impression that ratelimits should trigger a notice on the UI?
A really nasty problem, that really hurts the play-by-play game chats. We have been running these for years, and I have never seen this before.