Maker Forums has a large user base, but rarely has enough users simultaneously online to collapse the Who’s Online avatar display. We found that the Who’s Online plugin was responsible for sluggish performance, even to the point of timing out requests, even with only a couple online users. We use an external nginx config with an offline page and we were periodically displaying the maintenance page when no maintenance was underway as a result of those timed-out requests.
We do serve images locally after migrating off of digital ocean spaces, which creates competition for unicorn workers. However, we’re running everything on a 2 CPU 4GB VM, and we have a relatively large database (about 400K posts, 6GB database) so we don’t have space to just add lots more unicorns at half a gigabyte each. I suspect that if we were using object storage for images, the overall impact would be smaller due to less competition for unicorn workers, but it wouldn’t take much for it to be still significant, given that we saw this impact currently with even just two users online.
Update: Modifying the rate limiting flood zone from 12r/s to 36r/s, and burst from 12 to 36, has at least reduced the impact of who’s online. We are testing this configuration now.
Further update: with the tripled flood rate, we have had no reports of problems now.
I am using external nginx and have applied the rate limiting to the external nginx, using the ratelimited template as a starting point but am using rate limiting in the internal nginx, because the external nginx doesn’t know which routes are static, and so applies rate limiting to static routes as well as to dynamic ones, which when I did it caused lots of failures/retries on loading static assets. Note that this is a reason to run external nginx; otherwise I think that all IPv6 traffic is attributed to the docker IP address and is rate limited as if it is one IP address.