So all the claims that the UNICORN WORKER should be 2*vCPU are wrong?
My server is Intel Xeon E5-2686 v4 @ 2.30GHz—24vCPU+32GB Ram
How many UNICORN WORKER do I need to set up?
8? or 48?
My site have more than 7k users and about 1k daily engaged users. users sending 3k-7k posts per day. our community has 120k-200k pageviews/day including crawlers and anonymous users.
That depends on a lot of factors. Like how big your database size is in comparison to your RAM, the ratio of logged-in to anonymous traffic, whether you have plugins that keep your Sidekiq queue busier, if you are running YJIT, etc.
What is simple is looking at the MiniProfiler data during your peak hour and browsing the forum to see if the performance is any worse and identifying the bottleneck.
Since this CPU is an older one, I’d start with more than the usual number of Unicorn workers, as every request will take longer than usual. But if you are running PostgreSQL and Redis on the same server, you can’t starve those off by running too many workers.
Try running 16 workers to start with, and evaluate how the site is performing.
Is there a simple human-friendly description of what Unicorn workers are doing? The impression I get is that every user page request has to go to a Unicorn worker to be dealt with. If you don’t have enough, the user has to wait. If you have too many… well maybe that costs you a little RAM?
They’re the application web servers.
1 Like
Looks like your 10 years old CPU is showing it’s age. Increasing Unicorn workers will allow you to serve more users simultaneously but won’t make an individual request any faster.
Can you try enabling YJIT?
3 Likes
With your hardware I would expect to get a mean logged-in list/latest time of around 150ms app, 80ms SQL.
I’d start with 12 workers and see how it behaves with that. The best thing you can do is track metrics; if you want to know if you should add more workers, check to see if requests are getting queued waiting for app workers.
Are you tracking the metrics that Discourse itself exports via the prometheus exporter? These will give you a good view on how the instance is performing, overall.
What are performance numbers like for anonymous and regular (not admin) users?
5 Likes
There are many mega topics in our Discourse, and the largest one is already in the twelfth section.
Reply views
(/stops laughing, logs into hosting provider … )
wow, is this not the default by now??
edit: ah of course you might have built with an old app.yml and not updated since
1 Like
It is in our hosting, but it is kinda hard to make it a default as people may be running in RAM constrained scenarions…
That said, our JS build use so much more RAM than Discourse itself, that you may say that anyone that can build the JS assets has plenty of RAM to spare 
1 Like
In these pictures, how many workers do you have setup?
I’d say you should
- Increase workers a little, as you get a little queueing
- Enable YJIT, as your web time is quite slow
There are only 8 workers now, and YJIT already enabled.
How many workers should I increase?
BTW here is what Falco was looking at to make that recommendation:
I’d bump 8 → 12. Gives you some headroom and should clear out those queues.
That large spike, BTW, is indicative of the other requests waiting on… something, probably a common lock. Perhaps a megatopic post.
If you can get postgres usage metrics as well, that would be useful.
megatopics are a weak spot, see Improving Instance Performance (Megatopics, Database Size and Extreme Load)
Consider breaking these up or using chat instead (I see your largest one with 8.9k replies is explicitly a chat room).
Our community’s discussion culture is megatopics, which was formed even before we started using Discourse, and chat lacks the features of blurred spoilers and hidden details
1 Like
How to do it
We use GitHub - prometheus-community/postgres_exporter: A PostgreSQL metric exporter for Prometheus, though I’m not sure if there’s any guides on meta around setting it up.
But, given that you seem to already have prometheus set up, looks like you know what you’re doing.
now server ram is 16/32GB, UNICORN_WORKERS: 12, db_shared_buffers: “4096MB”
Because there is still RAM available and a few web request queued, I increased the UNICORN_WORKERS to 24. In the afternoon today, the server suddenly shut down and after it was restarted, users flooded in immediately. This led to a very low number of Active Web Requests and a large number of queued requests. A few days ago, we observed that 24 UNICORN_WORKERS could handle 150+ Active Web Requests, but only 30 Active Web Requests this afternoon. This is because we have just changed the domain, and there are many posts being rebaked by sidekiq. This has caused a lot of pressure on the server. What should we do?
