How to adjust parameters for large and heavy traffic site?

Support
1

I’m running a community with 38k users on a 24-core and 64GB VM. I set DISCOURSE_UNICORN_WORKERS to 48.

Recently the site seems to response slow after the UV reaches 20k one day and PV reaches 500k. User will receive their notification several hours after it actually happens.

The CPU workload is around 15%. So I believe this is due to too few Sidekiq workers and set DISCOURSE_UNICORN_SIDEKIQS to 10 and DISCOURSE_DB_POOL to 13. But this causes many 500 errors in many pages. And I think it’s too large to PostgreSQL.

So, I wonder whether Discourse is auto-scaling to a 24-core server without manual setting? Or how to set proper concurrent parameters for a large site like mine?

1 Like
2

The easiest way I’ve found to auto scale the resources is to just re-run ./discourse-setup which re-creates the app.yml with new values for relevant parameters.

1 Like
3

There are a few topics about tuning large sites. You likely need to give postgres more memory.

2 Likes
4

Yes, I tried to modify max_connections and shared_buffers in Postgres and it seems to work.

5

I can see db_shared_buffers change in app.yml, but Postgres’s shared_buffers inside the container seems unchanged.

Anyway, my problem is mostly solved. Thanks a lot.

1 Like