# Optimizing a Large Discourse Multisite: Database and Sidekiq Bottlenecks
**URL:** https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363
**Category:** Self-hosting
**Tags:** multisite, hosting
**Created:** [September 2, 2025, 8:16pm UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363 "2025-09-02T20:16:41Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@emonunix](https://meta.discourse.org/u/emonunix)
#### Post date: [September 2, 2025, 8:16pm UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363/1 "2025-09-02T20:16:41Z")
I’m seeking some expert guidance on optimizing a Discourse multisite setup. I have a single web VM and a separate database VM on a major cloud provider. While both machines have decent specs, I’m finding that my system gets overwhelmed by a large volume of background jobs, which seems to be stressing the database.
My current `app.yml` configuration is:
- `UNICORN_WORKERS: 4`
- `UNICORN_SIDEKIQS: 4`
- `DISCOURSE_SIDEKIQ_WORKERS: 10`
- `DISCOURSE_DB_POOL: 8`
Based on my observations, the bottleneck isn’t hitting a hard connection limit, but rather the sheer volume of jobs competing for database resources at the same time. The queues in Sidekiq are constantly backing up, which makes the site feel slow, even for basic administrative tasks.
I’m looking for a generic approach to tune the system for stability and performance. Specifically, I’d like to understand the best practices for:
- **Sidekiq Concurrency:** How should `DISCOURSE_SIDEKIQ_WORKERS` be sized in a multisite environment to handle a high job volume without stressing the database?
- **Queue Separation:** Is it recommended to run separate Sidekiq processes to handle different queues (e.g., `critical` vs. `low` priority)? This would ensure that heavy jobs don’t block more urgent ones.
I’m not looking for a solution that requires a major architectural change or moving to a different web server at this time, as I want to keep the process as simple and low-risk as possible. I’m hoping to get advice on a safe and effective path forward.
Thanks!
---
### Author:  [@itsbhanusharma](https://meta.discourse.org/u/itsbhanusharma)
#### Post date: [September 2, 2025, 11:26pm UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363/2 "2025-09-02T23:26:18Z")
Discourse should be able to adapt those based on your system resources. Note that it is safe to re-run `./discourse-setup` script if you have recently increased your system resources. The script can adapt to the increased resources and adjust your .yml accordingly
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [September 3, 2025, 12:36am UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363/3 "2025-09-03T00:36:25Z")
> [@emonunix](#):
>
> How should `DISCOURSE_SIDEKIQ_WORKERS` be sized in a multisite environment to handle a high job volume without stressing the database?
Sounds like you need fewer?
I’m pretty sure that it already knows to prioritize the high priority jobs.
---
### Author:  [@emonunix](https://meta.discourse.org/u/emonunix)
#### Post date: [September 5, 2025, 7:58pm UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363/4 "2025-09-05T19:58:24Z")
@itsbhanusharma @pfaffman Thanks for your help. I appreciate you both taking the time to share your expertise.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [September 22, 2025, 5:04pm UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363/5 "2025-09-22T17:04:57Z")
> [@emonunix](#):
>
> database resources at the same time
If the database is the bottleneck, have you considered moving to larger instance type for your cloud database?
---
### Author:  [@tobiaseigen](https://meta.discourse.org/u/tobiaseigen)
#### Post date: [October 22, 2025, 5:05pm UTC](https://meta.discourse.org/t/optimizing-a-large-discourse-multisite-database-and-sidekiq-bottlenecks/381363/6 "2025-10-22T17:05:26Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.