Can some please provide clarity on Sidekiq worker database pooling? Currently, we’ve set DISCOURSE_DB_POOL to 15, with 2 Sidekiq workers each having a concurrency of 10. After reading the Sidekiq documentation, it appears that Sidekiq threads share the DB pool, hence I anticipate a maximum of 15 * 2 = 30 DB connections from Sidekiq workers. However, despite this understanding, we’re observing spikes in DB connections exceeding the expected maximum of 30. [image] These spikes occur approximately every 15 minutes and seem to be primarily associated with the PeriodicalUpdates scheduled job. Could anyone kindly provide insights that will help us to avoid these spikes?

@david I noticed that you have done amazing work in this area. Can you please provide some insights?

I’m afraid I don’t know the details of how all this works off-hand. It may be worth checking whether you’ve customized the UNICORN_SIDEKIQS env. That indicates how many sidekiq processes are launched (each of which will have many threads). What exactly is your graph showing? Is it ‘concurrent connections’? Or is it ‘number of connections created’? If it’s the latter, perhaps some connections are being created & destroyed very quickly. And one last question: what problem are you trying to solve here? Is there some issue being caused by the number of connections?

Thanks for getting back. Following are the related configurations we have: DISCOURSE_DB_POOL: “15” DISCOURSE_SIDEKIQ_WORKERS: “10” UNICORN_SIDEKIQS: “2” This means we have 2 Sidekiq processes, each with 10 threads. The graph shows the current number of client connections. We are trying to configure the size of the connection pool (RDS proxy sitting in front of PostgreSQL) correctly. To do so, we need a better understanding of the connection pooling on the application side. Why is the application not respecting the DISCOURSE_DB_POOL config as expected? If DISCOURSE_DB_POOL is working as expected, why do we see such big spikes in DB connections? These spikes seem to be aligned with the PeriodicalUpdates scheduled job runs. Please let me know if you have more questions. Appreciate the help to uncover the mystery.

Do we need to have any config files in addition to DISCOURSE_DB_POOL environment variable to configure database pooling?

[image] Mudit Maheshwari: RDS proxy Are you 100% your proxy is not at fault here?

Yes, I don’t think the proxy is at fault here.

Understanding DB Pooling for Sidekiq Workers

Development

sam (Sam Saffron) September 12, 2024, 4:35am 6

Are you 100% your proxy is not at fault here?

Topic		Replies	Views
Too many connections to DB, how to optimize Product Support	17	4169	July 27, 2017
UNICORN_WORKERS vs UNICORN_SIDEKIQS, DISCOURSE_DB_POOL? Self-hosting	6	2781	February 23, 2020
Optimizing a Large Discourse Multisite: Database and Sidekiq Bottlenecks Self-hosting multisite , hosting	4	382	September 22, 2025
Issue: Extremely Slow Sidekiq Processing After Large Imports on Multisite Instance Product Support	2	228	April 23, 2025
"Your Redis network connection is performing extremely poorly" Self-hosting	6	852	February 16, 2026

Understanding DB Pooling for Sidekiq Workers

Related topics