I have frequent 500 errors due to exceeding number of connections to my DB. This results in errors like this in /logs: Job exception: FATAL: remaining connection slots are reserved for non-replication superuser connections My config: UNICORN_WORKERS: 12 UNICORN_SIDEKIQS: 2 DISCOURSE_DB_POOL: 90 …

Did you change sidekiq threads manually? I believe the default is 25, and you have 2 process, so 50 connections, only with sidekiq.

At high loads I strongly recommend running pgbouncer, how is your db configured?

I would have to check, but /sidekiq reports only 5 threads per sidekiq: [图片]

Each web runs multiple threads as well, the numbers add up real quick

I have a data container running on a second server. I have changed the data.yml to expose db and redis ports and to give some extra memory: db_shared_buffers: "8GB" db_work_mem: "100MB"

[图片] sam: Each web runs multiple threads as well, the numbers add up real quick I have 25 connections from unicorn workers and over 70 from sidekiqs.

Restarting the web_only container doesn’t solve the problem, it seems like sidekiq connection persist. Restarting the data container solvers the problem for another couple of days. It seems like there are some issues with sidekiq connections.

[图片] TomekB: It seems like there are some issues with sidekiq connections. Interesting. We are usually internally totally shielded from this bug cause we use pg bouncer, but we will have a look internally to see if we see similar leaks. :man_farmer:t4: Can you debug into the “connections” ha…

How do you rotate connections with pgbouncer (session pooling, transaction pooling)? I will try to debug these stalled connections but I think most of them were idle.

We use transaction pooling, session pooling is a no-go with Rails.

Too many connections to DB, how to optimize

Support

TomekB (Tomek) 2017 年7 月 19 日 12:19 15

select pg_size_pretty(pg_database_size('discourse'));

returns 55 GB

The directory_items table is 207 MB.

话题		回复	浏览量
Primary Postgres database process (postmaster) eating all CPU Self-hosting server-resources	13	4462	2020 年5 月 28 日
Discourse Crash due to PSQL connection issue Self-hosting	9	589	2024 年3 月 17 日
Understanding DB Pooling for Sidekiq Workers Development	6	843	2025 年3 月 6 日
Very slow Sidekiq issue with large queue due to massive numbers of unread user notifications Self-hosting	68	8314	2020 年4 月 11 日
"Your Redis network connection is performing extremely poorly" Self-hosting	6	826	2026 年2 月 16 日

Too many connections to DB, how to optimize

相关话题