# Very slow Sidekiq issue with large queue due to massive numbers of unread user notifications

**URL:** <https://meta.discourse.org/t/very-slow-sidekiq-issue-with-large-queue-due-to-massive-numbers-of-unread-user-notifications/140716>\
**Category:** Self-hosting\
**Created:** [February 5, 2020, 12:07pm UTC](https://meta.discourse.org/t/very-slow-sidekiq-issue-with-large-queue-due-to-massive-numbers-of-unread-user-notifications/140716 "2020-02-05T12:07:57Z")\
**Posts on this page:** 1\
**Showing post:** 1

<div class="post-metadata">

**Author:** ![markersocial](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/markersocial/32/170136_2.png) [@markersocial](https://meta.discourse.org/u/markersocial)\
**Post date:** [February 5, 2020, 12:07pm UTC](https://meta.discourse.org/t/very-slow-sidekiq-issue-with-large-queue-due-to-massive-numbers-of-unread-user-notifications/140716/1 "2020-02-05T12:07:57Z")

</div>

So have an issue with sidekiq.

It will run amazingly fast through jobs when monitoring via the sidekiq web ui. But occasionally it appears like it gets overwhelmed and starts running extremely slow. Running at 1-5% or so of it’s normal speed and does not recover unless I flush redis, despite server resource usage being fine/low.

It appears like once the queue hits a certain size, it seizes up and slows down drastically. Causing the queue to grow even more. I’m just guessing here though, maybe the queue is just large due to it slowing down for some other reason.

This gif describes what it looks like to me.

 ![simpsons_EDIFIL20161024_0001](https://global.discourse-cdn.com/meta/original/3X/e/d/ed597221a6a970f2ea87466bb32a4673cff62d39.gif)

There are plenty of server resources available, CPU usage is very low right now - under 10%. Plenty of ram and ssd available also. Regarding the server, it has 16 CPU cores with 32 threads. I’ve tried running between 8-14 unicorn\_sidekiqs. I also tried 20, but that created a lot of 5xx errors.

I was able to speed up slow jobs displayed on the ‘busy’ tab of the sidekiq web ui using  
[Could sidekiq queue be reason for 500 errors? - #30 by bartv](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/30) (  
adding ‘vm.overcommit\_memory = 1’ to /etc/sysctl.conf file and rebooting) and also decreasing unicon\_sidekiqs down to 8 (from 12).

It’s still running slow though. I did see this in the redis log yesterday (the only other warning was regarding not having overcommit\_memory set to 1, which I modified above):

`# WARNING: /proc/sys/net/core/somaxconn is set to the lower value of 128`

^ Has anyone fixed this warning above?

Anyhow, if anyone has any ideas as to what could be the cause and/or fix - please let me know. I’d appreciate it.

Would be really great to resolve this issue so it doesn’t happen again, rather than flushing.

Here is a screenshot of what I’m seeing on the sidekiq dashboard:

 ![Screen Shot 2020-02-05 at 6.52.24 PM](https://global.discourse-cdn.com/meta/original/3X/7/5/75e5dc6f1eb55811b5251bc5900bf62753f3e012.png)

And some screenshots of the jobs under the busy tab:

 ![Screen Shot 2020-02-05 at 6.42.03 PM](https://global.discourse-cdn.com/meta/original/3X/a/6/a68694a3dc91b7c0c2066126ee3062a196291bac.png)

Also, does anyone know if it is safe to use this option? Deleting the low priority queue from the sidekiq web ui?

 ![Screen Shot 2020-02-05 at 6.47.18 PM](https://global.discourse-cdn.com/meta/original/3X/4/8/48b402602802f12dc4a8c988def563f373f94403.png)

---

_[View the full topic](https://meta.discourse.org/t/very-slow-sidekiq-issue-with-large-queue-due-to-massive-numbers-of-unread-user-notifications/140716)._