# Could sidekiq queue be reason for 500 errors?
**URL:** https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631
**Category:** Self-hosting
**Tags:** server-resources
**Created:** [June 11, 2018, 7:06pm UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631 "2018-06-11T19:06:47Z")
**Posts on this page:** 12
**Page:** 2
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [June 13, 2018, 10:05am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/21 "2018-06-13T10:05:21Z")
> [@danmaby](#):
>
> TBH I’m not sure how to check what’s keeping the Unicorn Workers busy?
Click on the ‘Busy’ tab at the top of the /sidekiq screen and you’ll see your queues and the jobs inside them. Each job also shows how long it’s been active, which is a great indicator of problems.
I assume your Critical queue jobs are getting handled first, but let’s confirm that these are indeed the jobs that are causing the slowness.
---
### Author:  [@danmaby](https://meta.discourse.org/u/danmaby)
#### Post date: [June 13, 2018, 10:11am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/22 "2018-06-13T10:11:10Z")
That was one of those _it can’t be that easy_ moments 🤣
OK, so it looks like things are processing, nothing seems to be obvious in terms of hold ups:
---
### Author:  [@danmaby](https://meta.discourse.org/u/danmaby)
#### Post date: [June 13, 2018, 10:13am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/23 "2018-06-13T10:13:49Z")
And now the sites gone down again 😕
```
# free -m
total used free shared buff/cache available
Mem: 7983 4829 128 2116 3025 755
Swap: 2047 72 1975
```
---
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [June 13, 2018, 10:31am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/24 "2018-06-13T10:31:36Z")
Looks like you don’t have any seriously slow tasks there.. What’s your CPU load like during the processing? If it’s low you can try increasing `UNICORN_SIDEKIQS`. It’s currently set to 1 for you, adding more will add 5 job processors at a time.
In contrast, the `UNICORN_WORKERS` setting affects the number of concurrent web requests that can be handled - this is not related to Sidekiq and increasing the value won’t help solve this issue.
> [@danmaby](#):
>
> And now the sites gone down again 😕
Do you see anything useful in the logs? They’re located in `/var/discourse/shared/standalone/log`
---
### Author:  [@danmaby](https://meta.discourse.org/u/danmaby)
#### Post date: [June 13, 2018, 10:49am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/25 "2018-06-13T10:49:42Z")
Thanks again @bartv, I’ve added `UNICORN_SIDEKIQS=5` to the app.yml and run `./launcher restart app` now looking back at the sidekiq dashboard it’s still only processing around 10 per second.
Have I got `UNICORN_SIDEKIQS=5` correct or should it be `UNICORN_SIDEKIQS: 5`
The logs are showing thousands of entries for:
`Started GET "/sidekiq/stats" for 86.1.10.29 at 2018-06-13 10:34:22 +0000`
---
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [June 13, 2018, 10:51am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/26 "2018-06-13T10:51:06Z")
It should be `UNICORN_SIDEKIQS: 5` - the same formatting as any other setting in app.yml. You can verify this by going to the busy tab in Sidekiq again - the number of processes should match the value you entered here.
And a tip: to quickly update these settings you don’t need to do a full rebuild; just do this:
```plaintext
./launcher destroy app
./launcher start app
```
---
### Author:  [@danmaby](https://meta.discourse.org/u/danmaby)
#### Post date: [June 13, 2018, 11:04am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/27 "2018-06-13T11:04:11Z")
OK so I updated the Unicorn Sidekiqs to 5 and this temporarily doubled the speed to around 10 per second, until the server fell over again.
```
# free -m
total used free shared buff/cache available
Mem: 7983 6086 125 971 1771 629
Swap: 2047 46 2001
```
I’ll try adjusting the number to see if I can get a stable increase without the server bugging out.
---
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [June 13, 2018, 11:05am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/28 "2018-06-13T11:05:01Z")
I really urge you to inspect your log files after your server crashes; they might provide actionable information.
---
### Author:  [@danmaby](https://meta.discourse.org/u/danmaby)
#### Post date: [June 13, 2018, 11:17am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/29 "2018-06-13T11:17:11Z")
I see this error 10,000’s times in `/var/discourse/shared/standalone/log/rails/production.log`
> [@danmaby](#):
>
> Redis::CommandError: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.
As well as a very similar message, again thousands of times over, in `/var/discourse/shared/standalone/log/rails/unicorn.stderr.log`
---
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [June 13, 2018, 11:20am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/30 "2018-06-13T11:20:19Z")
> [@danmaby](#):
>
> Please check Redis logs for details about the error.
What does the Redis log say? I had a similar issue with Redis running out of memory; the rebuild log provided the solution to this:
```plaintext
186:M 01 Jun 11:02:31.042 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
186:M 01 Jun 11:02:31.042 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
```
Perform these commands then restart your Discourse:
```plaintext
sysctl vm.overcommit_memory=1
echo never > /sys/kernel/mm/transparent_hugepage/enabled
```
Note that you’ll still need to make these persistent! (See the quoted text above to learn how)
---
### Author:  [@danmaby](https://meta.discourse.org/u/danmaby)
#### Post date: [June 13, 2018, 11:37am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/31 "2018-06-13T11:37:10Z")
This seems to have helped! We’re five minutes in now and it’s holding steady at around 20 queued items per second with 5 unicorn Sidekiqs
> [@bartv](#):
>
> Perform these commands then restart your Discourse:
I really appreciate your time on this!
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [July 13, 2018, 11:37am UTC](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631/32 "2018-07-13T11:37:12Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
[Previous page](https://meta.discourse.org/t/could-sidekiq-queue-be-reason-for-500-errors/89631.md?page=1)