# Sidekiq is consuming too much memory, restarting
**URL:** https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395
**Category:** Self-hosting
**Created:** [August 9, 2016, 2:00pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395 "2016-08-09T14:00:59Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@gkln](https://meta.discourse.org/u/gkln)
#### Post date: [August 9, 2016, 2:00pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/1 "2016-08-09T14:00:59Z")
Hi,
We get this message in the logs about twice a day since the update to Discourse 1.6.0:
> Sidekiq is consuming too much memory (using: 532.09M) for `forum.detendusdupad.com`, restarting
Is it a normal behaviour, a misconfiguration or is Sidekiq actually leaking?
The `UNICORN_WORKERS` value is set to 4 on a 16GB install. The 1.6.0 update was applied by doing a clean rebuild of our split containers setup.
Thank you.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [August 9, 2016, 10:22pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/2 "2016-08-09T22:22:13Z")
Any ideas here @sam?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [August 9, 2016, 10:28pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/3 "2016-08-09T22:28:36Z")
I see it a lot in our logs, will be investigating. Something about Ruby 2.3 means that in some cases our Sidekiq process gets more memory hungry.
---
### Author:  [@waza](https://meta.discourse.org/u/waza)
#### Post date: [August 10, 2016, 2:54am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/4 "2016-08-10T02:54:36Z")
Slightly off topics, when developing my own projects, Sidekiq with Sidetiq consumses memory too much (heap increase 100mb per hour). Same configuration, ruby 2.3.1, rails 4.x, latest sidekiq. Seems like this is the problem with Ruby 2.3.1.
Some people recommend me to just switch to jRuby because of JVM does GC better, but i don’t really know.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [August 10, 2016, 3:14am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/5 "2016-08-10T03:14:38Z")
> [@waza](#):
>
> Sidetiq
FYI we removed sidetiq quite a while back. I will look at sorting out the current issue, just need to gather lots of metrics for a while.
---
### Author:  [@dandrews](https://meta.discourse.org/u/dandrews)
#### Post date: [August 17, 2016, 7:23pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/6 "2016-08-17T19:23:47Z")
Any new insights on this problem? We see this warning every hour in our logs.
---
### Author:  [@kgish](https://meta.discourse.org/u/kgish)
#### Post date: [September 16, 2016, 10:33am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/7 "2016-09-16T10:33:02Z")
We are also experiencing the same problem, would it alleviate the problem if we bump up memory from 2GB to 4GB?
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [September 16, 2016, 8:38pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/8 "2016-09-16T20:38:42Z")
It never _hurts_ to have more memory, I wouldn’t do it based on this message alone, though. You should correlate memory to traffic and usage.
---
### Author:  [@Caswal\_Parker](https://meta.discourse.org/u/Caswal_Parker)
#### Post date: [September 18, 2016, 9:35am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/9 "2016-09-18T09:35:40Z")
I am also getting this on my machine with 64gb of ram. Running 8 sidekiqs, and 16gb allocated to shared db.
Machine also runs apache2 serving up a wordpress site.
I am also getting periodic slow downs on the forums, dropping them to a crawl (a minute or so to load a thread). I have a feeling it is the daily backup. But havn’t really pinned it down yet.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [September 18, 2016, 5:13pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/10 "2016-09-18T17:13:59Z")
Do you mean 8 unicorns?
---
### Author:  [@Caswal\_Parker](https://meta.discourse.org/u/Caswal_Parker)
#### Post date: [September 18, 2016, 10:47pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/11 "2016-09-18T22:47:19Z")
Yes, sorry.
```
db_shared_buffers: "16GB"
db_work_mem: "40MB"
UNICORN_WORKERS: 8
```
I have reduced the workers to 4 for the time being. Seams I mistook my hardware configuration, and thought the server had dual 4 Core Xeons, but it only has one. We will see how it goes.
Edit: Seems it is an apache2 ProxyPass issue. Skipping the proxy makes it nice and fast.
---
### Author:  [@teichsta](https://meta.discourse.org/u/teichsta)
#### Post date: [September 30, 2016, 9:00am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/12 "2016-09-30T09:00:35Z")
any new insights @sam?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [November 22, 2016, 4:34am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/13 "2016-11-22T04:34:49Z")
I have added some GC tuning to help mitigate some of the memory issues, noticing it less in the logs these days.
>
Try:
```plaintext
cd /var/discourse
git pull
./launcher rebuild app
```
---
### Author:  [@mitchellk](https://meta.discourse.org/u/mitchellk)
#### Post date: [March 6, 2017, 11:22am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/14 "2017-03-06T11:22:17Z")
I’m running 1.8.0 Beta6 and seeing this in my logs. Already did a rebuild of the app since I changed the domain name of the forum on Saturday and everything appears (front end) to be running fine. I have ample memory 22GB available.
```
Sidekiq is consuming too much memory (using: 540.01M) for 'photographic.community', restarting
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.7/lib/logster/logger.rb:93:in `add_with_opts'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.7/lib/logster/logger.rb:50:in `add'
/usr/local/lib/ruby/2.3.0/logger.rb:498:in `warn'
config/unicorn.conf.rb:123:in `check_sidekiq_heartbeat'
config/unicorn.conf.rb:146:in `master_sleep'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:284:in `join'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/bin/unicorn:126:in `'
/var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn:22:in `load'
/var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn:22:in `'
```
My app.yml settings are:
```
db_shared_buffers: "5GB"
db_work_mem: "40MB"
UNICORN_WORKERS: 6
```
and also seeing this error in my logs
`console' is undefined Url: https://photographic.community/assets/ember_jquery-60dedd1dca1d8b8ce48b6d0087db3b96f3149b9b5af2d0c7c6357967d29031fb.js Line: 20 Window Location: https://photographic.community`
```
params
message 'console' is undefined Url: https://photographic.community/assets/ember_jquery-60dedd1dca1d8b8ce48b6d
url https://photographic.community/assets/ember_jquery-60dedd1dca1d8b8ce48b6d0087db3b96f3149b9b5af2d0c7c6
line 20
window_location https://photographic.community/
```
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [March 6, 2017, 6:10pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/15 "2017-03-06T18:10:46Z")
It is not unusual to see this on high volume sites from time to time.
---
### Author:  [@mitchellk](https://meta.discourse.org/u/mitchellk)
#### Post date: [March 7, 2017, 4:37pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/16 "2017-03-07T16:37:46Z")
That’s the problem @codinghorror it’s brand new and not really high volume yet. Any suggestions on changing any of my app.yml settings as you can see I have 22Gb available in total allocated to the box.
---
### Author:  [@cpradio](https://meta.discourse.org/u/cpradio)
#### Post date: [March 7, 2017, 4:39pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/17 "2017-03-07T16:39:53Z")
By chance did you import from another forum into Discourse? If so, can you make a backup, and the restore said backup to improve Postgres performance.
Oh, other random thoughts, is Discourse the only thing running on that server, or are there other apps on it too?
---
### Author:  [@mitchellk](https://meta.discourse.org/u/mitchellk)
#### Post date: [March 7, 2017, 4:53pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/18 "2017-03-07T16:53:14Z")
Nope it was a clean install. Only two small sites running on Nginx, neither of which are public only git wordpress sites I use for versioning control while developing. Other than that Discourse can have the entire server to itself.
---
### Author:  [@tophee](https://meta.discourse.org/u/tophee)
#### Post date: [July 13, 2017, 11:08am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/19 "2017-07-13T11:08:41Z")
I’ve also seen this error for a couple of days now on my small and low activity instance with 2GB RAM.
Do I understand this thread correctly in that this is currently just something that happens sometimes and we don’t quite know why?
---
### Author:  [@pelcami](https://meta.discourse.org/u/pelcami)
#### Post date: [August 25, 2017, 4:47pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/20 "2017-08-25T16:47:52Z")
Yep. Seeing the exact same thing too. I have three identical Discourse servers. They are dedicated (no other application is served off of them, and have never received any imports from another Forum). Right now, the logs are only writing about the issue on one of them (this one, however, does get the most hits, and posts though). The write to the log, however, followed a complete degredation of all services (app stopped responding, discourse site stopped drawing, and I could not ssh into the box). Had to cycle power to get back into it. 🤐
I’ve applied the band-aid of increasing memory on this one server from 4Gb to 8GB, and have not seen recurrence in the last 24 hours.
ruby 2.3.1p112
sidekiq 5.0.3
discourse 1.9.0.beta7
ubuntu 16.04.3 LTS
[Next page](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395.md?page=2)