# Sidekiq is consuming too much memory, restarting

**URL:** https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395
**Category:** Self-hosting
**Created:** [August 9, 2016, 2:00pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395 "2016-08-09T14:00:59Z")
**Posts on this page:** 20
**Page:** 2

<div class="post-metadata">

### Author: ![dandrews](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/dandrews/32/122413_2.png) [@dandrews](https://meta.discourse.org/u/dandrews)
#### Post date: [October 4, 2017, 7:22pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/21 "2017-10-04T19:22:47Z")

</div>

Has the issue resolved after you bumped up to 8gb? I’m considering doing the same as you since we’re still get these sidekiq errors on a 4gb machine.

---

<div class="post-metadata">

### Author: ![pelcami](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pelcami/32/59732_2.png) [@pelcami](https://meta.discourse.org/u/pelcami)
#### Post date: [October 4, 2017, 8:18pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/22 "2017-10-04T20:18:56Z")

</div>

Fortunately, after increasing to 8GB, the problem has not recurred.

---

<div class="post-metadata">

### Author: ![dandrews](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/dandrews/32/122413_2.png) [@dandrews](https://meta.discourse.org/u/dandrews)
#### Post date: [October 4, 2017, 8:20pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/23 "2017-10-04T20:20:01Z")

</div>

Thanks! I’ll try that approach then.

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [October 11, 2017, 3:54pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/24 "2017-10-11T15:54:13Z")

</div>

It suddenly started appearing to me a couple of days ago.

Before I never saw `Sidekiq is using too much memory` errors in the logs. Now I see them a few times a day.

And today it happened exactly as @pelcami has mentioned: the VM just plainly became unresponsive. SSH didn’t work. No access on HTTP or HTTPS.

The VM is on Azure, so I pulled up the charts, and it showed network activity suddenly dropping down completely to zero (like a cliff) and CPU stuck at around 40% (probably completely using up one core). Restarting the VM solved it.

What I want to say is, it happened so suddenly that it was almost like one minute it was business as usual, normal network traffic, then the next minute it fell off a cliff and dropped to zero. Not gradual at all.

Something is happening to trigger an infinite loop in Sidekiq or something…

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [October 11, 2017, 10:59pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/25 "2017-10-11T22:59:17Z")

</div>

You can follow up with @tgxworld who is working the issue.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [October 11, 2017, 11:07pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/26 "2017-10-11T23:07:14Z")

</div>

Be sure to do a full rebuild we recently bumped ruby version in image

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [October 12, 2017, 2:21am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/27 "2017-10-12T02:21:11Z")

</div>

ok. I’ll rebuild and observe.

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [October 12, 2017, 3:39am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/28 "2017-10-12T03:39:49Z")

</div>

Just as a reminder to people: If you get a “Docker version too low” error during rebuild, read the following:

> [@End of Life for Ubuntu 14.04 Trusty and Discourse 1.9.0.beta10?](https://meta.discourse.org/t/end-of-life-for-ubuntu-14-04-trusty-and-discourse-1-9-0-beta10/71886/4):
>
> Ok, all good. Here’s what I did incase it helps others who had an older docker. wget -qO- https://get.docker.com/ | sh This will update docker, or rather do the commands of adding the key and repository to the latest binaries for your system. Note: it will try to scare you with this: Warning: the "docker" command appears to already exist on this system. As I already had a snapshot, I went for it and it seems ok. I rerun the update for Discourse like this: ./launcher rebuild app After that …

---

<div class="post-metadata">

### Author: ![RBoy](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [November 16, 2017, 10:44pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/29 "2017-11-16T22:44:15Z")

</div>

I’ve started seeing this after an update a few days ago @sam after the Amazon lifecycle patch I did a complete rebuild and now I’m seeing this everyday. Never happened before that.

I’m on v1.9.0.beta14 +200

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [November 16, 2017, 11:59pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/30 "2017-11-16T23:59:44Z")

</div>

When is the last time you did a

```plaintext
git pull
./launcher rebuild app

```

You could be on an older version of Ruby.

---

<div class="post-metadata">

### Author: ![RBoy](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [November 18, 2017, 1:34am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/31 "2017-11-18T01:34:46Z")

</div>

> [@sam](#):
>
> ./launcher rebuild app

I did this for the build above beta14+200

I didn’t do a git pull, thought that rebuild does that.

EDIT: Okay I’m it again now, lets see how it goes.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [November 18, 2017, 1:41am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/32 "2017-11-18T01:41:59Z")

</div>

That’s good to know. Also, I just noticed that vagrant is still on Rails 4.

---

<div class="post-metadata">

### Author: ![RBoy](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [November 27, 2017, 7:15pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/33 "2017-11-27T19:15:54Z")

</div>

No issues so far after a git pull + rebuild instead of just a rebuild. Somehow I thought a rebuild would also do a pull

---

<div class="post-metadata">

### Author: ![Tumi](https://avatars.discourse-cdn.com/v4/letter/t/f475e1/32.png) [@Tumi](https://meta.discourse.org/u/Tumi)
#### Post date: [December 9, 2017, 8:31pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/34 "2017-12-09T20:31:25Z")

</div>

this is the dashboard from what ? pls teach me .

---

<div class="post-metadata">

### Author: ![JagWaugh](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jagwaugh/32/69335_2.png) [@JagWaugh](https://meta.discourse.org/u/JagWaugh)
#### Post date: [December 9, 2017, 8:37pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/35 "2017-12-09T20:37:10Z")

</div>

Go to _ **yourforumurl** _/sidekiq to see what sidekiq is doing

similarly

/logs to see the error log

/admin/upgrade#/processes to see processes

---

<div class="post-metadata">

### Author: ![RBoy](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [January 23, 2018, 6:26pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/36 "2018-01-23T18:26:00Z")

</div>

Just thought I’d share that updating to v2.0.0 beta1 a couple of weeks ago this problem started again.

> Sidekiq is consuming too much memory (using: 513.13M)

I checked the dashboard and it’s currently using about 6M but somehow it ramps up to 500 about once a day I see this warning in the logs. Oddly it’s always around 4:20am to 4:50am in the mornings.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [January 24, 2018, 12:19am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/37 "2018-01-24T00:19:05Z")

</div>

Almost certainly a scheduled daily task cc @sam

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [May 3, 2018, 8:37pm UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/38 "2018-05-03T20:37:38Z")

</div>

Also, if your server has enough total RAM, you can increase the memory allocation for Sidekiq.

Place this in the `env:` section of `app.yml`:

`UNICORN_SIDEKIQ_MAX_RSS: 1000`

The default maximum is, I believe, 500mb.

---

<div class="post-metadata">

### Author: ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)
#### Post date: [October 13, 2020, 3:17am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/39 "2020-10-13T03:17:26Z")

</div>



---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [October 13, 2020, 6:43am UTC](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/40 "2020-10-13T06:43:28Z")

</div>

Closing the loop here, over the past 2 years we had one large change that reduced sidekiq memory. We made a change to the hosted v8 instance to ensure it cleans up used memory and runs a gc. This resulted in a 10-20% memory reduction in our hosting.

Except for that, yes, on mega busy multisites you need to adjust `UNICORN_SIDEKIQ_MAX_RSS`

[Previous page](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395.md?page=1)

[Next page](https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395.md?page=3)
