# Memory creep in last couple of updates
**URL:** https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746
**Category:** Self-hosting
**Tags:** hosting
**Created:** [5월 14, 2019, 10:51오후 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746 "2019-05-14T22:51:34Z")
**Posts on this page:** 8
**Page:** 2
### Author:  [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [5월 17, 2019, 11:46오전 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/22 "2019-05-17T11:46:26Z")
Thanks
> [@sam](#):
>
> You are running too many unicorns
This is a default installation, there are no customizations other than adding 3 official plugins to it. So part of the question is, did the change a recent update which is why it’s beginning to creep up now?
What about sidekiq, is it normal for it to almost double memory consumption over time?
> [@sam](#):
>
> Cut unicorn count down by 3
I’ve seen this question asked in a couple of places but didn’t find the answer? Is this the `UNICORN_WORKERS` in app.yml
I see it’s currently set to 8 and I’m guessing from what I read on the forum this number is set when it rebuilds based on the number on CPU, which in this case is 4, so I’m struggling to understand why the memory consumption started increasing lately if there’s been no changes to the hardware/setup?
---
### Author:  [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [5월 17, 2019, 10:28오후 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/23 "2019-05-17T22:28:21Z")
Okay an update, after the last fork unicorn patch, the utilization seems to have stabilized around 73% (higher than before but atleast not going up past 85%)
I’m reducing the number of unicorns from 8 to 5 as suggested and will let you know how it goes.
I’m still not clear as to why SideKiq is increasing its memory consumption and if something can/should be done about it or if it should be ignored.
Thanks for your help
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [5월 17, 2019, 10:29오후 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/24 "2019-05-17T22:29:33Z")
How high is sidekiq? We have protection in place in case it goes above 500
---
### Author:  [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [5월 17, 2019, 10:32오후 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/25 "2019-05-17T22:32:47Z")
At last check `ps aux` reported this (when utilization was 71%)
```plaintext
49418 0.6 8.0 1041604 324192 ? SNl May15 7:19 sidekiq 5.2.7 discourse [0 of 5 busy]
```
However in the error logs I did see sidekiq being restarted every night around midnight with this error:
> Sidekiq is consuming too much memory (using: 502.99M)
I’ll continue to keep an eye on it
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [5월 17, 2019, 10:37오후 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/26 "2019-05-17T22:37:03Z")
@sam don’t we have sidekiq logging now for better tracking this courtesy of @david?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [5월 20, 2019, 4:38오전 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/27 "2019-05-20T04:38:25Z")
We have better logging in that we know what jobs ran, but finding memory leaks is still a very involved process. I will do an internal review to see if we are seeing this on any of our sites.
---
### Author:  [@RBoy](https://meta.discourse.org/u/RBoy)
#### Post date: [5월 20, 2019, 2:21오후 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/28 "2019-05-20T14:21:03Z")
Just an update here, after reducing the number of unicorn workers from 8 to 5 as suggested the memory utilization has now stabilized at 57% and this is the output of `ps aux --sort -%mem`
```plaintext
PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
8555 0.6 8.4 1114316 341412 ? SNl May17 24:03 sidekiq 5.2.7
8595 0.4 8.3 958024 334536 ? Sl May17 16:10 unicorn worker
8614 0.4 8.1 950856 329196 ? Sl May17 16:01 unicorn worker
8604 0.4 8.1 955980 328432 ? Sl May17 15:56 unicorn worker
8586 0.4 8.0 958544 323008 ? Sl May17 15:58 unicorn worker
8577 0.4 7.9 1072200 321624 ? Sl May17 16:05 unicorn worker
8446 0.1 4.9 481532 197740 ? Sl May17 6:03 unicorn master
```
@sam a follow up clarification on the impact of reducing the number of unicorn workers. I was reading on the forum that each worker supports upto 5 jobs, so with 5 workers that’s 25 jobs. I read somewhere else that each worker is good for about 400 connections. I’m not really clear on what this means for system scalability, I suspect it’ll be fine but it would nice if you could outline what the unicorn workers are used for and how many concurrent users (rough ballpark) could the system support with 5 workers. Thanks in advance.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [5월 21, 2019, 7:42오전 UTC](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746/29 "2019-05-21T07:42:39Z")
Sidekiq is the thing that does the jobs.
Unicorn is for web requests. Running 5 means you can handle 5 “slow” web requests concurrently. Certain web requests like “message bus” and “avatar caching” and “uploads” run in background threads, so the real number of concurrent requests tends to be a lot higher.
“concurrent users” highly depends on what the users are doing and logged in vs anon which is heavily cached. I can’t really provide a particular guideline, but I can provide instrumentation that will tell you if there is a problem.
> [@Discourse Prometheus](https://meta.discourse.org/t/prometheus-exporter-plugin-for-discourse/72666):
>
> discourse2Summary Discourse Prometheus is the official [Prometheus](https://prometheus.io/) exporter for Discoursehammer_and_wrenchRepository Link [https://github.com/discourse/discourse-prometheus](https://github.com/discourse/discourse-prometheus)open_bookInstall Guide [How to install plugins in Discourse](https://meta.discourse.org/t/install-plugins-in-discourse/19157) The Discourse Prometheus plugin collects key metrics from Discourse and exposes them in the /metrics path so prometheus can consume them. These metrics can be used to Graph all sorts of data like: Median and 99th percentile time…
[이전 페이지](https://meta.discourse.org/t/memory-creep-in-last-couple-of-updates/117746.md?page=1)