# Higher idle process activity after upgrade

**URL:** https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453
**Category:** Self-hosting
**Created:** [May 23, 2020, 3:09pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453 "2020-05-23T15:09:02Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![tormod](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tormod](https://meta.discourse.org/u/tormod)
#### Post date: [May 23, 2020, 3:09pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/1 "2020-05-23T15:09:02Z")

</div>

Hi,  
After upgrading Discourse to v2.5.0.beta4-399-gbf8085e436 (from 2.4 something, using command line bootstrap from discourse\_docker git, just before the Postgres 12 upgrade commits), there is notably more CPU churn on the server (even if no one is connected).

“top” tells that several “ruby” processes are active quite often. I have understood these are the unicorn sidekiq workers. Running redis-cli monitor I see that they make blocking BRPOP requests with a timeout of 2 s, to see if there is work to be done. So this amounts to a dozen of redis commands per second. Also there are statistics being updated every 5 seconds. redis-cli info stats tells there are in average 15-20 commands processed per second.

I don’t know if this activity can build up to the 2-4% CPU utilization that we see. With the old version it was less than 0.5%. I didn’t look at the same redis-cli stats back then.

I would be curious to try out a longer timeout to see if that helps, but I haven’t figured out how to do it.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [May 23, 2020, 6:18pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/2 "2020-05-23T18:18:16Z")

</div>

Hmm would this be related to your work @eviltrout?

---

<div class="post-metadata">

### Author: ![eviltrout](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/eviltrout/32/5275_2.png) [@eviltrout](https://meta.discourse.org/u/eviltrout)
#### Post date: [May 25, 2020, 2:08pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/3 "2020-05-25T14:08:22Z")

</div>

Which work were you thinking of? I don’t think anything I worked on recently would be likely to cause this.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [May 25, 2020, 5:12pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/4 "2020-05-25T17:12:56Z")

</div>

My bad I was thinking of ember-cli

---

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [May 25, 2020, 5:14pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/5 "2020-05-25T17:14:44Z")

</div>

Check `forum.example.com/sidekiq` - there might be some background reprocessing jobs going on after the upgrade.

---

<div class="post-metadata">

### Author: ![eviltrout](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/eviltrout/32/5275_2.png) [@eviltrout](https://meta.discourse.org/u/eviltrout)
#### Post date: [May 25, 2020, 5:20pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/6 "2020-05-25T17:20:53Z")

</div>

Highly unlikely it touched redis. I’ll never say never because I’ve been burned before but I suspect this is something else 🙂

---

<div class="post-metadata">

### Author: ![tormod](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tormod](https://meta.discourse.org/u/tormod)
#### Post date: [May 25, 2020, 6:09pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/7 "2020-05-25T18:09:45Z")

</div>

Cool hint. I didn’t see anything alarming there, no jobs running and none with long execution (only a few over 100ms). Dashboard says ~1000 processed per day, same as before the upgrade.

Probably off-topic, but interesting also the Processed graph over 6 months, seems like there are two plateaus (one twice as high as the other) that it toggles between with weeks between. It does not correlate with reboots (and we didn’t update this stuff in that time either).

Is there an easy way to change that 2s BRPOP timeout? I didn’t find it in the code. It would at least indicate whether the work polling loops are the problem.

---

<div class="post-metadata">

### Author: ![tormod](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tormod](https://meta.discourse.org/u/tormod)
#### Post date: [May 25, 2020, 6:30pm UTC](https://meta.discourse.org/t/higher-idle-process-activity-after-upgrade/152453/9 "2020-05-25T18:30:58Z")

</div>

Such are the things I see in top sometimes:

```
    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND                                                                     
   1024 discour+ 20 0 333584 202888 21660 S 1.3 2.5 3:35.81 ruby                                                                        
   1156 discour+ 20 0 715904 249416 24532 S 1.3 3.1 3:26.50 ruby                                                                        
   1178 discour+ 20 0 726664 251468 24564 S 1.3 3.1 3:26.11 ruby                                                                        
   1189 discour+ 20 0 714368 247912 24444 S 1.3 3.1 3:24.56 ruby                                                                        
   1200 discour+ 20 0 709760 249708 24632 S 1.3 3.1 3:22.96 ruby                                                                        
   1234 discour+ 20 0 713344 250288 24636 S 1.3 3.1 3:30.24 ruby                                                                        
   1167 discour+ 20 0 712832 247928 24436 S 1.0 3.1 3:24.75 ruby                                                                        
 188658 me 20 0 10424 4240 3576 R 0.7 0.1 0:00.36 top                                                                         
    448 root 20 0 1748444 84900 45884 S 0.3 1.1 6:09.98 dockerd                                                                     

```

with no one connected…

The 3.5 minutes accumulated CPU time each is after 35 hours up, with very little user activity.

I also tried to use rbtrace, but it complains:  
`(pid is not listening for messages, did you `require “rbtrace”`)`  
Do I need to rebuild the image to fix this? Or can I just adjust or rebuild something inside the container?
