I have seen substantial increase in CPU usage since upgrading this weekend. The CPU usage of RUBY appears to be the primary driver. This was referenced by another discourse user in this topic. As you can see from the graphs below, CPU usage and load pre-upgrade were much lower than post-upgrade. The upgrade occurred on the evening of 1/31. [The image shows graphs depicting CPU and Load percentage from 27 January to 2 February. (Captioned by AI)] Here are two images of TOP taken 33 hours apart: [TOP data from 2/1 @ 1:00pm CT] [TOP data from 2/2 @ 9:45pm CT.] In 33 hours, there is a significant ruby CPU usage. Based on the top data, I have seen 2x CPU usage in the last 33 hours over 22 days. In 33 hours, I have seen 11 hours of CPU time. (648 minutes of CPU time across 5 PIDs) Additional Data: Traffic has been down over the last two days by about 10%. (analytics and dashboard) Standard single container discourse install (no-chat) Sidekiq queues are minimal (1K to 2K per day) Nothing seems unusual in the discourse logs I run on a DO server with 8GB RAM and 2 AMD vCPUs. This isn’t the case where the server is down critical, but servers that run at 5% to 7% are much happier than those that run at 25%. What info can I provide to assist in troubleshooting this issue? tia

OK this is reverted now, https://github.com/discourse/discourse/pull/31119 Let me know if the commit restores performance.

Lets leave this in support for a bit till we determine if there is a bug. Can you enter the container and run an htop from the inside (you will have to install it) that way you will be able to tell which specific process is consuming high amounts of cpu. You can get a bit more visibility using a technique like this: Debugging 100% CPU usage in production Ruby on Rails systems Most likely though, is sidekiq /sidekiq somehow overloaded on your instance. (I would look a schedular particularly)

htop views. Here is a 30 second video: Random Screen shots: [The image displays a Unix-like system’s process list or task manager showing various running processes, with a focus on multiple instances of 'unicorn worker' processes, and provides specific details such as user ID, process ID, priority, CPU, memory usage, and estimated runtime. (Captioned by AI)] [This image displays a Unix-like system's process and CPU usage output, featuring multiple worker processes associated with Chrome, javascript, and Redis, along with details such as PID, user, and percent CPU usage. (Captioned by AI)] [This image is a screenshot of a Unix system's process list showing various processes, primarily related to a Redis server, with details such as PID, user, priority, VIRT, and other system-specific information. (Captioned by AI)] [This image is a screenshot of a Unix-like operating system command-line terminal displaying a running "discourse" process output with various process details, system information, and commands executed. (Captioned by AI)] Tree View: [The image shows a section of server logs with timestamps, commands, and user actions, alongside repeated lines in green and red text indicating processes such as "sdekiq" and "unicorn worker" interacting with "discourse" and related configuration commands. (Captioned by AI)] [The image shows a screenshot of a Linux system using the top command, displaying processes and system metrics. (Captioned by AI)] sidekiq: [This image is a screenshot of a queue management dashboard displaying a processed count, a few failed images, and queues with detailed information like busyness, enqueued, retries, scheduled, and dead images, along with a delete action for each queue. (Captioned by AI)] [The line chart shows the progression of an image captioning task over several days, marked by vertical markers indicating failed attempts at different times. (Captioned by AI)] [This image shows a server panel for a process named "discourse-app:817" running with 0% utilization, 461 MB of RSS, and it can be paused or stopped using the provided buttons. (Captioned by AI)] Let me know if there is something specific you need to see. I

Yeah something is off: top -H -p PID_OF_UNICORN [A screenshot showing a top command output indicating system resources and processes, with 367 threads, predominantly running a ruby or colourer-2480 process, and a mostly free but used RAM usage. (Captioned by AI)] I suspect you will see V8 DefaultWorker there, I think this is a regressing in mini_racer… will revert it to see if it resolves this.

Yes, it resolved the high CPU issue. My 1-minute and 5-minute load is about 1/3 of the previous values. That is with htop and netdata now running on the system. HTOP video Do Graph [The image shows two line graphs depicting CPU percentage and load (1/5/15) over various time intervals, with different lines representing different loads (0.100, 1.000, 15.000, and 50.150). (Captioned by AI)] I would expect the CPU usage and load to slowly decrease as more database queries are cached in the system. Load Table: time Pre-fix post-fix 1 min 0.4 to 0.6 0.06 to 0.1 5 min 0.39 to 0.5 0.15 to 0.18 The 15-minute metric is impacted by a rebuild. I will post some more stats later this morning. Thank you for the late-night fix.

Sorry about this, the mini_racer upgrade has been a big adventure. Hopefully we get through it soon.

Thank you for the fast response to investigate. I am sure you had other things planned for the day versus a rollback. As a new Discourse user, 2 weeks since migration, the product has been great to work with.

Similar story here aswell. [The image displays a detailed graph with two charts: the upper chart is a colorful line graph showing CPU usage over time, highlighted with a significant peak on January 27th, marked with CPU usage of 2.03%, alongside the lower chart indicating system load in a line graph format, featuring negligible... (Captioned by AI)] [Edit: seems to be fixed now after updating to latest branch]

Here is a performance review 18 hours after the rebuild. The load table says it all. Load Table: time Pre-fix post-fix 1 min 0.4 to 0.6 0.03 to 0.05 5 min 0.39 to 0.5 0.09 15 min 0.68 0.12 Legend: Red arrow - rebuilt the app Purple arrow - turned off netdata [The image shows two graphs: one with a red arrow indicating high CPU usage and another with a purple arrow showing spikes in load, both occurring at different times during the day. (Captioned by AI)]

Note, closing the loop, the bug causing it was this: https://github.com/rubyjs/mini_racer/commit/edaa9e862abcd9945fbfcdd115d02dd7326569cf I updated the gem. One immediate advantage is that it appears this version of v8 uses slightly less memory which is nice.

I installed the latest release last night with the fix. CPU usage is back to historic levels. Thank you for all the great work.

עלייה בשימוש במעבד מאז עדכון 3.4.0.beta4-dev (58f75ed205)

תמיכה

sam (Sam Saffron) 3 בפברואר,‏ 2025,‏ 5:41am 4

Yeah something is off:

top -H -p PID_OF_UNICORN

I suspect you will see V8 DefaultWorker there, I think this is a regressing in mini_racer… will revert it to see if it resolves this.

נושא		תגובות	צפיות
CPU usage increases steadily Product Support	14	390	8 במרץ,‏ 2025
Higher idle process activity after upgrade Self-hosting	7	888	25 במאי,‏ 2020
High CPU usage (Ruby) Product Support server-resources	20	1142	3 בפברואר,‏ 2025
Server Performance Issues Due to High CPU Usage Product Support	5	457	23 בפברואר,‏ 2025
Strange CPU usage since latest upgrade Self-hosting server-resources	14	1230	11 באפריל,‏ 2019

עלייה בשימוש במעבד מאז עדכון 3.4.0.beta4-dev (58f75ed205)

נושאים קשורים