Hi @david ,
Does the memory usage subside as soon as the assets:build task is done?
Nope. I digged further and there is something strange.
Before precompiling:build, this is the list of plugins:
/var/www/discourse$ ls plugins/
automation discourse-akismet discourse-data-explorer discourse-hcaptcha discourse-microsoft-auth discourse-post-voting discourse-saved-searches discourse-user-notes styleguide
chat discourse-apple-auth discourse-details discourse-lazy-videos discourse-narrative-bot discourse-presence discourse-solved discourse-zendesk-plugin
checklist discourse-assign discourse-docs discourse-local-dates discourse-oauth2-basic discourse-prometheus discourse-subscriptions footnote
discourse-adplugin discourse-cakeday discourse-gamification discourse-login-with-amazon discourse-openid-connect discourse-reactions discourse-templates msgraph-poll-discourse-plugin
discourse-affiliate discourse-calendar discourse-github discourse-lti discourse-patreon discourse-rewind discourse-topic-trade-buttons poll
discourse-ai discourse-chat-integration discourse-graphviz discourse-math discourse-policy discourse-rss-polling discourse-topic-voting spoiler-alert
I’ve noticed the following behaviour after debugging the code.
/var/www/discourse$ script/rails runner "AssetProcessor.ember_version"
Plugin name is 'msgraph-polling', but plugin directory is named 'msgraph-poll-discourse-plugin'
# hangs here forever
The AssetProcessor.ember_version corresponds to the line discourse/lib/plugin/js_manager.rb at latest · discourse/discourse · GitHub
So I did some modifications to this file (attached) , basically to print where it stucks while processing, and to remove the AssetProcessor.ember_version in discourse/lib/plugin/js_manager.rb at latest · discourse/discourse · GitHub , just putting 5 to continue the hex generation.
Then, I reduced parallelism to 1 to ease things in discourse/lib/plugin/js_manager.rb at latest · discourse/discourse · GitHub (parallel_count = [Etc.nprocessors, 1].min).
After this, I executed bundle exec rake assets:precompile:build, resulting in the following:
/var/www/discourse$ bundle exec rake assets:precompile:build
Plugin name is 'msgraph-polling', but plugin directory is named 'msgraph-poll-discourse-plugin'
[assemble_ember_build] Reusing existing core ember build. All done.
Plugin name is 'msgraph-polling', but plugin directory is named 'msgraph-poll-discourse-plugin'
[Plugin::JsManager] Compiling 49 plugins...
Compiling automation...
end of files.sort
end of files.sort
hex_digest 103dc9ebebb80a7065cb8dd41fb3356b30f151f7
########### recursive
# hang here forever, consuming the full memory
I have my suspicious that it may be related to the ulimit value (that we have set to unlimited instead to something like ulimit -n 1048576;), given the fact that you are opening quite a lot number of files and storing its content in memory (via the recursive calls).
Let me know if this rings you a bell, or you have any other hint about what may be the issue.
Cheers,
Ismael
js_manager.rb.txt (7.7 KB)