2.6.0 beta 3 update failed on disk and/or memory space

[Paul_King]

I was prompted to update my 2.6.0 beta 2 site to beta 3, but the update process reportedly failed for me both times I tried it (disconcerting because I don’t recall any previous updates thus far throwing errors)

Weirdly, even part way through the update in progress, with the progress bar about half way through, the ‘Upgrade now’ button became highlighted again, even as the log below showed ongoing activity was occurring. The first time this happened, I absent mindedly clicked it, though hard to tell what effect this had.

Each time I found the process appeared to stall before the end, with error message ‘Update failed’

The log from 2nd attempt is pasted below

*** Please be patient, next steps might take a while ***

Cycling Unicorn, to free up memory
Restarting unicorn pid: 32161
Waiting for Unicorn to reload.
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Waiting for Unicorn to reload…
Stopping 1 Unicorn worker(s), to free up memory
Stopping job queue to reclaim memory, master pid is 15550
$ cd /var/www/discourse && git fetch --tags --force && git reset --hard HEAD@{upstream}
From [GitHub - discourse/discourse: A platform for community discussion. Free, open, simple.](https://github.com/discourse/discourse)
t [tag update] latest-release -> latest-release
HEAD is now at 6acd2ab7d4 UX: Allow horizontal scroll on mobile post buttons
$ cd /var/www/discourse/plugins/discourse-adplugin && git fetch --tags --force && git reset --hard HEAD@{upstream}
HEAD is now at a626b8e DEV: Add testing workflow
$ cd /var/www/discourse/plugins/discourse-bbcode && git fetch --tags --force && git reset --hard HEAD@{upstream}
HEAD is now at 093fc66 DEV: linting
$ cd /var/www/discourse/plugins/discourse-data-explorer && git fetch --tags --force && git reset --hard HEAD@{upstream}
HEAD is now at 4a167a7 DEV: attempts to fix indentation
$ cd /var/www/discourse/plugins/procourse-static-pages && git fetch --tags --force && git reset --hard HEAD@{upstream}
HEAD is now at 52f6f7c Merge pull request #17 from jbrw/master
$ bundle install --deployment --jobs 4 --without test development
[DEPRECATED] The `--deployment` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set deployment 'true'` , and stop using this flag
[DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'test development'` , and stop using this flag
Fetching gem metadata from https://rubygems.org/…
Using rake 13.0.1
Using concurrent-ruby 1.1.7
Using thread_safe 0.3.6
Using zeitwerk 2.4.0
Using builder 3.2.4
Using erubi 1.9.0
Using mini_portile2 2.4.0
Using crass 1.0.6
Using rack 2.2.3
Using mini_mime 1.0.2
Using aws-eventstream 1.1.0
Using aws-partitions 1.329.0
Using jmespath 1.4.0
Using ember-source 2.18.2
Using execjs 2.7.0
Using msgpack 1.3.3
Using bundler 2.1.4
Using byebug 11.1.3
Using cbor 0.5.9.6
Using chunky_png 1.3.12
Using coderay 1.1.3
Using colored2 3.1.2
Using connection_pool 2.2.3
Using openssl-signature_algorithm 1.0.0
Using cppjieba_rb 0.3.3
Using diffy 3.4.0
Using method_source 1.0.0
Using thor 1.0.1
Using discourse-ember-source 3.12.2.2
Fetching minitest 5.14.2
Fetching public_suffix 4.0.6
Fetching discourse-fonts 0.0.4
Installing minitest 5.14.2
Installing public_suffix 4.0.6
Fetching exifr 1.3.8
Using fspath 3.1.2
Using image_size 1.5.0
Using in_threads 1.5.4
Using progress 3.5.2
Using email_reply_trimmer 0.1.13
Using excon 0.76.0
Using multipart-post 2.1.1
Using fast_blank 1.0.0
Using fast_xs 0.8.0
Using fastimage 2.2.0
Using ffi 1.13.1
Using flamegraph 0.9.5
Using gc_tracer 1.5.1
Using guess_html_encoding 0.0.11
Using hashie 4.1.0
Using highline 2.0.3
Installing exifr 1.3.8
Using hkdf 0.3.0
Using htmlentities 4.3.4
Using http_accept_language 2.1.1
Using json 2.3.1
Using jwt 2.2.2
Using kgio 2.11.3
Using libv8 8.4.255.0 (x86_64-linux)
Using logstash-event 1.2.02
Fetching logster 2.9.4
Installing logster 2.9.4
Using lru_redux 1.1.0
Using lz4-ruby 0.3.3
Using maxminddb 0.1.22
Using memory_profiler 0.9.14
Fetching redis 4.2.2
Installing discourse-fonts 0.0.4
Installing redis 4.2.2
Using mini_sql 0.3
Using multi_json 1.15.0
Using multi_xml 0.6.0
Using mustache 1.1.1
Fetching nio4r 2.5.4
Installing nio4r 2.5.4 with native extensions
Using oauth 0.5.4
Using optimist 3.0.1
Using pg 1.2.3
Using r2 0.2.7
Using raindrops 0.19.1
Using rchardet 1.8.0
Using rinku 2.0.6
Using rotp 6.1.0
Using rqrcode_core 0.1.2
Using rtlit 0.0.5
Using rubyzip 2.3.0
Using tilt 2.0.10
Using sshkey 2.0.0
Using stackprof 0.2.15
Using unf_ext 0.0.7.7
Using seed-fu 2.3.9
Using jquery-rails 4.4.0
Using lograge 0.11.2
Fetching rails_failover 0.5.7
Fetching rails_multisite 2.5.0
Using sassc-rails 2.1.2
Fetching discourse-ember-rails 0.18.6
Installing rails_multisite 2.5.0
Installing rails_failover 0.5.7
Installing discourse-ember-rails 0.18.6
Bundle complete! 124 Gemfile dependencies, 161 gems now installed.
Gems in the groups test and development were not installed.
Bundled gems are installed into `./vendor/bundle`
$ bundle exec rake plugin:pull_compatible_all
procourse-static-pages is already at latest compatible version
discourse-merge-users is already at latest compatible version
discourse-subscriptions is already at latest compatible version
discourse-data-explorer is already at latest compatible version
docker_manager is already at latest compatible version
discourse-adplugin is already at latest compatible version
discourse-multiselect-user-field is already at latest compatible version
discourse-bbcode is already at latest compatible version
discourse-jitsi is already at latest compatible version
Discourse-nav-links-component is already at latest compatible version
$ SKIP_POST_DEPLOYMENT_MIGRATIONS=1 bundle exec rake multisite:migrate
Multisite migrator is running using 1 threads

Migrating default
== 20200810053843 CreateDataExplorerQueries: migrating ========================
– create_table(:data_explorer_queries)
-> 0.0879s
– create_table(:data_explorer_query_groups)
-> 0.0115s
– add_index(:data_explorer_query_groups, [:query_id, :group_id], {:unique=>true})
-> 0.0019s
== 20200810053843 CreateDataExplorerQueries: migrated (0.1271s) ===============

== 20200819021210 AddUserSelectableColumnToColorSchemes: migrating ============
– add_column(:color_schemes, :user_selectable, :bool, {:null=>false, :default=>false})
-> 0.0361s
== 20200819021210 AddUserSelectableColumnToColorSchemes: migrated (0.0368s) ===

== 20200819030609 MigrateUserTopicTimersToBookmarkReminders: migrating ========
== 20200819030609 MigrateUserTopicTimersToBookmarkReminders: migrated (0.0227s)

== 20200819203846 AddColorSchemeIdToUserOptions: migrating ====================
– add_column(:user_options, :color_scheme_id, :integer)
-> 0.0032s
== 20200819203846 AddColorSchemeIdToUserOptions: migrated (0.0037s) ===========

== 20200820174703 AddPartialTargetIdIndexToReviewables: migrating =============
– add_index(:reviewables, [:target_id], {:where=>“target_type = ‘Post’”, :algorithm=>:concurrently, :name=>“index_reviewables_on_target_id_where_post_type_eq_post”})
-> 0.0134s
== 20200820174703 AddPartialTargetIdIndexToReviewables: migrated (0.0142s) ====

== 20200902054531 AddFirstUnreadPmAToGroupUser: migrating =====================
– add_column(:group_users, :first_unread_pm_at, :datetime, {:null=>false, :default=>#<Proc:0x00007f5e2c725088@/var/www/discourse/db/migrate/20200902054531_add_first_unread_pm_a_to_group_user.rb:5 (lambda)>})
-> 0.0272s
– execute(“UPDATE group_users gu\nSET first_unread_pm_at = gu.created_at\n”)
-> 0.0720s
== 20200902054531 AddFirstUnreadPmAToGroupUser: migrated (0.1001s) ============

== 20200902082203 AddFirstUnreadPmAtToUserStats: migrating ====================
– add_column(:user_stats, :first_unread_pm_at, :datetime, {:null=>false, :default=>#<Proc:0x00007f5e2c742d90@/var/www/discourse/db/migrate/20200902082203_add_first_unread_pm_at_to_user_stats.rb:5 (lambda)>})
-> 0.0288s
– execute(“UPDATE user_stats us\nSET first_unread_pm_at = u.created_at\nFROM users u\nWHERE u.id = us.user_id\n”)
-> 0.0754s
== 20200902082203 AddFirstUnreadPmAtToUserStats: migrated (0.1051s) ===========

== 20200910020909 MakeImapSyncLogColsNotNull: migrating =======================
– change_column_null(:imap_sync_logs, :message, false)
-> 0.0011s
– change_column_null(:imap_sync_logs, :level, false)
-> 0.0003s
== 20200910020909 MakeImapSyncLogColsNotNull: migrated (0.0020s) ==============

== 20200910051633 ChangeUploadsVerifiedToInteger: migrating ===================
– add_column(:uploads, :verification_status, :integer, {:null=>false, :default=>1})
-> 0.0556s
== 20200910051633 ChangeUploadsVerifiedToInteger: migrated (0.2459s) ==========

== 20200911031738 AddIndexToUploadsVerificationStatus: migrating ==============
– execute(“CREATE INDEX CONCURRENTLY IF NOT EXISTS\nidx_uploads_on_verification_status ON uploads USING btree (verification_status)\n”)
-> 0.0076s
== 20200911031738 AddIndexToUploadsVerificationStatus: migrated (0.0082s) =====

== 20200916085541 CreateUserIpAddressHistories: migrating =====================
– create_table(:user_ip_address_histories)
-> 0.0086s
– add_index(:user_ip_address_histories, [:user_id, :ip_address], {:unique=>true})
-> 0.0034s
== 20200916085541 CreateUserIpAddressHistories: migrated (0.0125s) ============

Seeding default
*** Bundling assets. This will take a while ***
$ RUBY_GC_MALLOC_LIMIT_MAX=20971520 RUBY_GC_OLDMALLOC_LIMIT_MAX=20971520 RUBY_GC_HEAP_GROWTH_MAX_SLOTS=50000 RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0.9 bundle exec rake assets:precompile
Purging temp files
Bundling assets
I, [2020-09-26T04:15:34.191484 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/lt-aff59463076fd6741c6fec826e5bc3502630659e40a89a52385304d0d733f79e.js
I, [2020-09-26T04:15:34.831688 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/sw-1395ffdab0bd6fb90088df32240eac995374d095fa941b1c2c1151600ac690a0.js
I, [2020-09-26T04:15:35.334795 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/sv-e2c0bd16da62d25e1def2038c63f5c6a5691956299c8d1b2f2cd4e930d7570e3.js
I, [2020-09-26T04:15:36.112998 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/pt-1652f51dd6fac1cb7b5bd0faa78711b334158b553e3c7ab3ffa4380636f7ec0f.js
I, [2020-09-26T04:15:36.615626 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/es-2472cdc3c9ad70b1e74f54eac90d9418f9c8eace508a410ed67af7f521944549.js
I, [2020-09-26T04:15:37.639016 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/ru-c915653c8cf6115bf1e025ffbac13d4e705a1f715405b57759e3730d923a0251.js
I, [2020-09-26T04:15:38.008935 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/et-624e57018cbb201b8651dd25c8beb55b64ec70181a51cc5a34a4b98c6e29b8a5.js
I, [2020-09-26T04:15:39.174525 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/uk-bc59cb69648cd93765329edb21899195ec0e5ce929999087903281dcb8084697.js
I, [2020-09-26T04:15:40.235470 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/zh_CN-80e7ae3829c3a54c2b6cd230f0c43de5c8c9486d4bbce223c58b24333123515a.js
I, [2020-09-26T04:15:41.316997 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/da-51f3429bfb7e22d37494982fdcaef124a9b3c69d14afb04037f02276abb15e9d.js
I, [2020-09-26T04:15:42.401166 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/lv-f70a3961f0c2542ce855bb85b1810a4f47d1e8eb6c09bf5e5eeac238c5e2fd72.js
I, [2020-09-26T04:15:43.580012 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/he-bcb6d095f5b8a42b573cdd74c97028bfdcc447219657760d03da61e10093b0a5.js
I, [2020-09-26T04:15:44.716505 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/id-ada97e90f6c87be672e04aa85bfbd50b804f41c265ae7854aaf8bd02e12b140d.js
I, [2020-09-26T04:15:45.594078 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/sr-5bc40f5193d1f586aec28ca766eb94c7ed7e3791939d14f8087347f7de4cf1e6.js
I, [2020-09-26T04:15:46.244516 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/ja-2dfe206addb2e7069a2ee3f052b544d6ca588df14d82e819b0c4fa31feca9dd7.js
I, [2020-09-26T04:15:46.668071 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/en_US-98e1ee15d08bd830e804ac18f7aeec58a1e4689f21070d75440a5d6cf72fdbce.js
I, [2020-09-26T04:15:46.842460 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/en-d82bead5db4ee27e1698099fce258e58d16d3f6fb10901b9c5192c0de842f9b1.js
I, [2020-09-26T04:15:47.668330 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/ur-05dfafea819b54aa09ba584e043d0c30f25e87fcb8818951ab59f943359ebc75.js
I, [2020-09-26T04:15:48.398426 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/pl_PL-dff34fd819af2ae2a12fd315ec1f33d99e54f2d841d5bd2de1817495d5dd617d.js
I, [2020-09-26T04:15:49.087841 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/te-96f5c93e18f5ba7c1e1d7f04031aeedb10166aaacadedda0bbc50c9ecee9b765.js
I, [2020-09-26T04:15:49.994768 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/cs-8fa82bc7a9d51be5c76f490b54e9a45e9a657fc76ad4a943175871fba81f47e8.js
I, [2020-09-26T04:15:51.006513 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/fa_IR-f9897cc43dd1de36c163584a1b0f83be558c9f6b1daec6c8ccc349b7ec813fa1.js
I, [2020-09-26T04:15:52.051896 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/it-56d672ee509b6949ce9c53bf5f6fc0dc392a21c1466745dda6070a41849061dc.js
I, [2020-09-26T04:15:52.977508 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/ko-7ddfe94800616c59283238ec775b47f4d97c8851b93dec269ec4bb30d1f44374.js
I, [2020-09-26T04:15:54.004067 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/nl-8c40a317a6ff534078af7ace0280b52018da73a6b5ce9e244ee4683b4fad9603.js
I, [2020-09-26T04:15:54.925540 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/nb_NO-adcf93de3a7e6599ac92d0b998ac1a84e2dc8dd47ad13923027703445c2055c5.js
I, [2020-09-26T04:15:56.296000 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/th-8e2e7820508fa6f59c54d73aa51fb5b62f31c6ac53d38daca44d8284180a55d2.js
I, [2020-09-26T04:15:57.370523 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/ar-6d3800d74b50c32ee4be893960604b974097f1919d5a3f645714192dea225e0b.js
I, [2020-09-26T04:15:58.413891 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/gl-998870ca224ed5491ff7678f2ac390f5613029833caf8e5f655a5b936308007c.js
I, [2020-09-26T04:15:59.566612 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/sl-55b7b296aa9faad86b9370b2fc5289591c632cb396a6f835faceda9d19e091e1.js
I, [2020-09-26T04:16:00.476105 #20861] INFO – : Writing /var/www/discourse/public/assets/locales/hu-bc3fdd2c5161268dba71d58f87363776688eac5b6d7928458b0b13a3a9fece98.js
I, [2020-09-26T04:16:05.194849 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-adplugin-5d2f3eaacb0cdc2555b2a299fcb86eedbf23e2844628a623d4ec20ec907bcf84.js
I, [2020-09-26T04:16:05.747380 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-bbcode-1082e092e50e283df15df91614bccdf621e6342f98ec2e877bbfd58ed56e6b86.js
I, [2020-09-26T04:16:07.947593 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-data-explorer-10f42136f8ee7c543dc16393ee33d1587add4b0832d82e2e868fa50d7cd4da17.js
I, [2020-09-26T04:16:08.077099 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-details-0bd38e4002b071b9455a7fdd1038802c31d7226032c566be9ff26f168d23cefc.js
I, [2020-09-26T04:16:09.506385 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-local-dates-747acffabe6d8ac2bd99774aab5ca1cf227100c8db167286796368501e13c863.js
I, [2020-09-26T04:16:09.733285 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-merge-users-ba6ca6eef6e563e2b4936b343c09bef16dff611e867702773b840c2e283d1ce5.js
I, [2020-09-26T04:16:09.829293 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-multiselect-user-field-b338bd25aae99fd968ca39829ea166a20d381671236ecdab04c1e89f02960392.js
I, [2020-09-26T04:16:09.889116 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-narrative-bot-c43db87f1004dffa72bf81db5bfe561550f830b224352505680dde210d6ec6e1.js
I, [2020-09-26T04:16:10.355973 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-presence-550080222a76cbecd779c71e12e4088ff7bed2880db527c159342ccd66ccfc1c.js
I, [2020-09-26T04:16:15.254140 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/discourse-subscriptions-e80d52ef37a11a7026eb620965da785034738cbd2d69fcc2c140527fc3562a94.js
I, [2020-09-26T04:16:15.358506 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/docker_manager-4ced9f9c7bda9ba563c04006dbb93828d43f2c7d5b533065717c5a63da77d5fc.js
I, [2020-09-26T04:16:15.470571 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/lazy-yt-13878a92cc5d5039d6f32688a86500182843ceac829df20e96639d8026a65a8a.js
I, [2020-09-26T04:16:17.869982 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/poll-1d66ca8ab4ee4c249469ae3d51b7176593b240a8fc7a6b2fe2b416a6f7cc7a0d.js
I, [2020-09-26T04:16:18.329498 #20861] INFO – : Writing /var/www/discourse/public/assets/plugins/procourse-static-pages-0299b14a74e8a98373f3d5b5bf47f3e24e8218698628c8fa6b24a976f2367541.js
rake aborted!
MiniRacer::ScriptTerminatedError: JavaScript was terminated (either by timeout or explicitly)
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/mini_racer-0.3.1/lib/mini_racer.rb:209:in `eval_unsafe' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/mini_racer-0.3.1/lib/mini_racer.rb:209:in ` block (2 levels) in eval’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/mini_racer-0.3.1/lib/mini_racer.rb:350:in `timeout' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/mini_racer-0.3.1/lib/mini_racer.rb:208:in ` block in eval’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/mini_racer-0.3.1/lib/mini_racer.rb:206:in `synchronize' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/mini_racer-0.3.1/lib/mini_racer.rb:206:in ` eval’
/var/www/discourse/lib/discourse_js_processor.rb:134:in `block in perform' /var/www/discourse/lib/discourse_js_processor.rb:127:in ` synchronize’
/var/www/discourse/lib/discourse_js_processor.rb:127:in `perform' /var/www/discourse/lib/discourse_js_processor.rb:30:in ` transpile’
/var/www/discourse/lib/discourse_js_processor.rb:17:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:75:in ` call_processor’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:57:in `block
 in call_processors' 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:56:in
 ` reverse_each’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:56:in `call_processors' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:134:in ` load_from_unloaded’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:60:in `block in load' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:317:in ` fetch_asset_from_dependency_cache’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:44:in `load' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:20:in ` block in initialize’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:47:in `load' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/bundle.rb:23:in ` block in call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/utils.rb:200:in `dfs' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/bundle.rb:24:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:75:in `call_processor' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:57:in ` block in call_processors’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:56:in `reverse_each' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:56:in ` call_processors’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:134:in `load_from_unloaded' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:60:in ` block in load’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:317:in `fetch_asset_from_dependency_cache'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/loader.rb:44:in
 ` load’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:20:in `block in initialize' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:47:in ` load’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/base.rb:66:in `find_asset' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/base.rb:73:in ` find_all_linked_assets’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:134:in `block in find' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:133:in ` each’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:133:in `find' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:186:in ` compile’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-rails-3.2.2/lib/sprockets/rails/task.rb:67:in `block (3 levels) in define' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-3.7.2/lib/rake/sprocketstask.rb:147:in ` with_logger’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sprockets-rails-3.2.2/lib/sprockets/rails/task.rb:66:in `block (2 levels) in define' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/exe/rake:27:in ` <top (required)>’
/usr/local/bin/bundle:23:in `load' /usr/local/bin/bundle:23:in ` ’
Tasks: TOP => assets:precompile
(See full trace by running task with --trace)
Docker Manager: FAILED TO UPGRADE
#<RuntimeError: RuntimeError>
/var/www/discourse/plugins/docker_manager/lib/docker_manager/upgrader.rb:178:in `run' /var/www/discourse/plugins/docker_manager/lib/docker_manager/upgrader.rb:86:in ` upgrade’
/var/www/discourse/plugins/docker_manager/app/controllers/docker_manager/admin_controller.rb:95:in `block in upgrade' /var/www/discourse/plugins/docker_manager/app/controllers/docker_manager/admin_controller.rb:91:in ` fork’
/var/www/discourse/plugins/docker_manager/app/controllers/docker_manager/admin_controller.rb:91:in `upgrade'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal/basic_implicit_render.rb:6:in
 ` send_action’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/abstract_controller/base.rb:195:in `process_action'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal/rendering.rb:30:in
 ` process_action’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/abstract_controller/callbacks.rb:42:in `block
 in process_action' 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.2/lib/active_support/callbacks.rb:135:in
 ` run_callbacks’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/abstract_controller/callbacks.rb:41:in `process_action' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal/rescue.rb:22:in ` process_action’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal/instrumentation.rb:33:in `block
 in process_action' 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.2/lib/active_support/notifications.rb:180:in
 ` block in instrument’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.2/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.2/lib/active_support/notifications.rb:180:in ` instrument’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal/params_wrapper.rb:245:in
 ` process_action’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.3.2/lib/active_record/railties/controller_runtime.rb:27:in `process_action' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/abstract_controller/base.rb:136:in ` process’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionview-6.0.3.2/lib/action_view/rendering.rb:39:in `process' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-mini-profiler-2.0.4/lib/mini_profiler/profiling_methods.rb:78:in ` block in profile_method’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal.rb:190:in `dispatch' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_controller/metal.rb:254:in ` dispatch’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/route_set.rb:50:in `dispatch' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/route_set.rb:33:in ` serve’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/mapper.rb:18:in `block
 in <class:Constraints>' 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/mapper.rb:48:in
 ` serve’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/journey/router.rb:49:in `block in serve' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/journey/router.rb:32:in ` each’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/journey/router.rb:32:in `serve' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/route_set.rb:834:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/engine.rb:527:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/railtie.rb:190:in ` public_send’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/railtie.rb:190:in `method_missing' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/mapper.rb:19:in ` block in class:Constraints’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/mapper.rb:48:in `serve' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/journey/router.rb:49:in ` block in serve’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/journey/router.rb:32:in `each' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/journey/router.rb:32:in ` serve’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/routing/route_set.rb:834:in `call' /var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:68:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/conditional_get.rb:40:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call' /var/www/discourse/lib/content_security_policy/middleware.rb:12:in ` call’
/var/www/discourse/lib/middleware/anonymous_cache.rb:336:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in ` context’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/cookies.rb:648:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.2/lib/active_support/callbacks.rb:101:in ` run_callbacks’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/actionable_exceptions.rb:17:in
 ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/show_exceptions.rb:33:in
 ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/logster-2.9.3/lib/logster/middleware/reporter.rb:43:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/rack/logger.rb:37:in ` call_app’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/rack/logger.rb:28:in `call' /var/www/discourse/config/initializers/100-quiet_logger.rb:19:in ` call’
/var/www/discourse/config/initializers/100-silence_logger.rb:31:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/remote_ip.rb:81:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/request_id.rb:27:in `call' /var/www/discourse/lib/middleware/enforce_hostname.rb:22:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/executor.rb:14:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
 
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.2/lib/action_dispatch/middleware/host_authorization.rb:76:in
 ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-mini-profiler-2.0.4/lib/mini_profiler/profiler.rb:200:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-3.3.1/lib/message_bus/rack/middleware.rb:61:in ` call’
/var/www/discourse/lib/middleware/request_tracker.rb:176:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/engine.rb:527:in ` call’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/railtie.rb:190:in `public_send' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.2/lib/rails/railtie.rb:190:in ` method_missing’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in ` each’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.6.0/lib/unicorn/http_server.rb:632:in ` process_client’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.6.0/lib/unicorn/http_server.rb:728:in `worker_loop' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.6.0/lib/unicorn/http_server.rb:548:in ` spawn_missing_workers’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.6.0/lib/unicorn/http_server.rb:562:in `maintain_worker_count' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.6.0/lib/unicorn/http_server.rb:295:in ` join’
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.6.0/bin/unicorn:128:in `<top (required)>' /var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in ` load’
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `’
Spinning up 1 Unicorn worker(s) that were stopped initially

After the 2nd apparently failed attempt, on clicking the ‘back to site link’ in the update page, I was first met with error ‘502 bad gateway’ error, but then refreshing browser, the site reappeared normally.

Going back to admin/update settings, I saw the site now reports itself as up to date.

I now don’t know whether I have a corrupted site or not, especially given the reported error reference in the log:

Docker Manager: FAILED TO UPGRADE
#<RuntimeError: RuntimeError> 

How do I tell?

Superficially everything seems to be working as I navigate around my site. The software, all plugins and even Docker Manager all now reported as up to date.

Is there any sort of site/verification utility available that would help allay fears of corruption?

Even if all is ok however, this upgrade process is probably not the most confidence inspiring user experiance for the average site administrator. Errors and ‘upgrade failed’ reports aside, basic things like the progress bar indicator apparently not actually corresponding to progress, and the ‘upgrade now’ button reactivating part way through upgrading are a worry.

[Don]

Hi Paul,

Please remove the log texts from quoting.

Just simple select all your log texts and press this button in composer, after that you can check the result on preview. It will show your log with fix height and make it much readable for everyone.

[Paul_King]

Thanks - I wondered what those </> were for. Much better!

[jks (Jouni Seppänen)]

I saw some weirdness recently: I upgraded via the …/admin/upgrade URL last night, and the site seemed to be up when I tried, but I got many emails from Uptime Robot overnight. The site seemed to be occasionally returning 502.

I ran ./launcher rebuild app fifteen minutes ago and haven’t seen any errors since, but the problem seemed to be somewhat random so I can’t really say if that helped.

[bletch (Danny Goodall)]

Have you tried updating via the shell?

Assuming you have a vanilla installation…

cd /var/discourse
./launcher rebuild app

[IAmGav (Gavin Perch)]

hey @Paul_King,

I would recommend doing this upgrade via the command line instead of using the web interface.

once you logged into the command line, here are the cmds I used to upgrade

1. apt-get update

2. apt-get upgrade

3. cd /var/discourse
./launcher rebuild app

4. ./launcher cleanup (this cleans up old unused files)

[Paul_King]

@bletch

cd /var/discourse
./launcher rebuild app

Possible clue – when I tried this from my Digital Ocean Droplet console, I got error “you have less then 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue". When prompted to recover space, I get the below:

Digital Ocean control panel reports the droplet is only using 1GB out of the 25 allocated (Discourse is the only installed software, as far as I know), and while I have my droplet set to back up weekly via the paid Digital Ocean service for this, I understand no space in the droplet itself is used. I also have no snapshots stored in my droplet.

Four questions:

1. Why does the rebuild app command conclude I have used 21 out of 25GB if Digital Ocean disagrees?
2. What could possibly be filling 21GB with a 7 month old forum with 700 users, very sporadic usage, and minimal file uploading (the very occasional small image included in a post)?
3. Why is a clear 5GB disk space even a requirement for rebuild app? Surely Discourse is not that big?
4. Could this suggest a bug?

[IAmGav (Gavin Perch)]

@IAmGav

./launcher cleanup (this cleans up old unused files)

i would recommend using this often just after you normally rebuild.

use that command now, before you rebuild again to clear up space

[Ed_S]

@Paul_King

• What could possibly be filling 21GB with a 7 month old forum with 700 users, very sporadic usage, and minimal file uploading (the very occasional small image included in a post)?

You can try to find out, for example at your root shell prompt:
du -kx / | sort -n | tail -33
and perhaps also
find / -xdev -size +1000k -ls | sort -n -k 2 | tail

[Paul_King]

OK, du command (=disk usage?) reports

root@nz:/var/discourse# du -kx / | sort -n | tail -33
747432  /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5/diff/var/www/discourse/vendor/bundle
750792  /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5/diff/var/www/discourse/vendor
819620  /var/log/journal/8bebc832e1a692c83690ffe65e1256e3
860600  /var/log/journal
926492  /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354/diff/var/www/discourse
926504  /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354/diff/var/www
998192  /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354/diff/var
1059968 /var/log
1202004 /var/discourse/shared/standalone/import/data
1237720 /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354/diff/usr
1307816 /var/discourse/shared/standalone/import
1399332 /var/discourse/shared/standalone/backups/default
1399336 /var/discourse/shared/standalone/backups
1636032 /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5/diff/var/www/discourse
1636040 /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5/diff/var/www
1678944 /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5/diff/var
1692708 /usr
1820540 /var/lib/docker/volumes
2235084 /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5/diff
2235104 /var/lib/docker/overlay2/794d92242be8354a3ba468161d1d2ea416ab43a6f70ddd7ab060384bd5a9fcd5
2285628 /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354/diff
2285644 /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354
2437984 /var/discourse/shared/standalone/postgres_data/base/16384
2461228 /var/discourse/shared/standalone/postgres_data/base
2545304 /var/discourse/shared/standalone/postgres_data
5184184 /var/lib/docker/overlay2
5692972 /var/discourse/shared/standalone
5693056 /var/discourse/shared
5695776 /var/discourse
7076692 /var/lib/docker
7446508 /var/lib
14417296        /var
18614840        /
root@nz:/var/discourse#

18.6 GB?

The find command gives
    root@nz:/var/discourse# find / -xdev -si                                                                      ze +1000k -ls | sort -n -k 2 | tail
       524219  217044 -rw-------   1 lxd      mlocate          222248960 May 22 03:39 /var/discourse/shared/standalone/postgres_data/base/16384/20379
       523733  234132 -rw-------   1 lxd      mlocate          239747072 May 23 01:29 /var/discourse/shared/standalone/postgres_data/base/16384/19615
       610933  244612 -rw-------   1 lxd      mlocate          250478592 Sep 25 03:37 /var/discourse/shared/standalone/postgres_data/base/16384/153088
       610816  248180 -rw-------   1 lxd      mlocate          254132224 Sep 26 03:59 /var/discourse/shared/standalone/postgres_data/base/16384/152767
      2596079  307188 -r--r--r--   1 ReadyNAS ReadyNAS  314556297 Apr 29 20:50 /var/lib/docker/overlay2/fadc3460a6249ba37efa266d885f5ea3d6f6de8fee0e9b056c5376599bb7f354/diff/var/www/discourse/.git/objects/pack/pack-ffd1b8da21b9e26b4475a3fef6537a89f21989d6.pack
      1548730  464076 -rw-r--r--   1 ReadyNAS www-data         475209681 Sep 24 03:31 /var/discourse/shared/standalone/backups/default/nz-architecture-2020-09-24-033014-v20200820232017.tar.gz
      1548729  467616 -rw-r--r--   1 ReadyNAS www-data         478832984 Sep 25 03:38 /var/discourse/shared/standalone/backups/default/nz-architecture-2020-09-25-033723-v20200820232017.tar.gz
      1548294  467636 -rw-r--r--   1 ReadyNAS www-data         478853932 Sep 26 03:54 /var/discourse/shared/standalone/backups/default/nz-architecture-2020-09-26-035209-v20200820232017.tar.gz
       794198  627936 -rw-r--r--   1 ReadyNAS ReadyNAS         643002368 Mar 28  2020 /var/discourse/shared/standalone/import/data/index.db
        60454 2097156 -rw-------   1 root     root     2147483648 Jan  4  2020 /swapfile
    root@nz:/var/discourse#

Does anything stand out as unexpectedly large ?
I have no way to tell what is normal, and what if anything can be safely deleted…

(the ReadyNAS files are forum backups, generated by Discourse)

[riking (Kane York)]

@Paul_King

2545304

Database is 2.5GB. Backups combined are 1.3GB.

You’ve got several large Docker layers - are you sure that ./launcher cleanup actually completed?

[Paul_King]

./launcher cleanup gives following result for me, almost instantaneously - not sure what I am missing here…

root@nz:/var/discourse# ./launcher cleanup
WARNING! This will remove all stopped containers.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0B
WARNING! This will remove all images without at least one container associated to them.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0B
root@nz:/var/discourse#

[Ed_S]

That large import area might be suspicious. But it’s only 640M.

There are some tips here about seeing what docker is using, and trying to reduce it. (It’s possible some of the advice is wrong or dangerous.) I saw this in my rather small world:

# docker volume ls -qf dangling=true
# docker images -a
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
local_discourse/app   latest              33ce56b34841        3 months ago        2.59GB
<none>                <none>              991acdba0b1f        4 months ago        2.22GB
# docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              1                   1                   2.591GB             0B (0%)
Containers          1                   1                   920.9MB             0B (0%)
Local Volumes       0                   0                   0B                  0B
Build Cache         0                   0                   0B                  0B

[Ed_S]

Did you see this advice about vacuuming your database and rebuilding indexes after the recent postgresql update?

[Paul_King]

Thanks Ed - both those things helped

@Ed_S

There are some tips here about seeing what docker is using, and trying to reduce it.

Most posts in that thread did not help, but one suggested docker system prune --all --volumes --force which cleaned more than 1GB

@Ed_S

Did you see this advice about vacuuming your database and rebuilding indexes after the recent postgresql update?

That procedure reclaimed 2.3GB - enough to give me the minimum 5GB

I then repeated the first two commands of

@IAmGav

• apt-get update
• apt-get upgrade
• cd /var/discourse
./launcher rebuild app
• ./launcher cleanup (this cleans up old unused files)

But the process seemed to fill things back up again, because at the third command I got

root@nz:/var/discourse# ./launcher rebuild app
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        25G   22G  3.0G  88% /
 
Would you like to attempt to recover space by cleaning docker images and containers in the system?(y/N)y
If the cleanup was successful, you may try again now

At that point I tried ./launcher cleanup again, but got

root@nz:/var/discourse# /launcher cleanup
-bash: /launcher: No such file or directory

[Ed_S]

(That should be ./launcher not /launcher)

[Paul_King]

@Ed_S

(That should be ./launcher not /launcher)

Ah - thanks Ed! That got back 2.6GB - just enough to let installation complete.

Many thanks to everyone for the helpful suggestions.

It still worries me a bit that I am apparently using so much space, at least as reported with Ubuntu - because future forum updates may presumably hit the same issue.

I note within my Discourse installation, free disk space available for storing backups is reported as 9 GB - a major discrepancy. What could lie behind that?

** Edit ** - retrying du -kx / | sort -n | tail -33 I see Discourse is correct - my disk usage has shrunk to 15GB as a result of all of the above

[IAmGav (Gavin Perch)]

if you go to yourdomain.com/admin/backups

How many backups do you have there? maybe you can keep one and delete the rest.

[Paul_King]

@IAmGav

How many backups do you have there?

Only 3 backups now @ 457 MB each - but looks like disk space headroom problem has resolved through the various clean ups… for now at least.

I read in passing something about dockers accumulating ‘difference data’ and logs at a ferocious rate, and a lot of people reporting similar issues with disks filling themselves up

[Ed_S]

I would agree that this following message is worrying, and it’s odd that discourse believes it is fully updated:

@Paul_King

Tasks: TOP => assets:precompile
(See full trace by running task with --trace)
Docker Manager: FAILED TO UPGRADE
#<RuntimeError: RuntimeError>