Upgrade of docker manager beta7 >> beta9 fails

Thomas_Rother · January 5, 2020, 9:42am

Hi,
I have a problem during upgrade to the latest discourse version (2.4.0 beta7 >> beta9). The docker manager upgrade hangs repeatedly at the following step:

/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:605:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:700:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:548:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:144:in `start'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.1/bin/unicorn:128:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `<main>'
Spinning up 1 Unicorn worker(s) that were stopped initially

Re-starting and resetting the upgrade has no effect. Any ideas?

Thanks, Thommie

itsbhanusharma · January 5, 2020, 10:47am

Please SSH into Your server and rebuild.

cd /var/discourse
git pull
./launcher rebuild app

Thomas_Rother · January 5, 2020, 11:52am

OK, that helped, but now I have a discrepancy between the information in the linux shell and in the UI of the Discourse container:

The admin page says: Installed 2.4.0.beta7 and latest 2.4.0.beta9, although the /admi/upgrade page shows “up to date” for discourse (latest-release +88), docker manager and plugins.

/var/discourse/container/app.yml contains different e-mail settings than those displayed in the Discourse UI.

It looks as if something fails during the rebuild …

Thommie

itsbhanusharma · January 5, 2020, 11:53am

Did you receive any errors during rebuild?
If so, sharing those will be really helpful in understanding what went wrong.

Thomas_Rother · January 5, 2020, 12:08pm

OK, I repeated the rebuild again, but I see no obvious errors during rebuild in the logs, only messages with INFO level. The last lines are:

2020-01-05 12:01:12.911 UTC [53] LOG:  shutting down
166:M 05 Jan 2020 12:01:13.087 * DB saved on disk
166:M 05 Jan 2020 12:01:13.087 # Redis is now ready to exit, bye bye...
2020-01-05 12:01:13.669 UTC [49] LOG:  database system is shut down
sha256:da70fb2de1f09e179e7b603a2e3aa17140229011b7af40b28eea4615d3fceeb6
9e56de01e8b7c16e0e24c1221ab32cf161b6b9ecd2e9bfd6f722f5226498d3cc
Removing old container
+ /usr/bin/docker rm app
app

+ /usr/bin/docker run --shm-size=512m -d --restart=always -e LANG=de_DE.UTF-8 -e RAILS_ENV=production -e UNICORN_WORKERS=4 -e UNICORN_SIDEKIQS=1 -e RUBY_GLOBAL_METHOD_CACHE_SIZE=131072 -e RUBY_GC_HEAP_GROWTH_MAX_SLOTS=40000 -e RUBY_GC_HEAP_INIT_SLOTS=400000 -e RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=1.5 -e DISCOURSE_DB_SOCKET=/var/run/postgresql -e DISCOURSE_DB_HOST= -e DISCOURSE_DB_PORT= -e LETSENCRYPT_DIR=/shared/letsencrypt -e DISCOURSE_HOSTNAME=forum.netzwissen.de -e DISCOURSE_DEVELOPER_EMAILS=admin@netzwissen.de -e DISCOURSE_SMTP_ADDRESS=10.10.10.19 -e DISCOURSE_SMTP_PORT=25 -e DISCOURSE_SMTP_USER_NAME= -e DISCOURSE_SMTP_PASSWORD= -e LETSENCRYPT_ACCOUNT_EMAIL=me@example.com -h develd-app -e DOCKER_HOST_IP=172.17.0.1 --name app -t -p 80:80 -p 443:443 -v /var/discourse/shared/standalone:/shared -v /var/discourse/shared/standalone/log/var-log:/var/log --mac-address 02:b7:65:5c:bd:c6 local_discourse/app /sbin/boot
7201804f7d5312ceee286af3a5e7db6e3e57a5327504ce752252f238aa9a4e98

Which looks normal as far as I can guess … The e-mail parameters shown here are correct and reflect those from the app.yml. But the UI still shows old data …

itsbhanusharma · January 5, 2020, 12:13pm

This seems absolutely normal to me. I can’t think of a reason why the UI would show outdated data.

Thomas_Rother · January 5, 2020, 2:44pm

… yep, I dont understand it either. The container still uses old config data for sending mail (which are thus not delivered), although the last lines of the rebuild process show the correct target smtp server:

/usr/bin/docker run -[…]
DISCOURSE_SMTP_ADDRESS=10.10.10.19 -e DISCOURSE_SMTP_PORT=25 -e
DISCOURSE_SMTP_USER_NAME= -e DISCOURSE_SMTP_PASSWORD= -e

This means for me that the update failed and we are still running an older version in the docker container plus an older configuration from app.yml - although the update page tells a different story:

Any ideas how to fix this?

tophee · January 8, 2020, 3:17pm

I’m having the same problem on two discourse instances.

The first error seems to be this one:

*** Bundling assets. This will take a while *** $ RUBY_GC_MALLOC_LIMIT_MAX=20971520 RUBY_GC_OLDMALLOC_LIMIT_MAX=20971520 RUBY_GC_HEAP_GROWTH_MAX_SLOTS=50000 RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0.9 bundle exec rake assets:precompile Purging temp files Bundling assets I, [2020-01-08T13:52:00.932032 #29478] INFO -- : Writing /var/www/discourse/public/assets/docker-manager-app-888922f3ec4fc42543c1916f533f3fecb9af503803eb6b8b59904a2310219d1d.js I, [2020-01-08T13:52:01.102633 #29478] INFO -- : Writing /var/www/discourse/public/assets/docker-manager-app-5256678af40d7479a0d37d3bddcca6fa5c2505b83a654a6d23e9d6f4f5fd406a.css I, [2020-01-08T13:52:01.277866 #29478] INFO -- : Writing /var/www/discourse/public/assets/docker-manager-vendor-91af87077979e715f4d5034ea05c612f734c3aece05878051e901d07e84878ee.js Skipping: vendor-a681e94dd2b8c886f184865617b09b4f67b433321acef2bd98bb99bd9495307e.js already compressed #<Thread:0x0000559b93cea1d8@/var/www/discourse/lib/tasks/assets.rake:223 run> terminated with exception (report_on_exception is true): /usr/local/lib/ruby/2.6.0/net/http.rb:949:in `rescue in block in connect': Failed to open TCP connection to geolite.maxmind.com:443 (getaddrinfo: Name or service not known) (SocketError) from /usr/local/lib/ruby/2.6.0/net/http.rb:946:in `block in connect' from /usr/local/lib/ruby/2.6.0/timeout.rb:93:in `block in timeout' from /usr/local/lib/ruby/2.6.0/timeout.rb:103:in `timeout' from /usr/local/lib/ruby/2.6.0/net/http.rb:945:in `connect' from /usr/local/lib/ruby/2.6.0/net/http.rb:930:in

And further down:

Done compressing docker-manager-vendor-91af87077979e715f4d5034ea05c612f734c3aece05878051e901d07e84878ee.js : 0.12 secs Done compressing all JS files : 4.44 secs rake aborted! SocketError: Failed to open TCP connection to geolite.maxmind.com:443 (getaddrinfo: Name or service not known) /var/www/discourse/lib/final_destination.rb:414:in `safe_session' /var/www/discourse/lib/final_destination.rb:365:in `safe_get' /var/www/discourse/lib/final_destination.rb:131:in `get' /var/www/discourse/lib/file_helper.rb:47:in `download' /var/www/discourse/lib/discourse_ip_info.rb:30:in `mmdb_download' /var/www/discourse/lib/tasks/assets.rake:226:in `block (3 levels) in <top (required)>' /var/www/discourse/lib/tasks/assets.rake:225:in `each' /var/www/discourse/lib/tasks/assets.rake:225:in `block (2 levels) in <top (required)>' Caused by: SocketError: getaddrinfo: Name or service not known /var/www/discourse/lib/final_destination.rb:414:in `safe_session' /var/www/discourse/lib/final_destination.rb:365:in `safe_get' /var/www/discourse/lib/final_destination.rb:131:in `get' /var/www/discourse/lib/file_helper.rb:47:in `download' /var/www/discourse/lib/discourse_ip_info.rb:30:in `mmdb_download' /var/www/discourse/lib/tasks/assets.rake:226:in `block (3 levels) in <top (required)>' /var/www/discourse/lib/tasks/assets.rake:225:in `each' /var/www/discourse/lib/tasks/assets.rake:225:in `block (2 levels) in <top (required)>' Tasks: TOP => assets:precompile (See full trace by running task with --trace) Docker Manager: FAILED TO UPGRADE #<RuntimeError: RuntimeError> /var/www/discourse/plugins/docker_manager/lib/docker_manager/upgrader.rb:159:in `run' /var/www/discourse/plugins/docker_manager/lib/docker_manager/upgrader.rb:81:in `upgrade' /var/www/discourse/plugins/docker_manager/app/controllers/docker_manager/admin_controller.rb:95:in `block in upgrade' /var/www/discourse/plugins/docker_manager/app/controllers/docker_manager/admin_controller.rb:91:in `fork' /var/www/discourse/plugins/docker_manager/app/controllers/docker_manager/admin_controller.rb:91:in `upgrade' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_controller/metal/basic_implicit_render.rb:6:inbox_tray:

I will try rebuilding, but I’m still wondering what is causing these upgrade problems…

Topic		Replies	Views
Down after Docker Update Installation server-resources	7	568	December 6, 2020
Upgrade failure mmdb_download (getaddrinfo: Name or service not known) Support	6	1501	March 19, 2020
Upgrade Error from 2.4.0.beta8 to 2.4.0.beta9 Support	2	437	February 14, 2020
Blank upgrade page after updating docker manager to 2.0.0.beta10 Bug	16	946	May 19, 2018
Cannot upgrade from 2.5.0beta2 to 2.6.0beta1 Support	3	502	August 2, 2020

Upgrade of docker manager beta7 >> beta9 fails

Related topics