Discourse failure after upgrade

Just did a routine upgrade, and the site is throwing a 504 error. Unicorn log is repeatedly cycling the following:

Parent was terminated!
Terminated
I, [2018-12-03T23:40:24.956767 #19874]  INFO -- : Refreshing Gem list
Prometheus Exporter, failed to send message Network is unreachable - connect(2) for "localhost" port 9405
Prometheus Exporter, failed to send message Network is unreachable - connect(2) for "localhost" port 9405
2018-12-03 23:40:33 +0000: Starting Prometheus Collector pid: 19941 port: 9405
2018-12-03 23:40:33 +0000: Prometheus Collector is monitoring 19874
E, [2018-12-03T23:40:33.860897 #19874] ERROR -- : adding listener failed addr=0.0.0.0:3000 (in use)
E, [2018-12-03T23:40:33.860974 #19874] ERROR -- : retrying in 0.5 seconds (4 tries left)
E, [2018-12-03T23:40:34.361379 #19874] ERROR -- : adding listener failed addr=0.0.0.0:3000 (in use)
E, [2018-12-03T23:40:34.361532 #19874] ERROR -- : retrying in 0.5 seconds (3 tries left)
E, [2018-12-03T23:40:34.862022 #19874] ERROR -- : adding listener failed addr=0.0.0.0:3000 (in use)
E, [2018-12-03T23:40:34.862179 #19874] ERROR -- : retrying in 0.5 seconds (2 tries left)
E, [2018-12-03T23:40:35.362648 #19874] ERROR -- : adding listener failed addr=0.0.0.0:3000 (in use)
E, [2018-12-03T23:40:35.362808 #19874] ERROR -- : retrying in 0.5 seconds (1 tries left)
E, [2018-12-03T23:40:35.863287 #19874] ERROR -- : adding listener failed addr=0.0.0.0:3000 (in use)
E, [2018-12-03T23:40:35.863440 #19874] ERROR -- : retrying in 0.5 seconds (0 tries left)
E, [2018-12-03T23:40:36.363854 #19874] ERROR -- : adding listener failed addr=0.0.0.0:3000 (in use)
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/socket_helper.rb:164:in `bind': Address already in use - bind(2) for 0.0.0.0:3000 (Errno::EADDRINUSE)
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/socket_helper.rb:164:in `new_tcp_server'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/socket_helper.rb:144:in `bind_listen'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:241:in `listen'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:851:in `block in bind_new_listeners!'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:851:in `each'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:851:in `bind_new_listeners!'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:140:in `start'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/unicorn-5.4.0/bin/unicorn:126:in `<top (required)>'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/bin/unicorn:23:in `load'
        from /var/www/discourse/vendor/bundle/ruby/2.5.0/bin/unicorn:23:in `<main>'
Parent was terminated!
Terminated
I, [2018-12-03T23:40:37.345773 #19959]  INFO -- : Refreshing Gem list

Currently trying a ./launcher rebuild app after disabling the prometheus plugin

You are going to need to do a rebuild ASAP. The Prometheus plugin is good. I have noticed admin/upgrade is a bit more flaky recently @tgxworld.

4 Likes

The rebuild fixed it. Thanks @sam

3 Likes