Problem after update to 1.8.0.beta10 (today)


(Andrew Waugh) #1

This on my sandbox (plain install per 30 minute on an AWS Ubuntu 16.04 isntance), previous update was 4 days ago, after which it ran fine.

All I did was:

cd /var/discourse
git pull
./launcher rebuild app

No errors during the update, but since it finished the site gives 502 bad gateway nginx errors, and docker logs 4b7 shows:

ok: run: redis: (pid 47) 390s
ok: run: postgres: (pid 44) 390s
supervisor pid: 2066 unicorn pid: 2071
config/unicorn_launcher: line 44: kill: (2071) - No such process
config/unicorn_launcher: line 10: kill: (2071) - No such process
exiting
ok: run: redis: (pid 47) 396s
ok: run: postgres: (pid 44) 396s
supervisor pid: 2104 unicorn pid: 2109
config/unicorn_launcher: line 44: kill: (2109) - No such process
config/unicorn_launcher: line 10: kill: (2109) - No such process

I’ve done a shutdown -r, and apt-get update, apt-get upgrade didn’t show any out of date packages.


Maybe a bug on meta, post disappeared
(Sam Saffron) #4

what is memory doing on the box?


(Andrew Waugh) #5
free -h
              total        used        free      shared  buff/cache   available
Mem:           2.0G        195M        1.3G         28M        475M        1.7G
Swap:          2.0G         56K        2.0G

(Kai Liu) #7

Only 195M memory used, that means your Discourse application, probably even the docker container, is not up at all.

How about docker ps -a output and ps axf output?


(Mitchell Krog) #8

I just upgraded now and no problems.

I only ever do ./launcher rebuild app which does any git pulls during it’s rebuild.

Maybe reboot your server seems like your docker container did not restart for some reason.


(Andrew Waugh) #9

The docker container is up.

My best guess is that the unicorn process is the issue:

Here is the bottom of docker logs, followed immediately by a ps -ef|grep unicorn

ok: run: redis: (pid 47) 688s
ok: run: postgres: (pid 44) 688s
supervisor pid: 3743 unicorn pid: 3748
config/unicorn_launcher: line 44: kill: (3748) - No such process
config/unicorn_launcher: line 10: kill: (3748) - No such process
exiting
ok: run: redis: (pid 47) 695s
ok: run: postgres: (pid 44) 695s
supervisor pid: 3782 unicorn pid: 3787
config/unicorn_launcher: line 44: kill: (3787) - No such process
config/unicorn_launcher: line 10: kill: (3787) - No such process
exiting
ok: run: redis: (pid 47) 702s
ok: run: postgres: (pid 44) 702s
supervisor pid: 3822 unicorn pid: 3827
root@ip-172-31-29-7:/var/discourse/shared/standalone# ps -ef|grep unicorn
root       402   400  0 13:41 ?        00:00:00 runsv unicorn
ubuntu    4837   402  6 13:53 ?        00:00:00 /bin/bash config/unicorn_launcher -E production -c config/unicorn.conf.rb
ubuntu    4842  4837 67 13:53 ?        00:00:01 ruby /var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn -E production -c config/unicorn.conf.rb

I’ve done a(nother) ./launcher rebuild app, as well as a reboot in the meantime.

Could it be that the unicorn is falling over on it’s own before the start tries to kill it off and restart it, and the pids are getting out of sync?


502 Bad Gateway nginx error post rebuilding
(Sam Saffron) #10

Maybe it is failing to boot, what plugins do you have, try removing all third party ones and isolating which is at fault.


(Kai Liu) #11

I have a feeling that your rebuild didn’t finish successfully. Can you upload the full rebuild log?


(Andrew Waugh) #12

Just doing that now. brb


(Andrew Waugh) #13

It was indeed a beta plugin.

My apologies, I should have checked that before posting here.


(Sam Saffron) #14