Failed to start discouse


(Kyle Boyce) #1
FAILED
--------------------
Pups::ExecError: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 failed with return #<Process::Status: pid 21 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1"
a6a7d88a5f54074888809da6317aba14c57f39d56f7fa93ef88c4fb1c69fe06a
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
root@work:/var/discourse#

(David Taylor) #2

^^


(Kyle Boyce) #3

I, [2017-08-20T19:21:08.747081 #15]  INFO -- : File > /usr/local/bin/discourse  chmod: +x
I, [2017-08-20T19:21:08.752033 #15]  INFO -- : File > /usr/local/bin/rails  chmod: +x
I, [2017-08-20T19:21:08.756681 #15]  INFO -- : File > /usr/local/bin/rake  chmod: +x
I, [2017-08-20T19:21:08.761082 #15]  INFO -- : File > /etc/update-motd.d/10-web  chmod: +x
I, [2017-08-20T19:21:08.763989 #15]  INFO -- : File > /etc/logrotate.d/rails  chmod: 
I, [2017-08-20T19:21:08.768180 #15]  INFO -- : File > /etc/logrotate.d/nginx  chmod: 
I, [2017-08-20T19:21:08.774814 #15]  INFO -- : File > /etc/runit/1.d/00-ensure-links  chmod: +x
I, [2017-08-20T19:21:08.785343 #15]  INFO -- : File > /root/.bash_profile  chmod: 644
I, [2017-08-20T19:21:08.786177 #15]  INFO -- : Replacing (?-mix:server.+{) with limit_req_zone $binary_remote_addr zone=flood:10m rate=$reqs_per_secondr/s;
limit_req_zone $binary_remote_addr zone=bot:10m rate=$reqs_per_minuter/m;
limit_req_status 429;
limit_conn_zone $binary_remote_addr zone=connperip:10m;
limit_conn_status 429;
server {
 in /etc/nginx/conf.d/discourse.conf
I, [2017-08-20T19:21:08.787149 #15]  INFO -- : Replacing (?-mix:location @discourse {) with location @discourse {
  limit_conn connperip $conn_per_ip;
  limit_req zone=flood burst=$burst_per_second nodelay;
  limit_req zone=bot burst=$burst_per_minute nodelay; in /etc/nginx/conf.d/discourse.conf
I, [2017-08-20T19:21:08.788039 #15]  INFO -- : > echo "Beginning of custom commands"
I, [2017-08-20T19:21:08.792163 #15]  INFO -- : Beginning of custom commands

I, [2017-08-20T19:21:08.792634 #15]  INFO -- : > echo "End of custom commands"
I, [2017-08-20T19:21:08.796821 #15]  INFO -- : End of custom commands

I, [2017-08-20T19:21:08.797169 #15]  INFO -- : Terminating async processes
I, [2017-08-20T19:21:08.797261 #15]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 157
157:signal-handler (1503256868) Received SIGTERM scheduling shutdown...
157:M 20 Aug 19:21:08.823 # User requested shutdown...
157:M 20 Aug 19:21:08.823 * Saving the final RDB snapshot before exiting.
157:M 20 Aug 19:21:08.905 * DB saved on disk
157:M 20 Aug 19:21:08.905 # Redis is now ready to exit, bye bye...
sha256:a98640b41f43efed010fb82454bb811b3a9f274881faea22c37f09a325e01d84
071f485e29165f21a3c312da6ca39758794c95b0e3f87261bde953ef7a52e96d
Removing old container
+ /usr/bin/docker rm app
Error response from daemon: You cannot remove a running container 80088f5216c1347d955cccf0edc21f3badfaa92422b495627484acba5b32e302. Stop the container before attempting removal or force remove
80088f5216c1
Nothing to do, your container has already started!

I got it to run but it stops here

Edit:

I tried again here is the error:

+ /usr/bin/docker stop -t 10 app
app
cd /pups && git pull && /pups/bin/pups --stdin
Already up-to-date.
I, [2017-08-20T19:34:56.567769 #15]  INFO -- : Loading --stdin
I, [2017-08-20T19:34:56.577409 #15]  INFO -- : > mkdir -p /shared/postgres_run
I, [2017-08-20T19:34:56.581418 #15]  INFO -- : 
I, [2017-08-20T19:34:56.581929 #15]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2017-08-20T19:34:56.584728 #15]  INFO -- : 
I, [2017-08-20T19:34:56.584926 #15]  INFO -- : > chmod 775 /shared/postgres_run
I, [2017-08-20T19:34:56.587104 #15]  INFO -- : 
I, [2017-08-20T19:34:56.587256 #15]  INFO -- : > rm -fr /var/run/postgresql
I, [2017-08-20T19:34:56.655505 #15]  INFO -- : 
I, [2017-08-20T19:34:56.655785 #15]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2017-08-20T19:34:56.689368 #15]  INFO -- : 
I, [2017-08-20T19:34:56.689723 #15]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
I, [2017-08-20T19:34:56.697265 #15]  INFO -- : postgres already running stop container



FAILED
--------------------
Pups::ExecError: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 failed with return #<Process::Status: pid 22 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1"
738398b21bedbc2e92f3adfc2cbb312b25cf024a612cca626258fc8d59b7ce50
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
root@work:/var/discourse# 

(David Taylor) #4

What command are you running? ./launcher rebuild app?

What plugins do you have installed?


(Kyle Boyce) #5

Yes I am doing ./launcher rebuild app

Plugins:

          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-oauth2-basic.git
          - git clone https://github.com/ellotheth/distodis.git
          - git clone https://github.com/discourse/discourse-solved.git
          - git clone https://github.com/discourse/discourse-spoiler-alert.git
          - git clone https://github.com/discourse/discourse-tagging.git
          - git clone https://github.com/mozilla/discourse-email-all.git
          - git clone https://github.com/davidtaylorhq/discourse-whos-online.git
          - git clone https://github.com/discourse/discourse-chat-integration.git
          - git clone https://github.com/discoursehosting/discourse-sitemap.git
          - git clone https://github.com/cpradio/discourse-plugin-replygif.git```

(David Taylor) #6

This isn’t even a discourse plugin, so you should probably remove that, although I doubt it’s the cause of this issue.

I haven’t seen this before - not sure what could cause it… You could try rebooting your server. Do you have enough free disk space? (run df -h to check)


(Kyle Boyce) #7

Tried all of that and I get this


I, [2017-08-21T15:19:19.481661 #15]  INFO -- : Terminating async processes
I, [2017-08-21T15:19:19.481740 #15]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main pid: 44
2017-08-21 15:19:19.481 UTC [44] LOG:  received fast shutdown request
2017-08-21 15:19:19.481 UTC [44] LOG:  aborting any active transactions
2017-08-21 15:19:19.482 UTC [51] LOG:  autovacuum launcher shutting down
I, [2017-08-21T15:19:19.484152 #15]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 157
157:signal-handler (1503328759) Received SIGTERM scheduling shutdown...
2017-08-21 15:19:19.485 UTC [48] LOG:  shutting down
157:M 21 Aug 15:19:19.536 # User requested shutdown...
157:M 21 Aug 15:19:19.536 * Saving the final RDB snapshot before exiting.
157:M 21 Aug 15:19:19.624 * DB saved on disk
157:M 21 Aug 15:19:19.624 # Redis is now ready to exit, bye bye...
2017-08-21 15:19:19.949 UTC [48] LOG:  database system is shut down
sha256:aeb13af1d49b40b1c18347a6b6a91f233f56f2a354e2f77aa1770f62f4e64036
f5e1af6cc8c65965d19d7f4fc02ae43bd7ec570f7e98b525713beb1d648c320e
Removing old container
+ /usr/bin/docker rm app
app

+ /usr/bin/docker run -d --restart=always -e LANG=en_US.UTF-8 -e RAILS_ENV=production -e UNICORN_WORKERS=8 -e UNICORN_SIDEKIQS=1 -e RUBY_GLOBAL_METHOD_CACHE_SIZE=131072 -e RUBY_GC_HEAP_GROWTH_MAX_SLOTS=40000 -e RUBY_GC_HEAP_INIT_SLOTS=400000 -e RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=1.5 -e DISCOURSE_DB_SOCKET=/var/run/postgresql -e DISCOURSE_DB_HOST= -e DISCOURSE_DB_PORT= -e DISCOURSE_HOSTNAME=blog.gayming.online -e DISCOURSE_DEVELOPER_EMAILS=xxponagexx@outlook.com -e DISCOURSE_SMTP_ADDRESS=smtp.gmail.com -e DISCOURSE_SMTP_PORT=587 -e DISCOURSE_SMTP_USER_NAME=gaymingxyz@gmail.com -e DISCOURSE_SMTP_PASSWORD= -h work-app -e DOCKER_HOST_IP=172.17.0.1 --name app -t -p 80:80 -p 443:443 -v /var/discourse/shared/standalone:/shared -v /var/discourse/shared/standalone/log/var-log:/var/log --mac-address 02:3b:21:f8:cc:78 local_discourse/app /sbin/boot
739077a68345303610eb905a648e87350960c456328c36338a806607e359f378
/usr/bin/docker: Error response from daemon: driver failed programming external connectivity on endpoint app (4f5666c5b9e0409c4244189758c2a663d089284ed3ca033e9a3879cade32185c): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use.
root@work:/var/discourse# 

I have lots of storage left


(Eli the Bearded) #8

That last line:

Something else is running a web server and it has claimed the port (80 is http) already.


(Kyle Boyce) #9

Hmm, nothing is running on my domain though…

blog.gayming.online


(Kyle Boyce) #10

How do I stop what is running?

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      680/mysqld          
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      698/nginx: master p ```

(Joshua Rosenfeld) #11

Assuming you’re running standard Ubuntu Server 16.04, run

systemctl stop nginx.service

(Kyle Boyce) #12

I am running Debian, but I will try that


(Kyle Boyce) #13

Didn’t work on Debian. Here is the crash I got after typing that:

I, [2017-08-21T20:49:45.251834 #14]  INFO -- : Beginning of custom commands

I, [2017-08-21T20:49:45.252200 #14]  INFO -- : > echo "End of custom commands"
I, [2017-08-21T20:49:45.254994 #14]  INFO -- : End of custom commands

I, [2017-08-21T20:49:45.255280 #14]  INFO -- : Terminating async processes
I, [2017-08-21T20:49:45.255380 #14]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main pid: 43
I, [2017-08-21T20:49:45.255568 #14]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 156
2017-08-21 20:49:45.255 UTC [43] LOG:  received fast shutdown request
2017-08-21 20:49:45.255 UTC [43] LOG:  aborting any active transactions
2017-08-21 20:49:45.255 UTC [50] LOG:  autovacuum launcher shutting down
156:signal-handler (1503348585) Received SIGTERM scheduling shutdown...
2017-08-21 20:49:45.261 UTC [47] LOG:  shutting down
156:M 21 Aug 20:49:45.270 # User requested shutdown...
156:M 21 Aug 20:49:45.270 * Saving the final RDB snapshot before exiting.
156:M 21 Aug 20:49:45.352 * DB saved on disk
156:M 21 Aug 20:49:45.352 # Redis is now ready to exit, bye bye...
2017-08-21 20:49:45.594 UTC [47] LOG:  database system is shut down
sha256:154afcaf74e7f9cf2b5119f25c111f10144cb03118fab7cc74c71d9ee6bf4b12
27470f000426e507521048abd648d55271d9395fb9926b1094f9b35c295aab1d
Removing old container
+ /usr/bin/docker rm app
app

+ /usr/bin/docker run -d --restart=always -e LANG=en_US.UTF-8 -e RAILS_ENV=production -e UNICORN_WORKERS=8 -e UNICORN_SIDEKIQS=1 -e RUBY_GLOBAL_METHOD_CACHE_SIZE=131072 -e RUBY_GC_HEAP_GROWTH_MAX_SLOTS=40000 -e RUBY_GC_HEAP_INIT_SLOTS=400000 -e RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=1.5 -e DISCOURSE_DB_SOCKET=/var/run/postgresql -e DISCOURSE_DB_HOST= -e DISCOURSE_DB_PORT= -e DISCOURSE_HOSTNAME=blog.gayming.online -e DISCOURSE_DEVELOPER_EMAILS=xxponagexx@outlook.com -e DISCOURSE_SMTP_ADDRESS=smtp.gmail.com -e DISCOURSE_SMTP_PORT=587 -e DISCOURSE_SMTP_USER_NAME=gaymingxyz@gmail.com -e DISCOURSE_SMTP_PASSWORD=-h work-app -e DOCKER_HOST_IP=172.17.0.1 --name app -t -p 80:80 -p 443:443 -v /var/discourse/shared/standalone:/shared -v /var/discourse/shared/standalone/log/var-log:/var/log --mac-address 02:3b:21:f8:cc:78 local_discourse/app /sbin/boot
1aaaaa07e3e32569837aa4d10cc5ed7496f8aa9052872707fe622f75eb9d9630

(David Taylor) #14

That doesn’t look like a crash, the container started successfully. Is your forum still down?


(Kyle Boyce) #15

Yes you can see it here http://blog.gayming.online/


(David Taylor) #16

Loads fine for me


(Kyle Boyce) #17

Hmmm? Not for me…

The domain has been registered to the IP for weeks and was working on day but not the next


(David Taylor) #18

Hmm… looks like something weird going on with your DNS settings then, rather than anything to do with the rebuild


(Kyle Boyce) #19

Huh… I can’t access it via mobile data either… Anyhelp?


(David Taylor) #20

Yeah, it is no longer working for me either.

What DNS provider are you using? Can you check that the IP address is correct?