Discourse has stopped opening

[itamerlanium (Tarlan Isaev)]

Hi dears,
unfortunately, my website http://specknz.me/ has stopped opening and I don’t really know what has caused it. Also I’ve got an error:

502 Bad Gateway
nginx

How could be it sort out without touching the Snapshot on the Digitalocean?
Thank you.

[Falco]

SSH inside and do the tree magic phrases:

cd /var/discourse
git pull
./launcher rebuild app

[itamerlanium (Tarlan Isaev)]

Thanks Rafael, I’ve already rebuilded the app, but it still doesn’t work.

…

Removing old container
+ /usr/bin/docker rm app
app

+ /usr/bin/docker run -d --restart=always -e LANG=en_US.UTF-8 -e RAILS_ENV=production -e UNICORN_WORKERS=3 -e UNICORN_SIDEKIQS=1 -e RUBY_GLOBAL_METHOD_CACHE_SIZE=131072 -e DISCOURSE_DB_SOCKET=/var/run/postgresql -e DISCOURSE_DB_HOST= -e DISCOURSE_DB_PORT= -e DISCOURSE_HOSTNAME=speckme.nz -e DISCOURSE_DEVELOPER_EMAILS=tamerlanium@gmail.com -e DISCOURSE_SMTP_ADDRESS=smtp.gmail.com -h discourse-app -e DOCKER_HOST_IP=172.17.0.1 --name app -t -p 25654:80 -p 443:443 -v /var/discourse/shared/standalone:/shared -v /var/discourse/shared/standalone/log/var-log:/var/log --mac-address 02:f9:8c:dd:3d:63 local_discourse/app /sbin/boot
357203ee7df50c1e85272786c05d1826f773412b6706ad3f9d6de262eb148119
root@discourse:/var/discourse# reboot

Broadcast message from itamerlanium@discourse
	(/dev/pts/0) at 18:51 ...

The system is going down for reboot NOW!
root@discourse:/var/discourse# Connection to 159.203.205.195 closed by remote host.
Connection to 159.203.205.195 closed.

[Falco]

I didn’t understood the log, why the reboot after the rebuild?

[itamerlanium (Tarlan Isaev)]

I tried to open the website before and after the reboot, anyway it doesn’t work.

[Falco]

Please tail the logs. My bet is a wrong permission so start by tailing nginx log.

[itamerlanium (Tarlan Isaev)]

Unfortunately the access.log is empty. I found some logs through the error files.

access.log  error.log.1     error.log.3.gz
error.log   error.log.2.gz  error.log.4.gz
root@discourse:/var/log/nginx#

 GNU nano 2.2.6             File: error.log.1

2016/07/19 00:22:26 [emerg] 1536#0: BIO_new_file("/etc/letsencrypt/live/specknz$
2016/07/19 17:31:32 [emerg] 1525#0: BIO_new_file("/etc/letsencrypt/live/specknz$
2016/07/19 17:39:44 [emerg] 4211#0: BIO_new_file("/etc/letsencrypt/live/specknz$
2016/07/24 07:27:11 [emerg] 1499#0: BIO_new_file("/etc/letsencrypt/live/specknz$
2016/07/24 18:13:19 [emerg] 1537#0: BIO_new_file("/etc/letsencrypt/live/specknz$
2016/07/24 18:52:03 [emerg] 1542#0: BIO_new_file("/etc/letsencrypt/live/specknz$
2016/07/24 22:57:59 [emerg] 1540#0: BIO_new_file("/etc/letsencrypt/live/specknz$

  GNU nano 2.2.6            File: error.log.4.gz

^_�^H^@JsoW^@^C��Mo�@^P�{��X�^G�R^Uf?
skչK��=�^XE�^C�^F^U���V��/·Y�*^PB/e�oև̇>�+f<� ��^@L �^AGmP�� n��~Ÿ^@^]�^Cd7i�̸^W�$
�}H^Rg˒�2gWɞ�9ەv�V�,�%�Օ?JM�^D�^?��U�e,.vY^Õ��zXʏ^U^R9�ҏǏ؏ȏ,ŏX�{$^_(��  {rؽ�Y^@
ӹt^B��jDɝهk&p^B�(x�^Ç^V(e^S�"^U�)�+�+D^S8�O^S�_a+�^�VrL͇^]ɇ_!��  {Rؽ�YE��ތ�Q4掣+$
,�oWi~����ŹQ�ŹŹ^Wdֹ¹!^H^@^N �ܗ�ݰ,�q+�ǰu��9~-X��oٰa=V8��ΰưU���|�2�ٰ���-^^V�^W6� ^D�$
1��?5HC�x� [2�`�`^@^H��^N�
�!�N^]R�ӹ'^A^Q9Թr����5g^E^@J�C-�Ĺ^Ry��^RRޟ��C�CB{^N)�ɡơ!α�*�SkImȱ3�������αQ�6�
]Ϲ_�^]ı�C�^RSԃڃ^Fу8��A^B�i��ʐ^Lz4H^^6d Qq�^W��A��a?�R{^[2C��ѐ Ðj^~�Ɓ^R�7�^OG�p�$

[tgxworld (Alan Tan)]

What does

./launcher logs app

give you?

Our nginx logs are actually in /shared/log/var-log/nginx

[Falco]

Those emerg entries on the nginx log mean he’s not gonna run. This is a problem with your ssl setup, it is missing deparams.

I recommend setting up ssl again.

[itamerlanium (Tarlan Isaev)]

@tgxworld

/shared/log/var-log/nginx

Unfortunately, it gives nothing.

root@discourse:~# ./launcher logs app
-su: ./launcher: No such file or directory

root@discourse:/# cd /shared/log/var-log/nginx
-su: cd: /shared/log/var-log/nginx: No such file or directory

[riking (Kane York)]

Yeah, this is the problem. (“Emerg” = Emergency)

[itamerlanium (Tarlan Isaev)]

Those emerg entries on the nginx log mean he’s not gonna run. This is a problem with your ssl setup, it is missing deparams.

I recommend setting up ssl again.

I went through this tutorial https://www.digitalocean.com/community/tutorials/how-to-install-discourse-behind-nginx-on-ubuntu-14-04#prerequisites to recreate SSL Certificate on Nginx for Ubuntu 14.04 and got a message.

root@discourse:/opt/letsencrypt# ./letsencrypt-auto certonly --standalone --email tamerlanium@gmail.com --agree-tos -d specknz.me

The program exe (process ID 1304) is already listening on TCP port   │
   │ 443. This will prevent us from binding to that port. Please stop the │
   │ exe program temporarily and then try again.

At least one of the (possibly) required ports is already taken.

root@discourse:/opt/letsencrypt# cd /etc/letsencrypt/live
-su: cd: /etc/letsencrypt/live: No such file or directory

root@discourse:/opt/letsencrypt# ls
acme                        letsencrypt-nginx
certbot                     letshelp-certbot
certbot-apache              letshelp-letsencrypt
certbot-auto                LICENSE.txt
certbot-compatibility-test  linter_plugin.py
certbot-nginx               MANIFEST.in
CHANGES.rst                 pep8.travis.sh
CONTRIBUTING.md             README.rst
docker-compose.yml          readthedocs.org.requirements.txt
Dockerfile                  setup.cfg
Dockerfile-dev              setup.py
docs                        tests
examples                    tools
letsencrypt                 tox.cover.sh
letsencrypt-apache          tox.ini
letsencrypt-auto            Vagrantfile
letsencrypt-auto-source

And didn’t get a response similar to this:

Let’s Encrypt Output
IMPORTANT NOTES:

• If you lose your account credentials, you can recover through
  e-mails sent to sammy@example.com.
• Congratulations! Your certificate and chain have been saved at
  /etc/letsencrypt/live/discourse.example.com/fullchain.pem. Your
  cert will expire on 2016-04-26. To obtain a new version of the
  certificate in the future, simply run Let’s Encrypt again.
• Your account credentials have been saved in your Let’s Encrypt
  configuration directory at /etc/letsencrypt. You should make a
  secure backup of this folder now. This configuration directory will
  also contain certificates and private keys obtained by Let’s
  Encrypt so making regular backups of this folder is ideal.

[tgxworld (Alan Tan)]

Hmm how was your Discourse instance setup? Are you on the docker based install?

[itamerlanium (Tarlan Isaev)]

Yeah, it have been installed through the Docker instance.

[tgxworld (Alan Tan)]

Try following

Set up HTTPS support with Let's Encrypt

This is a guide for enabling HTTPS on an existing Discourse installation using Let’s Encrypt. It assumes prior installation without HTTPS enabled. Required user level: System Administrator This guide is only for existing installs where HTTPS is not enabled. Following the official setup guide automatically enables HTTPS by default. So you’d like to add https to your Discourse absolutely free, courtesy of our friends at Let’s Encrypt? I…

to install a Let’s Encrypt cert instead

[itamerlanium (Tarlan Isaev)]

Sorry Guo, should I rebuild the Discourse container?

4. Rebuild your container

./launcher rebuild <container name>

[sam (Sam Saffron)]

@itamerlanium

Sorry Guo, should I rebuild the Discourse container?

Yes

Can you please try rebuilding?

Please SSH into your server and run:

cd /var/discourse
git pull
./launcher rebuild app

[sam (Sam Saffron)]

See:

Discourse has stopped opening

Yes Can you please try rebuilding? Please SSH into your server and run: cd /var/discourse git pull ./launcher rebuild app

run ./launcher rebuild app

[itamerlanium (Tarlan Isaev)]

Ups I missed it
I’ve resized the Droplet to 1Gb by the using SWAP and tried to rebuild it, but it didn’t help me. However, yesterday I rebuild it successfully.

if ($http_host != $$ENV_DISCOURSE_HOSTNAME) {
   rewrite (.*) https://$$ENV_DISCOURSE_HOSTNAME$1 permanent;
} in /etc/nginx/conf.d/discourse.conf
I, [2016-07-27T06:45:30.779342 #13]  INFO -- : > if [ -z "$LETSENCRYPT_ACCOUNT_EMAIL" ]; then echo "LETSENCRYPT_ACCOUNT_EMAIL ENV variable is required and has not been set."; exit 1; fi
I, [2016-07-27T06:45:30.782199 #13]  INFO -- : LETSENCRYPT_ACCOUNT_EMAIL ENV variable is required and has not been set.

I, [2016-07-27T06:45:30.789702 #13]  INFO -- : Terminating async processes
I, [2016-07-27T06:45:30.790433 #13]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main pid: 42
2016-07-27 06:45:30 UTC [42-2] LOG:  received fast shutdown request
2016-07-27 06:45:30 UTC [42-3] LOG:  aborting any active transactions
2016-07-27 06:45:30 UTC [49-2] LOG:  autovacuum launcher shutting down
I, [2016-07-27T06:45:30.801912 #13]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 155
155:signal-handler (1469601930) Received SIGTERM scheduling shutdown...
2016-07-27 06:45:30 UTC [46-1] LOG:  shutting down
2016-07-27 06:45:30 UTC [46-2] LOG:  database system is shut down
155:M 27 Jul 06:45:30.851 # User requested shutdown...
155:M 27 Jul 06:45:30.852 * Saving the final RDB snapshot before exiting.
155:M 27 Jul 06:45:30.990 * DB saved on disk
155:M 27 Jul 06:45:30.990 # Redis is now ready to exit, bye bye...


FAILED
--------------------
Pups::ExecError: if [ -z "$LETSENCRYPT_ACCOUNT_EMAIL" ]; then echo "LETSENCRYPT_ACCOUNT_EMAIL ENV variable is required and has not been set."; exit 1; fi failed with return #<Process::Status: pid 3087 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params {"cmd"=>["if [ -z \"$LETSENCRYPT_ACCOUNT_EMAIL\" ]; then echo \"LETSENCRYPT_ACCOUNT_EMAIL ENV variable is required and has not been set.\"; exit 1; fi", "/bin/bash -c \"if [[ ! \\\"$LETSENCRYPT_ACCOUNT_EMAIL\\\" =~ ([^@]+)@([^\\.]+) ]]; then echo \\\"LETSENCRYPT_ACCOUNT_EMAIL is not a valid email address\\\"; exit 1; fi\""]}
b723307a27a32d6c808e520b0cc3000fa2cbfe732291dd1bb312e7c9b8bfb375
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
root@discourse:/var/discourse#

[sam (Sam Saffron)]

LETSENCRYPT_ACCOUNT_EMAIL ENV variable is required and has not been set.

hmmm… is that set?