I woke up this morning to reports that my discourse server was “down”.
I was able to ssh into the server without a problem, noticed some security updates needed to be applied, applied them with apt full-upgrade, and restarted the server with reboot.
Took a little while to come back up but when it did I checked that the discourse docker container was running. Oddly, no problem.
Tried stopping and restarting with ./launcher app stop and ./launcher app start. Didn’t help.
Tried doing a ./launcher app rebuild and got the following error.
fatal: unable to access 'https://github.com/discourse/pups.git/': Could not resolve host: github.com
d1412324832190f43a2d51b5f10c53d6fa671056f91d0be2178d17a5ba1ab692
I know this isn’t exactly a lot to go on but I’m stumped. Any suggestions would be greatly appreciated.
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 8.8.8.8
nameserver 8.8.4.4
@supermathie Not sure if this has anything to do with it but in ./launcher logs app I see
[Tue 01 Dec 2020 07:07:13 PM UTC] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 6
[Tue 01 Dec 2020 07:07:13 PM UTC] Can not init api.
[Tue 01 Dec 2020 07:07:13 PM UTC] Domains not changed.
[Tue 01 Dec 2020 07:07:13 PM UTC] Skip, Next renewal time is: Sat Dec 26 00:31:17 UTC 2020
[Tue 01 Dec 2020 07:07:13 PM UTC] Add '--force' to force to renew.
[Tue 01 Dec 2020 07:07:13 PM UTC] Installing key to:/shared/ssl/community.acescentral.com.key
[Tue 01 Dec 2020 07:07:13 PM UTC] Installing full chain to:/shared/ssl/community.acescentral.com.cer
[Tue 01 Dec 2020 07:07:13 PM UTC] Run reload cmd: sv reload nginx
fail: nginx: runsv not running
[Tue 01 Dec 2020 07:07:13 PM UTC] Reload error for :
[Tue 01 Dec 2020 07:07:33 PM UTC] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 6
[Tue 01 Dec 2020 07:07:34 PM UTC] Can not init api.
[Tue 01 Dec 2020 07:07:34 PM UTC] Domains not changed.
[Tue 01 Dec 2020 07:07:34 PM UTC] Skip, Next renewal time is: Sat Dec 26 00:31:20 UTC 2020
[Tue 01 Dec 2020 07:07:34 PM UTC] Add '--force' to force to renew.
[Tue 01 Dec 2020 07:07:34 PM UTC] Installing key to:/shared/ssl/community.acescentral.com_ecc.key
[Tue 01 Dec 2020 07:07:34 PM UTC] Installing full chain to:/shared/ssl/community.acescentral.com_ecc.cer
[Tue 01 Dec 2020 07:07:34 PM UTC] Run reload cmd: sv reload nginx
fail: nginx: runsv not running
[Tue 01 Dec 2020 07:07:34 PM UTC] Reload error for :
Started runsvdir, PID is 458
chgrp: invalid group: ‘syslog’
ok: run: redis: (pid 473) 0s
ok: run: postgres: (pid 470) 0s
rsyslogd: imklog: cannot open kernel log (/proc/kmsg): Operation not permitted.
rsyslogd: activation of module imklog failed [v8.1901.0 try https://www.rsyslog.com/e/2145 ]
supervisor pid: 471 unicorn pid: 497
I might try purging and reinstalling docker; it’s hard to guess exactly what’s going on here and why docker broke.
Before you do anything though, ensure you have a backup of your data (either a Discourse backup, or at least postgres & uploads). Image the VM if you can.