Can't reach web server after update to 3.4

[tomtjes (Thomas Reintjes)]

I just updated using ./launcher rebuild app and now the website can’t be reached. There were no errors during the Postgres upgrade or the rebuild. Here’s the output from discourse doctor.

DISCOURSE DOCTOR Tue Feb 4 10:56:05 PM UTC 2025
OS: Linux redacted 6.8.0-52-generic #53-Ubuntu SMP PREEMPT_DYNAMIC Sat Jan 11 00:06:25 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux


Found containers/app.yml

==================== YML SETTINGS ====================
DISCOURSE_HOSTNAME=redacted.tld
SMTP_ADDRESS=cp120.redacted.net
DEVELOPER_EMAILS=host@redacted.tld
SMTP_PASSWORD=redacted
SMTP_PORT=587
SMTP_USER_NAME=record@redacted.tld
LETSENCRYPT_ACCOUNT_EMAIL=host@redacted.tld

==================== DOCKER INFO ====================
DOCKER VERSION: Docker version 26.1.3, build 26.1.3-0ubuntu1~24.04.1

DOCKER PROCESSES (docker ps -a)

CONTAINER ID   IMAGE                 COMMAND        CREATED          STATUS          PORTS                                                                      NAMES
26b5f30922a1   local_discourse/app   "/sbin/boot"   14 minutes ago   Up 12 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   app

26b5f30922a1   local_discourse/app   "/sbin/boot"   14 minutes ago   Up 12 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   app

Discourse container app is running


==================== PLUGINS ====================
          - sudo -E -u discourse git clone https://github.com/discourse/docker_manager.git
          - sudo -E -u discourse git clone https://github.com/discourse/discourse-data-explorer.git
          - sudo -E -u discourse git clone https://github.com/discourse/discourse-affiliate.git
##          - sudo -E -u discourse git clone https://github.com/paviliondev/discourse-locations.git
##          - sudo -E -u discourse git clone https://github.com/paviliondev/discourse-events.git
##          - sudo -E -u discourse git clone https://github.com/paviliondev/discourse-custom-wizard.git
##          - sudo -E -u discourse git clone https://github.com/gdpelican/mingle.git
##          - sudo -E -u discourse git clone https://github.com/synweap15/discourse-voice-record.git
          - sudo -E -u discourse git clone https://github.com/discourse/discourse-docs.git
          - sudo -E -u discourse git clone https://github.com/discourse/discourse-calendar.git
          - sudo -E -u discourse git clone https://github.com/discourse/discourse-adplugin.git

WARNING:
You have what appear to be non-official plugins.
If you are having trouble, you should disable them and try rebuilding again.

See https://github.com/discourse/discourse/blob/main/lib/plugin/metadata.rb for the official list.

========================================
Discourse version at redacted.tld: NOT FOUND
Discourse version at localhost: NOT FOUND


==================== MEMORY INFORMATION ====================
RAM (MB): 4005

               total        used        free      shared  buff/cache   available
Mem:            3820        1257        1137          52        1788        2562
Swap:           2047          75        1972

==================== DISK SPACE CHECK ====================
---------- OS Disk Space ----------
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        38G  6.0G   30G  17% /
/dev/sdb         25G   11G   13G  46% /var/discourse
/dev/sdb         25G   11G   13G  46% /var/lib/docker

==================== DISK INFORMATION ====================
Disk /dev/sda: 38.15 GiB, 40961572864 bytes, 80003072 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 4F4C86C1-C38F-4B0C-BA8A-01C10D997AE4

Device      Start      End  Sectors  Size Type
/dev/sda1  528384 80003038 79474655 37.9G Linux filesystem
/dev/sda14   2048     4095     2048    1M BIOS boot
/dev/sda15   4096   528383   524288  256M EFI System

Partition table entries are not in disk order.


Disk /dev/sdb: 25 GiB, 26843545600 bytes, 52428800 sectors
Disk model: Volume
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

==================== END DISK INFORMATION ====================

==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Or just send a test message to yourself.
Email address for mail test? ('n' to skip) [host@redacted.tld]:
Sending mail to host@redacted.tld. . .
Testing sending to host@redacted.tld using cp120.redacted.net:587, username:record@redacted.tld with plain auth.
SMTP server connection successful.
Sending to host@redacted.tld. . .
Mail accepted by SMTP server.
Message-ID: a9640924-a68b-4e5e-a769-3110de133c08@redacted.tld

If you do not receive the message, check your SPAM folder
or test again using a service like http://www.mail-tester.com/.

If the message is not delivered it is not a problem with Discourse.
Check the SMTP server logs for the above Message ID to see why it
failed to deliver the message.
Replacing: SMTP_PASSWORD
Replacing: LETSENCRYPT_ACCOUNT_EMAIL
Replacing: DEVELOPER_EMAILS
Replacing: DISCOURSE_DB_PASSWORD
Replacing: Sending mail to

==================== DONE! ====================

Any idea what could be going on here?

[david (David Taylor)]

Have you tried rebuilding in the last 2 hours or so? We did have an issue for a bit, but it’s now been resolved

Site offline after rebuild (4th Feb 2025)

Sorry for the disruption everyone! The fix is now live, so running another ./launcher rebuild app should bring things back online. Please let us know if you’re still seeing any issues after that.

[tomtjes (Thomas Reintjes)]

Yes, rebuilt twice within the last hour

[david (David Taylor)]

@tomtjes

now the website can’t be reached

What error are you seeing when trying to access the site?

[tomtjes (Thomas Reintjes)]

“Safari can’t connect to the server”

[david (David Taylor)]

On your server, can you try running

cd /var/discourse
git rev-parse HEAD

It should say

# git rev-parse HEAD
efd6bc8ccbd9057c04d909ff48ecf8b704cae3ff

[tomtjes (Thomas Reintjes)]

yes, it does

[david (David Taylor)]

Is there anything ‘unusual’ about your installation? Anywhere that you’ve deviated from the standard install?

The issue you’re describing sounds a lot like the one in the topic I linked. But if you’re seeing the efd6bc8... for the discourse_docker version, then you should have the fix

[tomtjes (Thomas Reintjes)]

Nothing I can think of, except I have /var/discourse mounted from a disk in fstab. But it’s been running like that for years.

Well, thanks for your help. Let me know if anything else comes to mind.

[david (David Taylor)]

To check whether the same issue is affecting you, you could do something like

./launcher enter app
cat /etc/nginx/conf.d/discourse.conf

Somewhere in that file should be

listen 443 ssl;

If it’s missing, then that’s the same symptom as the other topic.

If it’s present… then I’m afraid we have something totally different to figure out

[tomtjes (Thomas Reintjes)]

yeah, that’s there

[tomtjes (Thomas Reintjes)]

Finally looked into it again and did docker logs app right after another rebuild. Can anyone help me make sense of this?

run-parts: executing /etc/runit/1.d/00-ensure-links
run-parts: executing /etc/runit/1.d/00-fix-var-logs
run-parts: executing /etc/runit/1.d/01-cleanup-web-pids
run-parts: executing /etc/runit/1.d/anacron
run-parts: executing /etc/runit/1.d/cleanup-pids
Cleaning stale PID files
run-parts: executing /etc/runit/1.d/copy-env
run-parts: executing /etc/runit/1.d/letsencrypt
[Wed Feb 12 05:03:53 PM UTC 2025] Domains not changed.
[Wed Feb 12 05:03:53 PM UTC 2025] Skip, Next renewal time is: 2025-04-02T00:09:20Z
[Wed Feb 12 05:03:53 PM UTC 2025] Add '--force' to force to renew.
[Wed Feb 12 05:03:53 PM UTC 2025] Installing key to: /shared/ssl/redacteddomain.tld.key
[Wed Feb 12 05:03:53 PM UTC 2025] Installing full chain to: /shared/ssl/redacteddomain.tld.cer
[Wed Feb 12 05:03:53 PM UTC 2025] Run reload cmd: sv reload nginx
warning: nginx: unable to open supervise/ok: file does not exist
[Wed Feb 12 05:03:53 PM UTC 2025] Reload error for :
[Wed Feb 12 05:03:53 PM UTC 2025] Domains not changed.
[Wed Feb 12 05:03:53 PM UTC 2025] Skip, Next renewal time is: 2025-04-02T00:09:25Z
[Wed Feb 12 05:03:54 PM UTC 2025] Add '--force' to force to renew.
[Wed Feb 12 05:03:54 PM UTC 2025] Installing key to: /shared/ssl/redacteddomain.tld_ecc.key
[Wed Feb 12 05:03:54 PM UTC 2025] Installing full chain to: /shared/ssl/redacteddomain.tld_ecc.cer
[Wed Feb 12 05:03:54 PM UTC 2025] Run reload cmd: sv reload nginx
warning: nginx: unable to open supervise/ok: file does not exist
[Wed Feb 12 05:03:54 PM UTC 2025] Reload error for :
Started runsvdir, PID is 535
warning: redis: unable to open supervise/ok: file does not exist
nginx: [warn] duplicate extension "wasm", content type: "application/wasm", previous content type: "application/wasm" in /etc/nginx/conf.d/discourse.conf:4
nginx: [emerg] unknown "public" variable
ok: run: redis: (pid 550) 1s
ok: run: postgres: (pid 554) 1s
nginx: [warn] duplicate extension "wasm", content type: "application/wasm", previous content type: "application/wasm" in /etc/nginx/conf.d/discourse.conf:4
nginx: [emerg] unknown "public" variable
supervisor pid: 568 unicorn pid: 574
nginx: [warn] duplicate extension "wasm", content type: "application/wasm", previous content type: "application/wasm" in /etc/nginx/conf.d/discourse.conf:4
nginx: [emerg] unknown "public" variable
nginx: [warn] duplicate extension "wasm", content type: "application/wasm", previous content type: "application/wasm" in /etc/nginx/conf.d/discourse.conf:4

[tomtjes (Thomas Reintjes)]

Seems like it was caused by this:

Setup Let’s Encrypt + non-www > www

I finally made it work, and thought to put everything together in case someone needs it. This info is assembled from different topics on this forum. Thanks to @brahn and @techAPJ. This assumes you’ve completed the initial Let’s Encrypt setup: After you’ve done it, the following needs to be added to the app.yml in hooks section. It will do two things: setup SSL for both example.com and www.example.com. Just replace example.com with your domain name. redirect example.com to www.example.com I…