Indeed, this required a bit of trial and error, but I got it to work as follows (no guarantee that this is the best way of doing it - in fact, I know that there must be a better way - so corrections and improvements are very welcome):
Instructions that @pfaffman moved to OP
To start with, there are two ways of accessing your discourse instance: 1. by exposing a port, 2. via websocket. I believe I learned somewhere on this forum that the websocket is faster/ more efficient so this is what I’m using, but exposing a port should be a lot easier, so if you can’t get the socket to work, try exposing a port (more on that below).
So let’s assume you have completed the 30-minute standard installation and let’s assume that you didn’t let discourse acquire a lets encrypt cert yet - because you don’t need it when using a reverse proxy. NPM will take care of that. It doesn’t matter, though, if you already have certificate. NPM will simply get a new one.
1. Install NPM
Next step is to install NPM so that you will have two more docker containers running (NPM and its database container).
Next is the tricky part that you’re asking about.
The first obstacle is that discourse runs on the default docker bridge network while NPM by default runs on a default “user created network” (called npm_default in my case) which means that NPM can’t see discourse. 
2. Bring all containers into the default bridge network
So as long as I don’t know if and how discourse can be moved to a custom network, we have to move NPM into the default bridge network. We can do this by adding network_mode: bridge to both NPM containers in our docker compose file.
3. Use IP address instead of service name
The next problem is that the standard docker compose file won’t work any more if you just move it to the bridge network. NPM won’t be able to find its database container anymore. This is because internal DNS resolution for service names (which the docker-compose file relies on) is only available on user created networks, not on the default docker networks. So we have to resort to hard coded IP addresses (which is why this is definitely not the optimal solution because it will break if your container IPs change). So you need to start the container even though you know it wont work, note the IP of the NPM database container, and replace DB_MYSQL_HOST: "db" in your docker compose file with DB_MYSQL_HOST: "<db_container_IP>".
So now all containers should be on the default bridge network so that NPM can see both discourse and its database.
4. Make discourse accessible
But “seeing” discourse and being able to access it is not the same thing. So you need to make sure that discourse will accept whatever traffic NPM forwards to it. If you don’t care about using the websocket, I suppose you can just point NPM to port 80 (not 443) of your discourse container-IP, like this:
Havn’t tested this, though. As I mentioned, I’m using the websocket setup, which requires some further steps. Note that the hostname/IP and port above will be ignored when you use the websocket.
5. Configure app.yml to use websocket
This is explained in the OP, so I won’t go into this.
6. Mount websocket in NPM container
We need to give NPM access to the websocket by mounting it as a volume: - /var/discourse/shared/standalone/nginx.http.sock:/var/discourse/shared/standalone/nginx.http.sock. This is the final change to the default NPM docker compose file, so here is the final version that works for me:
version: '3'
services:
app:
image: 'jc21/nginx-proxy-manager:latest'
restart: unless-stopped
network_mode: bridge
ports:
- '80:80'
- '81:81'
- '443:443'
environment:
DB_MYSQL_HOST: "172.17.0.6"
DB_MYSQL_PORT: 3306
DB_MYSQL_USER: "npm"
DB_MYSQL_PASSWORD: "my-super-safe-pwd"
DB_MYSQL_NAME: "npm"
volumes:
- ./data:/data
- ./letsencrypt:/etc/letsencrypt
- /var/discourse/shared/standalone/nginx.http.sock:/var/discourse/shared/standalone/nginx.http.sock
db:
image: 'jc21/mariadb-aria:latest'
restart: unless-stopped
network_mode: bridge
environment:
MYSQL_ROOT_PASSWORD: 'my-super-safe-pwd'
MYSQL_DATABASE: 'npm'
MYSQL_USER: 'npm'
MYSQL_PASSWORD: 'my-super-safe-pwd'
volumes:
- ./data/mysql:/var/lib/mysql
7. Configure NPM to use the websocket
Last step: tell NPM to use the websocket. As far as I remember it wasn’t enough to just turn on “Websockets support” so I copied the NGINX location from the OP to the “Advanced” tab, like this:
I did not get this to work under the “Custom locations” tab.
8. Don’t forget to activate SSL
I didn’t mention the SSL configuration in NPM because it seems pretty self-evident and I don’t think it matters at which point in the process you activate it. So if you haven’t done it yet, this is what mine looks like:
9. Final disclaimer
As I wrote this post, it suddenly occurred to me that NPM and discourse probably don’t even need to be on the same docker network when we use the websocket. I don’t have time to check this at the moment, but if this is true, then you can just forget about steps 2, 3 and 4 above and it should work.
This is the most fascinating aspect of support forums: doing a good job in describing your problem often leads you to the solution without even posting your question. And in this case, I was answering someone else’s question but also might have found the answer to my own. 