Using Nginx Proxy Manager to manage multiple sites with Discourse

EDIT: @pfaffman re-wrote this as a standalone topic from what @tophee had written earlier. It’s not yet tested by me, and I’ve moved around Chris’s words, so any mistakes are likely those of @pfaffman.

Are there any reasons not to use NGINX Proxy Manager instead of doing this manually as described in Running other websites on the same machine as Discourse?

I’m already using it. I’ve had it on my home server for a while and when I migrated my discourse instance to a new cloud server, I realized that I’ve forgotten most of the NGINX reverse proxy setup that I did 4 years ago on the old server so I thought: why not do it with NPM? To my surprise I found that it has been mentioned rather rarely here on meta so I started wondering whether there are sone downsides that I might have missed…

Indeed, this required a bit of trial and error, but I got it to work as follows (no guarantee that this is the best way of doing it - in fact, I know that there must be a better way - so corrections and improvements are very welcome):

To start with, there are two ways of accessing your discourse instance: 1. by exposing a port, 2. via websocket. I believe I learned somewhere on this forum that the websocket is faster/ more efficient so this is what I’m using, but exposing a port should be a lot easier, so if you can’t get the socket to work, try exposing a port (more on that below).

So let’s assume you have completed the 30-minute standard installation and let’s assume that you didn’t let discourse acquire a lets encrypt cert yet - because you don’t need it when using a reverse proxy. NPM will take care of that. It doesn’t matter, though, if you already have certificate. NPM will simply get a new one.

1. Install NPM

Next step is to install NPM so that you will have two more docker containers running (NPM and its database container).

Next is the tricky part that you’re asking about.

The first obstacle is that discourse runs on the default docker bridge network while NPM by default runs on a default “user created network” (called npm_default in my case) which means that NPM can’t see discourse. :cry:

2. Bring all containers into the default bridge network

So as long as I don’t know if and how discourse can be moved to a custom network, we have to move NPM into the default bridge network. We can do this by adding network_mode: bridge to both NPM containers in our docker compose file.

3. Use IP address instead of service name

The next problem is that the standard docker compose file won’t work any more if you just move it to the bridge network. NPM won’t be able to find its database container anymore. This is because internal DNS resolution for service names (which the docker-compose file relies on) is only available on user created networks, not on the default docker networks. So we have to resort to hard coded IP addresses (which is why this is definitely not the optimal solution because it will break if your container IPs change). So you need to start the container even though you know it wont work, note the IP of the NPM database container, and replace DB_MYSQL_HOST: "db" in your docker compose file with DB_MYSQL_HOST: "<db_container_IP>".

So now all containers should be on the default bridge network so that NPM can see both discourse and its database.

4. Make discourse accessible

But “seeing” discourse and being able to access it is not the same thing. So you need to make sure that discourse will accept whatever traffic NPM forwards to it. If you don’t care about using the websocket, I suppose you can just point NPM to port 80 (not 443) of your discourse container-IP, like this:

Havn’t tested this, though. As I mentioned, I’m using the websocket setup, which requires some further steps. Note that the hostname/IP and port above will be ignored when you use the websocket.

5. Configure app.yml to use websocket

This is explained in the OP, so I won’t go into this.

6. Mount websocket in NPM container

We need to give NPM access to the websocket by mounting it as a volume: - /var/discourse/shared/standalone/nginx.http.sock:/var/discourse/shared/standalone/nginx.http.sock. This is the final change to the default NPM docker compose file, so here is the final version that works for me:

version: '3'
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    network_mode: bridge
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    environment:
      DB_MYSQL_HOST: "172.17.0.6"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "my-super-safe-pwd"
      DB_MYSQL_NAME: "npm"
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
      - /var/discourse/shared/standalone/nginx.http.sock:/var/discourse/shared/standalone/nginx.http.sock
  db:
    image: 'jc21/mariadb-aria:latest'
    restart: unless-stopped
    network_mode: bridge
    environment:
      MYSQL_ROOT_PASSWORD: 'my-super-safe-pwd'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'my-super-safe-pwd'
    volumes:
      - ./data/mysql:/var/lib/mysql

7. Configure NPM to use the websocket

Last step: tell NPM to use the websocket. As far as I remember it wasn’t enough to just turn on “Websockets support” so I copied the NGINX location from the OP to the “Advanced” tab, like this:

I did not get this to work under the “Custom locations” tab.

8. Don’t forget to activate SSL

I didn’t mention the SSL configuration in NPM because it seems pretty self-evident and I don’t think it matters at which point in the process you activate it. So if you haven’t done it yet, this is what mine looks like:

image


9. Final disclaimer

As I wrote this post, it suddenly occurred to me that NPM and discourse probably don’t even need to be on the same docker network when we use the websocket. I don’t have time to check this at the moment, but if this is true, then you can just forget about steps 2, 3 and 4 above and it should work.

This is the most fascinating aspect of support forums: doing a good job in describing your problem often leads you to the solution without even posting your question. And in this case, I was answering someone else’s question but also might have found the answer to my own. :smiley:

4 Likes

Discussing which reverse proxy you should use is clearly beyond the supported area. :wink: But after looking at NGINX Proxy Manager for 12 seconds or more, I’d think that it’ll do fine. There’s another automagic NGINX proxy thing that I’ve seen discussed before as well, but with my extensive knowledge of NGINX Proxy Manager, I think it might be better. I might take a look at it, as I’m growing disenchanted with Traefik.

EDIT: Now I’ve looked at it for 3 minutes. I prefer tools that facilitate automation, so Traefik and Docker Hub (I found it) that allow you to add some labels or environment variables to the docker container so that no clicking in a web interface is required are more what I want, and it looks like to make that thing work you have to use the web interface or contrive to update the database manually with hosts that you want to add. But if you’re willing to poke and click on a web interface like a Normal Person (rather than a system administrator “person”), then it looks like that thing could be just what you’re looking for.

4 Likes

I’ve tried installing Nginx Proxy Manager but I can’t understand how to “link” (proxy? sorry but I’m quite new to system management) the Discourse container in order to see Discourse when I access from web.
Can you give same hints? Should the Discourse container expose 80 and 443 ports or not, like this forum topic suggests?

1 Like

Indeed, this required a bit of trial and error, but I got it to work as follows (no guarantee that this is the best way of doing it - in fact, I know that there must be a better way - so corrections and improvements are very welcome):

Instructions that @pfaffman moved to OP

To start with, there are two ways of accessing your discourse instance: 1. by exposing a port, 2. via websocket. I believe I learned somewhere on this forum that the websocket is faster/ more efficient so this is what I’m using, but exposing a port should be a lot easier, so if you can’t get the socket to work, try exposing a port (more on that below).

So let’s assume you have completed the 30-minute standard installation and let’s assume that you didn’t let discourse acquire a lets encrypt cert yet - because you don’t need it when using a reverse proxy. NPM will take care of that. It doesn’t matter, though, if you already have certificate. NPM will simply get a new one.

1. Install NPM

Next step is to install NPM so that you will have two more docker containers running (NPM and its database container).

Next is the tricky part that you’re asking about.

The first obstacle is that discourse runs on the default docker bridge network while NPM by default runs on a default “user created network” (called npm_default in my case) which means that NPM can’t see discourse. :cry:

2. Bring all containers into the default bridge network

So as long as I don’t know if and how discourse can be moved to a custom network, we have to move NPM into the default bridge network. We can do this by adding network_mode: bridge to both NPM containers in our docker compose file.

3. Use IP address instead of service name

The next problem is that the standard docker compose file won’t work any more if you just move it to the bridge network. NPM won’t be able to find its database container anymore. This is because internal DNS resolution for service names (which the docker-compose file relies on) is only available on user created networks, not on the default docker networks. So we have to resort to hard coded IP addresses (which is why this is definitely not the optimal solution because it will break if your container IPs change). So you need to start the container even though you know it wont work, note the IP of the NPM database container, and replace DB_MYSQL_HOST: "db" in your docker compose file with DB_MYSQL_HOST: "<db_container_IP>".

So now all containers should be on the default bridge network so that NPM can see both discourse and its database.

4. Make discourse accessible

But “seeing” discourse and being able to access it is not the same thing. So you need to make sure that discourse will accept whatever traffic NPM forwards to it. If you don’t care about using the websocket, I suppose you can just point NPM to port 80 (not 443) of your discourse container-IP, like this:

Havn’t tested this, though. As I mentioned, I’m using the websocket setup, which requires some further steps. Note that the hostname/IP and port above will be ignored when you use the websocket.

5. Configure app.yml to use websocket

This is explained in the OP, so I won’t go into this.

6. Mount websocket in NPM container

We need to give NPM access to the websocket by mounting it as a volume: - /var/discourse/shared/standalone/nginx.http.sock:/var/discourse/shared/standalone/nginx.http.sock. This is the final change to the default NPM docker compose file, so here is the final version that works for me:

version: '3'
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    network_mode: bridge
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    environment:
      DB_MYSQL_HOST: "172.17.0.6"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "my-super-safe-pwd"
      DB_MYSQL_NAME: "npm"
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
      - /var/discourse/shared/standalone/nginx.http.sock:/var/discourse/shared/standalone/nginx.http.sock
  db:
    image: 'jc21/mariadb-aria:latest'
    restart: unless-stopped
    network_mode: bridge
    environment:
      MYSQL_ROOT_PASSWORD: 'my-super-safe-pwd'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'my-super-safe-pwd'
    volumes:
      - ./data/mysql:/var/lib/mysql

7. Configure NPM to use the websocket

Last step: tell NPM to use the websocket. As far as I remember it wasn’t enough to just turn on “Websockets support” so I copied the NGINX location from the OP to the “Advanced” tab, like this:

I did not get this to work under the “Custom locations” tab.

8. Don’t forget to activate SSL

I didn’t mention the SSL configuration in NPM because it seems pretty self-evident and I don’t think it matters at which point in the process you activate it. So if you haven’t done it yet, this is what mine looks like:

image


9. Final disclaimer

As I wrote this post, it suddenly occurred to me that NPM and discourse probably don’t even need to be on the same docker network when we use the websocket. I don’t have time to check this at the moment, but if this is true, then you can just forget about steps 2, 3 and 4 above and it should work.

This is the most fascinating aspect of support forums: doing a good job in describing your problem often leads you to the solution without even posting your question. And in this case, I was answering someone else’s question but also might have found the answer to my own. :smiley:

4 Likes

Thanks a lot for the great and articulated answer! I will try asap you hints. By the way, the ip to put in NPM are those of the server (i.e. the external ip to reach the server) or the Docker internal (I noticed Docker uses 172… for containers)?
Sorry if my questions seems trivial, I don’t know well net matters.

1 Like

I recommend that you use the websocket. Then you can put anything as the IP because it won’t be used. Otherwise it is the internal IP of the container, not the public IP of the host.

BTW: Looks like your reply-by-email wasn’t rendered correctly. Maybe you want to edit (shorten) your post above?

2 Likes

Yes I saw, I used “reply” from the email and it tried to put all original message, but I can’t find a way to modify it, is it possible? I’m new to Discourse even as a user… :pensive:

1 Like

You can edit your posts using the pencil icon at the bottom of your message. However… Trust Level 1 and below only get 24 hours (default), whereas Trust Level 2 and above get 30 days (default).

It looks like you’re on TL1 Basic at the moment, but you can find out more about what you need to do to ‘level up’ in Trust Levels. :+1:

2 Likes

Sorry if it’s quite a while from previous question, but I lost a lot of time trying to do what you suggested.
With no success at all. When I modified app.yml and rebuild app with you changes, it started displaying in logs “config/unicorn_launcher: line 71: kill: (898) - No such process”, so, whatever I tried, no way to stop this behavior. I also tried rebuilding app in its original state (with ports exposed, without websocket), stopping npm, but no chance at all, “unicorn” is not running.
I also tried following evertything google finds about this (it seems to be a diffused problem), but in no way I could figure how to rebuild a working discourse container. Problem now it’s (one of the most, I’m near to give up with discourse) that for some obscure and messed reason, internal postgres is always “restarting”.
I don’t know why, I simple made you changes, rebuilt app and from then, “unicorn” :roll_eyes: is dead.
Is there a way to fix this postgres problem? Why this happened? There’s a chance (I’m afraid not at all!) to don’t loose every changes I made to discourse when it was working?
And btw, it’s normal that every little change or trying to fix something, must end with something else not working? :rage:
I’m not angry, it’s all about discourse, not your suggestions: I spent a lot of time trying to make this “apparently” nice forum working, but every time there’s something else going wrong, my feelings that discourse is something very little reliable are getting stronger.

1 Like

If you had a working standard install, you should be able to put things back that way and have it all still work.

The postgres issue might be related to PostgreSQL 13 update?

If you made a backup before you started you can install on a fresh server and restore that backup. That should be a worst case scenario.

2 Likes

How can I know if postgres issue is related to 13 update? I didn’t choose to update it, I simply digited “./launcher rebuild app”, and all the… things happened.
Yes, it’s version 13, after hours reading on internet of other people with the same problem I discovered this could be the problem, but I didn’t find a way to bring discourse alive.

1 Like

Then that’s not the issue. Sorry.

1 Like

I’m sorry to hear your having these troubles. I know the frustrating feeling of spending hours on trying to fix something. But you also learn something everytime, even if the path to the solution rarely is a straight line…

I’m sorry, but I’m not the right person to help you with this. I have no experience with postgres or unicorns. There are three ways how I make it through such “nothing works” scenarios: 1. backup so that I can always go back to the original state. 2. try/change only one thing at a time and try it on a non-production machine (or less important forum) first. 3. invest even more hours trying to figure things out.

BTW: writing detailed problem descriptions/ support tickets more than once helped me solve the problem. I didn’t even have to submit the ticket. Writing it up made me see the solution.

So in your case, what I’d try to understand is: under what circumstances can changing the app.yml and then changing it back to its original state lead to a different outcome than the original outcome. When looking into this, you either realize that you didn’t actually restore the exact original or you understand what else you need to “reset” in order for it to work.

5 Likes

I’m really sorry but I do not understand: first you asked me if the postgres issue might be related to Postgresql 13 update, and when I answered “yes, it’s version 13” (I swear I didn’t know what was before, lot of time I didn’t rebuild the app), you answer that is not the issue… so, why postgres is always “starting” and discourse don’t go?

1 Like

Hey @Wanderer . I can’t tell what your problem might be, so the postgres upgrade was a wild guess. I’m pretty sure that if you are running postgres 13, then the issue is not that you are stuck getting upgraded from 10 or 12, so while there could be some problem with postgres, it probably is not related to the postgres 13 upgrade.

My best recommendation for someone who isn’t expert in these matters is to do a clean installation and restore your most recent backup.

If you would like to get more specific help for your issue and have a budget, you can contact me or post in #marketplace.

I’m going to work on improving the Nngin Proxy Manager instructions, but my guess is that your problem, though brought to light by trying to move to this complicated setups, likely is not because of these instructions being faulty. (I don’t know, but that’s my best guess.)

2 Likes

Here’s my version of this. I almost gave up, but @tophee linked to a post that I (!?) wrote that provided the necessary magic! This is now a straight-forward way to configure Nginx Proxy Manager for Discourse. I think this makes this similar to Running other websites on the same machine as Discourse - #396.

Install Nginx Proxy Manager per their instructions at

Remove SSL and Let’s Encrypt templates:

See that these lines in your yml file are commented out or deleted:

## Uncomment these two lines if you wish to add Lets Encrypt (https)
#- "templates/web.ssl.template.yml"
#- "templates/web.letsencrypt.ssl.template.yml"

Have Discourse use the npm-default network.

If you blindly follow the Nginx Proxy Manager install instructions, it will create a docker network called npm_default.

Add this stanza to your yml file(s). If you have separate web_only and data containers, you’ll need to add this to each of them (I didn’t test the mail-receiver container.). docker_args is not indented.

docker_args: |
  --network npm_default

No need to expose any ports

Comment out or remove these lines from your yml file:

# expose:
#  - "80:80"   # http
#  - "443:443" # https

You can then rebuild your container(s) and configure Nginx Proxy Manager like this:

image

A simple (but not necessarily recommended) way to spin up a second Discourse site would be this:

cd /var/discourse/containers
cp app.yml othersite.yml
# somehow edit, at a minimum, the hostname in othersite.yml
../launcher rebuild othersite

Then add it to NPM as above, using othersite instead of app.

I tested this with an app.yml plus two web_only-style containers and a single data container plus a separate othersite-redis container that is a copy of the data container containing only the redis templates. (But an easier solution would be to put the extra redis in the web_only container).

1 Like

So, after some struggling, I managed to make all the stuff working.

I must premise: although I’m a “old generation” developer (first '80), I always looked for the best and new form to develop or manage, so I really hate in 2021 to write weird commands full of cryptic options like with the old cp/m - dos, I always look for some interface that make my life easier and clearer.

Then, I use e.g. Portainer to manage containers, it permits to start/stop/edit/duplicate all containers on the fly, without “bashing” up and down the filesystem in search of one-in-a-million file; e.g. to change the container network it’s just to choose one from a list and make a click, idem for adding some parameters, or a volume like in the example @tophee wrote; for this reason I tried NPM, because I prefer to “contain” my Nginx proxy and because apparently with just a few click, well understanding what I’m doing but without to remember a new set of weird command-and-options.

Back to my Discourse container, I had to “discourse-setup” it again; evertything went right, “evil” Postgres installed in version 13, no “drunk unicorn” (I’m sorry but thinking about an “unicorn” running in my server makes me laugh! :laughing:), in short all went the right way. Then, I made the modification in order to make Discourse running with websocket: everything ok also this time. Fortunately, previous Discourse setup made automatic backups, so with just few clicks I restored everything (the most I use Discourse, the most I love it!).
I had to try several times the setting for NPM, in the beginning I had some problem with certificates, but in the end even it run well.

I added a second proxy pointing to my Wordpress container (yes, I’m “containing” everything, I like the idea of a most clean server with all the main packages contained in a managed place), and even it run well.

So, in the end, I have my server (a VPS) with its email server (I also tried to “contain” it, but after weeks of harsh fight, I gave up), Discourse pointing to it, Worpress running in another container, and NPM managing both: all on my server, without depending on other (and much, much more expensive) services for deploing, emailing, etc.

Next step, importing some hundred thousand post from the “old good Phpbb”: prepare yourself for other posts from me! :grinning_face_with_smiling_eyes:

A great thank to @tophee and @pfaffman for the help, I can understand how difficult can it be to help people when they work in non standard way like me.

3 Likes

Glad you were able to get it to work with Websocket. For anyone else struggling with that, go for @pfaffman 's instructions for doing it without websocket above.

I don’t know what caused your problem, but it strikes me that this might be a point to clarify for anyone who is relatively new to discourse administration: you need to understand how the let’s encrypt certificate works in the default installation (w/o any outer proxy) vs how it works with NPM (and if you wonder why its called outer proxy, you need to figure that out too).

Since I originally set up my outer proxy manually and also set up let’s encrypt manually, I had an understanding for all the magic that discourse as well as NPM do for you so that https just works, and therefore I was able to avoid various pitfalls when switching from discourse-managed certificates to NPM-managed certificates.

I don’t really see why you would want to put the mail server behind NPM…

1 Like

No Christoph, not behind NPM, just on a container. I tried Zimbra, but was a huge mess. Than a simple “contained” Postfix, but with no success too. I was in the beginning of my Linux experience (I’m still a beginner, but I’m getting more and more confident at least regarding some administration concept), so I gave up and tried directly on the server, it starts without great problems so I proceeded that way, even though it was quite hard to configure Discourse to use my email server. But now, evertything seems to work.

2 Likes