Discourse konfigurieren - Umgebung AWS Linux 2 AMI und Apache2 (httpd)

Hi All

I have been through the docs and read through the configuration setups and recommendations posted and have tried to modify the same for my unique situation.

For something that appears to be quite simple, it is incredibly frustrating that none of the proposed solutions have worked.

Our environment:

  • We run an AWS EC2 instance, more specifically an AWS Linux 2 AMI;
  • We have apache2 running and configured;
  • The instance is part of an AWS target group which is referenced by our AWS load balancer;
  • Our SSL / TLS certificate is AWS provided and is associated with the load balancer;
  • Our domain provider is name.com;
  • As part of our setup we have multiple sub domains routed using CNAME records all pointing to the AWS load balancer which then manages traffic to the EC2 instance;
  • All traffic routed to the load balancer and served to the web user is secure;
  • On our server we run three separate node express applications;
  • The ports are defined in express as part of the express server setup, currently ports 3000, 4000 and 5000;
  • Traffic is routed by apache using the sub domain to the relevant web app, for example:
<VirtualHost *:80>
    ServerName subdomain.domain.io
    ProxyPreserveHost On
    ProxyPass "/" "http://localhost:4000/"
    ProxyPassReverse "/" "http://localhost:4000/"
</VirtualHost>

Current status:

  • Docker is running
  • The discourse app (container) is running
  • The express apps are all running and are being properly routed by apache

Notes:

  • I tried running building the app initially without exposing the ports per the documentation, when that didn’t work I did expose port 8000 over http port 80;
  • In my simple understanding docker is like a house and the various containers are the rooms. Docker determines how the app is accessed i.e. which room to access. My thought by exposing port 8000 was that I could then reference port 8000 as I do with my other express apps now that 8000 was exposed. But this does not work, below is an example of what I tried to do in apache:
# FAQ (DISCOURSE ROUTES)

<VirtualHost *:80>
    ServerName discourse.domain.io
    ProxyPreserveHost On
    ProxyPass "/" "http://localhost:8000/"
    ProxyPassReverse "/" "http://localhost:8000/"
</VirtualHost>
  • The simple test was to try and put localhost:8000 into the browser on the server and test. The server is the AWS guid with chromium installed. The fact that this does not resolve immediately tells me that there is an issue and that my understanding is missing something fundamental. My web apps run and are available on the ports I specify in the express setup, my expectation was that it was the same for the docker container.

Additional notes:

  • I realise that there is a potential solution by putting NGINX “in front of” apache and docker to route traffic from discourse.domain.io to say port 8000 on the local server i.e. the port that is exposed for the docker container and also route all other traffic from *.domain.io to apache on say port 9000 and then change my virtual host config to route traffic from port 9000 accordingly, i.e. :
# EXPRESS APP ROUTING IN APACHE

<VirtualHost *:9000>
    ServerName other_sub_domains.domain.io
    ProxyPreserveHost On
    ProxyPass "/" "http://localhost:4000/"
    ProxyPassReverse "/" "http://localhost:4000/"
</VirtualHost>
  • To me this seems completely unnecessary. If the domain discourse.domain.io is routed to apache and apache receives the request and then tries to route it should be as simple as pointing it to the docker container exposed port.
  • Also if localhost:8000 on the server itself isn’t resolving then this would not work anyway.

Currently I get a 502 error in chrome when I try to navigate to my discourse docker container. (See above screenshot).

Extract from app.yml

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/web.socketed.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "8000:80"   # http
  #- "443:443" # https

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  #db_shared_buffers: "256MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  #UNICORN_WORKERS: 3

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: 'discourse.domain.io'

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: 'user@domain.io'

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: email-smtp.us-east-1.amazonaws.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: ******
  DISCOURSE_SMTP_PASSWORD: ******
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  #DISCOURSE_SMTP_DOMAIN: ******    # (required by some providers)
  #DISCOURSE_NOTIFICATION_EMAIL: ******  # (address to send notifications from)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com
  
  ## The maxmind geolocation IP address key for IP address lookup
  ## see https://meta.discourse.org/t/-/137387/23 for details
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec: echo "End of custom commands"

My domain, email address and email config are correct. As mentioned, I do not require SSL/TLS configuration.

Request:

  • Is the NGINX solution the only way I can get this setup working and if so is there a clearer explanation available with regards to the setup steps? I am not in a position to start with the server from scratch just to get discourse working and with no guarantees.
  • In Discourse behind reverse proxy and https - #2 by itsbhanusharma @Dark Matter provided the following as an apache2 solution, but I do not believe this solution has a node express environment in mind and with my localhost:8000 not resolving I am not sure if this would work:
<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  ServerName  discourse.example.com
  DocumentRoot /website/discourse

  RewriteEngine On
  ProxyPreserveHost On
  ProxyRequests Off
  ProxyPass / unix:/var/discourse/shared/socket-only/nginx.http.sock|http://localhost/
  ProxyPassReverse  / unix:/var/discourse/shared/socket-only/nginx.http.sock|http://localhost/
  ErrorLog /var/log/apache2/discourse.error.log
  LogLevel warn
  CustomLog /var/log/apache2/discourse.access.log combined

  RewriteCond %{SERVER_NAME} =discourse.example.com
  RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
  • Is the issue here with my discourse config? Is the issue perhaps my apache routing (I suspect)? Or am I missing something more fundamental here?
  • I do think that resolving this will help a lot of users in the future.
  • First prize would be to route to discourse on the server locally.
  • Then routing via the url would be the next prize.

Best regards
Matthew Lucas

Das ist zu kompliziert für mich, um es vollständig zu verstehen, betrachten Sie dies also eher als eine wilde Vermutung denn als eine Lösung. Wenn das Problem darin besteht, dass Sie etwas nicht über localhost mit Ihrem laufenden Docker-Container verbinden können, könnten Sie versuchen, die Docker-IP zu verwenden. Es könnte auch möglich sein, die WebSocket-Vorlage zu verwenden und den Socket anstelle des Ports zu verwenden. Einige argumentieren, dass dies aus einer Reihe von Gründen vorzuziehen ist.

MKJ’s Opinionated Discourse Deployment Configuration könnte nützlich sein zu lesen.

@pfaffman@Matthew_Lucas verwendet bereits die Socket-Vorlage, aber…

Ich verwende definitiv localhost für meinen externen Proxy, und das sollte mit der Einstellung expose funktionieren. Ich würde nicht erwarten, dass er die Docker-IP verwenden muss. Und ich würde nicht erwarten, dass dies aus diesem Grund nur mit Apache vor dem Docker-Container fehlschlägt.
Sie benötigen möglicherweise eine Header-Konfiguration – siehe Add an offline page to display when Discourse is rebuilding or starting up - #2 by codinghorror

Ich habe mein Bestes getan, um die Apache-Konfiguration zu vergessen, aber dieser Teil der Nginx-Konfiguration ist etwas, das Sie meiner Meinung nach mit Apache nachbilden möchten.

    proxy_set_header Host $http_host;
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Real-IP $remote_addr;

Insbesondere glaube ich nicht, dass es ohne den Host-Header richtig funktioniert.
Da Sie kein HTTPS verwenden, sollten Sie mit tcpdump Paketspuren erfassen können, um genau zu sehen, was schief geht.

ABER warum überhaupt Apache dort einbauen? Es ist nur ein zusätzlicher Umweg. Wenn ich so etwas versuchen würde, würde ich den Amazon Load Balancer so einstellen, dass er direkt mit Port 8000 auf der EC2-Instanz spricht. Ich gehe davon aus, dass Sie HTTPS ohnehin am ELB beenden. Ich vermute, dass ihr Load Balancer die normalen Header kennt, die hinzugefügt werden müssen, ohne dass Sie sie angeben müssen, aber wenn nicht, geben Sie sie offensichtlich an. Stellen Sie einfach sicher, dass der Port über die Firewall zugänglich ist.

Ich bezweifle, dass ich hier noch viel Wertvolles beitragen kann, aber hoffentlich hilft Ihnen einiges davon bei Ihrer Konfiguration. Viel Glück.

1 „Gefällt mir“

INSTALLATION OF DOCKER AND DISCOURSE ON AWS EC2 LINUX WITH EXISTING HTTPD REVERSE PROXY

Hi All

I have figured it out. I went back to the drawing board and started fresh with clean docker and discourse install. My process is detailed below:

Assumptions

• You already have an AWS EC2 Linux instance
• You can access your EC2 instance using SSH
• Apache 2 is already configured and running on your server

ACCESS YOUR EC2 INSTANCE USING SSH

  1. Open CMD / Terminal

  2. SSH into AWS EC2 instance

Example of screen once connected to EC2 instance

image

PERFORM SYSTEM UPDATES

  1. Perform system updates by running the following command in the console / terminal

$ sudo yum update

INSTALL DOCKER

  1. Search for the AWS Docker package by running the following command in the console / terminal

$ sudo yum search docker

  1. Get version information by running the following command in the console / terminal

$ sudo yum info docker

  1. Install docker by running the following command in the console / terminal

$ sudo yum install docker

  1. Add group membership for the default ec2-user so you can run all docker commands without using the sudo command by running the following command in the console / terminal
$ sudo usermod -a -G docker ec2-user
$ id ec2-user

# Reload a Linux user's group assignments to docker without logout

$ newgrp docker
  1. Install docker-compose by running the following command in the console / terminal
# 1. Get pip3 (python installer package)
$ sudo yum install python3-pip
 
# 2. Then run any one of the following
$ sudo pip3 install docker-compose # with root access
 
# OR
 
$ pip3 install --user docker-compose # without root access for security reasons

# Check permissions
$ sudo chmod -v +x /usr/local/bin/docker-compose
  1. Enable docker service to start automatically on boot by running the following command in the console / terminal

$ sudo systemctl enable docker.service

  1. Start the docker service by running the following command in the console / terminal

$ sudo systemctl start docker.service

  1. Check the docker service is running by running the following command in the console / terminal, the output will look similar to this

Basic docker commands that are good to know :

# Docker version
$ docker version

# Docker compose version
$ docker-compose version

# Start Docker service
$ sudo systemctl start docker.service

# Stop Docker service
$ sudo systemctl stop docker.service

# Restart Docker service
$ sudo systemctl restart docker.service

# Docker service status
$ sudo systemctl status docker.service

INSTALL DISCOURSE

  1. Create discourse folder

$ sudo mkdir /var/discourse

  1. Clone the discourse docker image to the newly created directory

$ sudo git clone https://github.com/discourse/discourse_docker.git /var/discourse

  1. Copy the standalone.yml file to the containers folder

$ cp /var/discourse/samples/standalone.yml /var/discourse/containers/discourse.yml

  1. Edit the new discourse.yml file as follows, (the important bits exposed ports for http and email details for smtp, modify in the file don’t copy and paste)

Editing options:

1. You can use nano and edit via command line / terminal
2. On AWS if you are using MATE with PLUMA, you can use PLUMA to edit the file
3. You can use WINSCP to SSH into the machine and edit the file via the Windows GUI

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  ## Uncomment the next line to enable the IPv6 listener
  #- "templates/web.ipv6.template.yml"
  - "templates/web.ratelimited.template.yml"
  ## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "8080:80"   # http (run on local port 8080, docker port 80)
  #- "443:443" # https (make sure this is commented out)

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  #db_shared_buffers: "256MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  #UNICORN_WORKERS: 3

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: 'faq.mobiloan.io'

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: ‘ enter email here ’

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: ENTER SMTP HERE
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: ENTER USERNAME HERE
  DISCOURSE_SMTP_PASSWORD: ENTER PASSWORD HERE
  DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true, we use Amazon SES)

 

  #DISCOURSE_SMTP_DOMAIN: discourse.example.com    # (required by some providers)
  #DISCOURSE_NOTIFICATION_EMAIL: noreply@discourse.example.com    # (address to send notifications from)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com
  
  ## The maxmind geolocation IP address key for IP address lookup
  ## see https://meta.discourse.org/t/-/137387/23 for details
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec: echo "End of custom commands"
  1. Once the file is edited and saved, rebuild the discourse app using the following command in the terminal

$ /var/discourse/launcher rebuild discourse

Note if your yaml file is called something else then you would replace discourse with your yaml file name

Note that the rebuild takes time.

  1. One your app has been rebuilt the app should be running on port 8080. To verify this run the following command in the terminal

$ sudo lsof -i -P -n | grep LISTEN

Example output of ports

Notice docker is listening on port 8080, httpd (apache) is listening on port 80.

In terms of traffic routing apache will receive the web request and where appropriate will be routed to the discourse container on port 8080.

To make sure the routing happens you must configure the virtual host in the apache conf file.

APACHE VIRTUAL HOST CONFIGURATION

  1. The apache conf file is located at /etc/httpd/conf.d/00-virtualhosts.conf. httpd / apache / apache2 are different apache versions. Your files might be in different locations if you are not using the httpd service but the virtual host config in this case will be the same.

Editing options:

1. You can use nano and edit via command line / terminal
2. On AWS if you are using MATE with PLUMA, you can use PLUMA to edit the file
3. You can use WINSCP to SSH into the machine and edit the file via the Windows GUI

Edit the /etc/httpd/conf.d/00-virtualhosts.conf file by adding the following:

# FAQ (DISCOURSE ROUTES)

<VirtualHost *:80>
  	ServerName  sub.domain.com
  	ProxyPreserveHost On
    ProxyPass "/" "http://localhost:8080/"
    ProxyPassReverse "/" "http://localhost:8080/"
</VirtualHost>
  1. You need to create the relevant CNAME DNS record having the sub domain pointing to your AWS server / load balancer.

Our setup is a little complicated as we have our domain provider who points routes all traffic to an AWS load balancer.

The load balancer handles our SSL/TLS through an AWS provided certificate.

The loan balancer also routes traffic to the AWS instance.

Apache is installed on the AWS instance and routes the traffic to our:

  • Node express applications
  • Docker / discourse application.

CONCLUSION

This is only one solution, but it is the one that worked with our existing setup.

A big thank you to:

@Kane York for his article on Discourse,
@Axel Fernandes’ article on medium his notes on downloading the necessary discourse files and
@Vivek Gite from nixCraft and his article on cyberciti on installing Docker on AWS Linux 2.