Discourse の設定 - 環境 AWS Linux 2 AMI および Apache2 (httpd)

Hi All

I have been through the docs and read through the configuration setups and recommendations posted and have tried to modify the same for my unique situation.

For something that appears to be quite simple, it is incredibly frustrating that none of the proposed solutions have worked.

Our environment:

  • We run an AWS EC2 instance, more specifically an AWS Linux 2 AMI;
  • We have apache2 running and configured;
  • The instance is part of an AWS target group which is referenced by our AWS load balancer;
  • Our SSL / TLS certificate is AWS provided and is associated with the load balancer;
  • Our domain provider is name.com;
  • As part of our setup we have multiple sub domains routed using CNAME records all pointing to the AWS load balancer which then manages traffic to the EC2 instance;
  • All traffic routed to the load balancer and served to the web user is secure;
  • On our server we run three separate node express applications;
  • The ports are defined in express as part of the express server setup, currently ports 3000, 4000 and 5000;
  • Traffic is routed by apache using the sub domain to the relevant web app, for example:
<VirtualHost *:80>
    ServerName subdomain.domain.io
    ProxyPreserveHost On
    ProxyPass "/" "http://localhost:4000/"
    ProxyPassReverse "/" "http://localhost:4000/"
</VirtualHost>

Current status:

  • Docker is running
  • The discourse app (container) is running
  • The express apps are all running and are being properly routed by apache

Notes:

  • I tried running building the app initially without exposing the ports per the documentation, when that didn’t work I did expose port 8000 over http port 80;
  • In my simple understanding docker is like a house and the various containers are the rooms. Docker determines how the app is accessed i.e. which room to access. My thought by exposing port 8000 was that I could then reference port 8000 as I do with my other express apps now that 8000 was exposed. But this does not work, below is an example of what I tried to do in apache:
# FAQ (DISCOURSE ROUTES)

<VirtualHost *:80>
    ServerName discourse.domain.io
    ProxyPreserveHost On
    ProxyPass "/" "http://localhost:8000/"
    ProxyPassReverse "/" "http://localhost:8000/"
</VirtualHost>
  • The simple test was to try and put localhost:8000 into the browser on the server and test. The server is the AWS guid with chromium installed. The fact that this does not resolve immediately tells me that there is an issue and that my understanding is missing something fundamental. My web apps run and are available on the ports I specify in the express setup, my expectation was that it was the same for the docker container.

Additional notes:

  • I realise that there is a potential solution by putting NGINX “in front of” apache and docker to route traffic from discourse.domain.io to say port 8000 on the local server i.e. the port that is exposed for the docker container and also route all other traffic from *.domain.io to apache on say port 9000 and then change my virtual host config to route traffic from port 9000 accordingly, i.e. :
# EXPRESS APP ROUTING IN APACHE

<VirtualHost *:9000>
    ServerName other_sub_domains.domain.io
    ProxyPreserveHost On
    ProxyPass "/" "http://localhost:4000/"
    ProxyPassReverse "/" "http://localhost:4000/"
</VirtualHost>
  • To me this seems completely unnecessary. If the domain discourse.domain.io is routed to apache and apache receives the request and then tries to route it should be as simple as pointing it to the docker container exposed port.
  • Also if localhost:8000 on the server itself isn’t resolving then this would not work anyway.

Currently I get a 502 error in chrome when I try to navigate to my discourse docker container. (See above screenshot).

Extract from app.yml

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/web.socketed.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "8000:80"   # http
  #- "443:443" # https

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  #db_shared_buffers: "256MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  #UNICORN_WORKERS: 3

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: 'discourse.domain.io'

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: 'user@domain.io'

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: email-smtp.us-east-1.amazonaws.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: ******
  DISCOURSE_SMTP_PASSWORD: ******
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  #DISCOURSE_SMTP_DOMAIN: ******    # (required by some providers)
  #DISCOURSE_NOTIFICATION_EMAIL: ******  # (address to send notifications from)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com
  
  ## The maxmind geolocation IP address key for IP address lookup
  ## see https://meta.discourse.org/t/-/137387/23 for details
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec: echo "End of custom commands"

My domain, email address and email config are correct. As mentioned, I do not require SSL/TLS configuration.

Request:

  • Is the NGINX solution the only way I can get this setup working and if so is there a clearer explanation available with regards to the setup steps? I am not in a position to start with the server from scratch just to get discourse working and with no guarantees.
  • In Discourse behind reverse proxy and https - #2 by itsbhanusharma @Dark Matter provided the following as an apache2 solution, but I do not believe this solution has a node express environment in mind and with my localhost:8000 not resolving I am not sure if this would work:
<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  ServerName  discourse.example.com
  DocumentRoot /website/discourse

  RewriteEngine On
  ProxyPreserveHost On
  ProxyRequests Off
  ProxyPass / unix:/var/discourse/shared/socket-only/nginx.http.sock|http://localhost/
  ProxyPassReverse  / unix:/var/discourse/shared/socket-only/nginx.http.sock|http://localhost/
  ErrorLog /var/log/apache2/discourse.error.log
  LogLevel warn
  CustomLog /var/log/apache2/discourse.access.log combined

  RewriteCond %{SERVER_NAME} =discourse.example.com
  RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
  • Is the issue here with my discourse config? Is the issue perhaps my apache routing (I suspect)? Or am I missing something more fundamental here?
  • I do think that resolving this will help a lot of users in the future.
  • First prize would be to route to discourse on the server locally.
  • Then routing via the url would be the next prize.

Best regards
Matthew Lucas

これは私には完全に理解するには複雑すぎるため、解決策というよりは当てずっぽうと考えてください。問題がlocalhost経由で実行中のDockerコンテナに何かを接続できないということであれば、Docker IPを試してみる価値があるかもしれません。WebSocketテンプレートを使用して、ポートではなくソケットを使用することも可能です。いくつかの理由からこちらの方が好ましいと主張する人もいます。

MKJ’s Opinionated Discourse Deployment Configuration を読むと役立つかもしれません。

@pfaffman@Matthew_Lucas はすでにソケット化されたテンプレートを使用しています…\n\n外部プロキシには localhost を使用しており、これは expose 設定で機能するはずです。Docker IP を使用する必要があるとは思いません。また、その理由で Apache のみが Docker コンテナの前に置かれている場合にのみ失敗するとは思いません。\n\nヘッダー設定が必要になる場合があります — Add an offline page to display when Discourse is rebuilding or starting up - #2 by codinghorror を参照してください。\n\nApache の設定については、できる限り忘れるようにしていますが、nginx 設定のこの部分は、Apache で再現する方法を見つけたいと思うでしょう。\n\n\n proxy_set_header Host $http_host;\n proxy_http_version 1.1;\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n proxy_set_header X-Forwarded-Proto https;\n proxy_set_header X-Real-IP $remote_addr;\n\n\n特に、Host ヘッダーなしでは正しく機能しないと思います。\n\nHTTPS を使用していないので、tcpdump を使用してパケットトレースをキャプチャし、何が問題になっているかを正確に確認できるはずです。\n\nしかし、なぜ Apache をそこに入れるのですか?単に追加のハードルです。もし私がこのようなことを試すなら、Amazon ロードバランサーに EC2 インスタンスのポート 8000 に直接話させるように設定します。HTTPS は ELB で終端していると仮定します。おそらく、ロードバランサーは指定する必要なしに通常のヘッダーを追加する方法を知っているでしょうが、そうでない場合は明らかに設定してください。ファイアウォール経由でポートにアクセスできることを確認してください。\n\nこれ以上価値のある貢献はあまりできないと思いますが、うまくいけば、その一部が設定に役立つでしょう。幸運を祈ります。

「いいね!」 1

INSTALLATION OF DOCKER AND DISCOURSE ON AWS EC2 LINUX WITH EXISTING HTTPD REVERSE PROXY

Hi All

I have figured it out. I went back to the drawing board and started fresh with clean docker and discourse install. My process is detailed below:

Assumptions

• You already have an AWS EC2 Linux instance
• You can access your EC2 instance using SSH
• Apache 2 is already configured and running on your server

ACCESS YOUR EC2 INSTANCE USING SSH

  1. Open CMD / Terminal

  2. SSH into AWS EC2 instance

Example of screen once connected to EC2 instance

image

PERFORM SYSTEM UPDATES

  1. Perform system updates by running the following command in the console / terminal

$ sudo yum update

INSTALL DOCKER

  1. Search for the AWS Docker package by running the following command in the console / terminal

$ sudo yum search docker

  1. Get version information by running the following command in the console / terminal

$ sudo yum info docker

  1. Install docker by running the following command in the console / terminal

$ sudo yum install docker

  1. Add group membership for the default ec2-user so you can run all docker commands without using the sudo command by running the following command in the console / terminal
$ sudo usermod -a -G docker ec2-user
$ id ec2-user

# Reload a Linux user's group assignments to docker without logout

$ newgrp docker
  1. Install docker-compose by running the following command in the console / terminal
# 1. Get pip3 (python installer package)
$ sudo yum install python3-pip
 
# 2. Then run any one of the following
$ sudo pip3 install docker-compose # with root access
 
# OR
 
$ pip3 install --user docker-compose # without root access for security reasons

# Check permissions
$ sudo chmod -v +x /usr/local/bin/docker-compose
  1. Enable docker service to start automatically on boot by running the following command in the console / terminal

$ sudo systemctl enable docker.service

  1. Start the docker service by running the following command in the console / terminal

$ sudo systemctl start docker.service

  1. Check the docker service is running by running the following command in the console / terminal, the output will look similar to this

Basic docker commands that are good to know :

# Docker version
$ docker version

# Docker compose version
$ docker-compose version

# Start Docker service
$ sudo systemctl start docker.service

# Stop Docker service
$ sudo systemctl stop docker.service

# Restart Docker service
$ sudo systemctl restart docker.service

# Docker service status
$ sudo systemctl status docker.service

INSTALL DISCOURSE

  1. Create discourse folder

$ sudo mkdir /var/discourse

  1. Clone the discourse docker image to the newly created directory

$ sudo git clone https://github.com/discourse/discourse_docker.git /var/discourse

  1. Copy the standalone.yml file to the containers folder

$ cp /var/discourse/samples/standalone.yml /var/discourse/containers/discourse.yml

  1. Edit the new discourse.yml file as follows, (the important bits exposed ports for http and email details for smtp, modify in the file don’t copy and paste)

Editing options:

1. You can use nano and edit via command line / terminal
2. On AWS if you are using MATE with PLUMA, you can use PLUMA to edit the file
3. You can use WINSCP to SSH into the machine and edit the file via the Windows GUI

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  ## Uncomment the next line to enable the IPv6 listener
  #- "templates/web.ipv6.template.yml"
  - "templates/web.ratelimited.template.yml"
  ## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "8080:80"   # http (run on local port 8080, docker port 80)
  #- "443:443" # https (make sure this is commented out)

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  #db_shared_buffers: "256MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  #UNICORN_WORKERS: 3

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: 'faq.mobiloan.io'

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: ‘ enter email here ’

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: ENTER SMTP HERE
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: ENTER USERNAME HERE
  DISCOURSE_SMTP_PASSWORD: ENTER PASSWORD HERE
  DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true, we use Amazon SES)

 

  #DISCOURSE_SMTP_DOMAIN: discourse.example.com    # (required by some providers)
  #DISCOURSE_NOTIFICATION_EMAIL: noreply@discourse.example.com    # (address to send notifications from)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com
  
  ## The maxmind geolocation IP address key for IP address lookup
  ## see https://meta.discourse.org/t/-/137387/23 for details
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec: echo "End of custom commands"
  1. Once the file is edited and saved, rebuild the discourse app using the following command in the terminal

$ /var/discourse/launcher rebuild discourse

Note if your yaml file is called something else then you would replace discourse with your yaml file name

Note that the rebuild takes time.

  1. One your app has been rebuilt the app should be running on port 8080. To verify this run the following command in the terminal

$ sudo lsof -i -P -n | grep LISTEN

Example output of ports

Notice docker is listening on port 8080, httpd (apache) is listening on port 80.

In terms of traffic routing apache will receive the web request and where appropriate will be routed to the discourse container on port 8080.

To make sure the routing happens you must configure the virtual host in the apache conf file.

APACHE VIRTUAL HOST CONFIGURATION

  1. The apache conf file is located at /etc/httpd/conf.d/00-virtualhosts.conf. httpd / apache / apache2 are different apache versions. Your files might be in different locations if you are not using the httpd service but the virtual host config in this case will be the same.

Editing options:

1. You can use nano and edit via command line / terminal
2. On AWS if you are using MATE with PLUMA, you can use PLUMA to edit the file
3. You can use WINSCP to SSH into the machine and edit the file via the Windows GUI

Edit the /etc/httpd/conf.d/00-virtualhosts.conf file by adding the following:

# FAQ (DISCOURSE ROUTES)

<VirtualHost *:80>
  	ServerName  sub.domain.com
  	ProxyPreserveHost On
    ProxyPass "/" "http://localhost:8080/"
    ProxyPassReverse "/" "http://localhost:8080/"
</VirtualHost>
  1. You need to create the relevant CNAME DNS record having the sub domain pointing to your AWS server / load balancer.

Our setup is a little complicated as we have our domain provider who points routes all traffic to an AWS load balancer.

The load balancer handles our SSL/TLS through an AWS provided certificate.

The loan balancer also routes traffic to the AWS instance.

Apache is installed on the AWS instance and routes the traffic to our:

  • Node express applications
  • Docker / discourse application.

CONCLUSION

This is only one solution, but it is the one that worked with our existing setup.

A big thank you to:

@Kane York for his article on Discourse,
@Axel Fernandes’ article on medium his notes on downloading the necessary discourse files and
@Vivek Gite from nixCraft and his article on cyberciti on installing Docker on AWS Linux 2.