Install Discourse on Amazon WS with Cloudflare


(FOSS dev/hacker) #1

Discourse is a pretty awesome piece of software and it come with plenty of options for hosting. We wanted to setup our on AWS with dedicated servers for each services for maximum scalability and performance

If you want an hassle free easy deployment I recommend using the default Docker install. If you want a fancier setup then read on.

For this tutorial let’s assure you have an Amazon Web Services and Cloudflare account with a registered domain name you control.

Before we start

Make sure you create all of your instances in the same region since there are not fees for bandwidth in the same region. You will also get better performance.

VPC

The first thing we need to create is the VPC. It’s your virtual network infrastructure and it’s a bit of a pain to get it working properly.

In the VPC dashboard create a VPC and a subnets within the VPC. The database need a second subnet in a different zone but within the same region. For example if your first subnet is in region us-west-1a then the second must be in us-west-1b or 1c. So go ahead and create another one.

To make things easy to setup we will create a “allow all” security group. Click on the “Security Groups” link and then “Create Security Group”. Make sure to select your VPC in the option. Once it’s created click on “Inbound Rules” and choose “All Traffic”, “All” protocols, “ALL” port and “0.0.0.0/0” as the source.

Make sure you also have a working Internet Gateway and Routing Table. Still not sure how those two should be setup up properly. I added 0.0.0.0/0 to my Routing Table to get my servers to talk to each other correctly.

EC2

In the EC2 Dashboard go to Instance and Launch Instance. Select Ubuntu. As the “Instance Type” you are free to choose whatever you need but t2.micro meet the minimum requirements and is eligible for free hosting.

Later you might want to take a look at reserved instances to reduce your costs.

Once you selected your instance type click “Next”. Make sure you create your instance in your VPC and your first subnet. In the storage pane hit next. The 8GB default is way enough since we’re not going to host images and database on the server. Chose the “Allow All” security group and start your instance.

Create a new key pair and save it somewhere safe.

ElasticIP

Create and allocate an elastic ip to your EC2 instance. If you don’t, you’ll run into a lot of problems.

RDS

Before we setup the database we need to find what PostgreSQL version Discourse use. You can look into this Docker postgres template. Search for “chown postgres:postgres /shared/postgres_run/” and you should see the version on this line.

As of December 29th 2015 Discourse use 3.9.5

Once you have the pgSQL version create your DB instance. Use whatever configs you want but make sure you set the same database version or you will run into problems while running backups.

t2.micro instances will work fine.

Elasticache

Next go the Elasticache dashboard and create a Redis cache cluster. t2.micro will once again work fine.

S3

Follow the first part of this guide.

CloudFlare

At this point you should add your domain to your Cloudflare account and use default config values. Make sure the address you’ll use for your Discourse install point to your EC2 instance elastic ip.

SES

Amazon allow any account to send 10,000 free e-mails from EC2 hosted servers. That’s way enough for starting Discourse community.

In the SES dashboard add and verify your domain. Do not check “Generate DKIM Settings”. Go to Cloudflare and add the verification TXT record. Your domain should get verified in about 15 minutes.

When it does, return to SES dashboard, visit “SMTP Settings” and hit “Create My SMTP Credentials”. Add a user and note the “Access Key ID” and the secret key. You can’t get the secret key back so save it in a safe place.

Installing Discourse

Now that our servers are ready to roll we can install Discourse. Read and follow the install Discourse in the Cloud under 30 minutes guide.

Of your EC2 is a t2.micro you will need to setup swap.

You’ll setup your AWS services in the app.yml :

  • For email use your SES settings and SMTP credentials.
  • For database use your RDS endpoint, db user/pass/name.
  • For Redis use your Elasticache endpoint.

Make sure you add the Cloudflare config to your app.yml like specified here.

If your build fails or you need to rebuild, make sure your run “./launcher clean” or your server will fill up fast.

SSL and Cloudflare

Cloudflare allow us to use their SSL certificate for free but we still need to create an SSL certificate for Cloudflare and our server to communicate securely.

This is the guide we used to create our certificates. Make sure you name your certificates ssl.key and ssl.cert instead of apache.key and apache.cert.

Once you got your certificates follow this guide to activate SSL.

In your Cloudflare dashboard go to Crypto and select “Full” in the SSL section and make sure everything works.

Security Groups

What we will do here is setup our servers to only allow required connections so we can secure things up. In order for this to happen you’ll need to create a security group for EC2, Elasticache and RDS.

Elastichache and RDS should only accept incoming connections from your EC2 VPC private ip.

Set your EC2 to only accept SSL (443) traffic from Cloudflare. Consult their official list of IPV4 address. Yes it’s a bit long.

If you need to connect to SSH, add your ip to the security group then remove when done.

Backups

Discourse can save your backups to Amazon S3 and Glacier. Saving them to S3 is dead simple but to Glacier can be quite confusing. Make sure you read those two guiides:

Finishing touches

Make sure you finish your S3 setup and activate automatic security updates.

Then you can follow all the optional guides to enable additional features and launch your community!

If you activate backup uploads to your S3 bucket you can even configure the bucket to store backups on Amazon Glacier which is even cheaper. The official documentation is a bit of a pain though.


Document for installing on AWS ECS
Bitnami install on AWS EC2 - email won't work
Thanks for Discourse cauze it's just pure awesome
Steam Login / Authentication Plugin
How Exactly should I add cloudflare CDN to discourse?
30 minute install guide did not work
(Sam Saffron) #2

Be sure to bypass cloudflare for long polling, otherwise message bus will slow down a fair bit.


(FOSS dev/hacker) #3

I read your post on what is the message bus but I’ll be honest, I totally didn’t understand.


(Sam Saffron) #4

See:


(FOSS dev/hacker) #5

Curently I setup my servers to only allow incoming traffic from Cloudflare. Setting a direct sub domain for long polling requests will allow attackers to find the real IP and attack the server directly which is not desirable.

I haven’t noticed any problem yet with the site but I’m sure you are right when saying the site is broken without a direct polling address. Thought, I would still like to validate this or find another to make it work without having to setup a direct IP.

I have contacted Cloudflare technical support before and I’m sure we could work out a solution with them and add the information to this post.

Would you mind if we look into that?


(Sam Saffron) #6

It is possible that cloudflare allow for this, they definitely do in the enterprise tier.

To test if this is working … open 2 web browsers …

Post a reply to a topic on web browser 1
It should show up right away on web browser 2

Repeat a few times, waiting 5-10-15-25 seconds.

If that is working long polling is working.


Issues when resuming a Discourse session on MS Edge mobile
Delay after editing a post due to Cloudflare
(FOSS dev/hacker) #7

It is definitely not working.

And further more, the entire setup fails.
This guide intend to use a self signed SSL certificate and let Cloudflare do the talking with the server.
Using a direct address fail because of the unverified certificate.

If Cloudflare allow long polling to work as a paid feature then there is probably no way to get this working with a free account.

Will setup https://letsencrypt.org automated SSL certificate, fix the long polling then update the guide.


(FOSS dev/hacker) #8

@sam It turn out that my “long polling url” settings was wrong. When I use the default / the polling works fine. I’ve tried adding comments and they show up almost instantly in my other browser. I am pretty confident that Cloudflare does not require any additional setup for long polling to work properly.

If that slows down the message bus, but I don’t think it is, then we’ll have to make note in the guide and let the owner decide if he want to use default or poll the server directly.

If you want to try for youself feel free to use this topic : https://discuss.doom.team/t/test-topic-for-long-polling-analysis/53

And here’s a screenshot of the polling requests, you’ll see they work fine.


(Philip Colmer) #9

That guide just mentions installing the Docker container and doesn’t seem to make any reference to configuring Discourse for external servers.

Am I misunderstanding the documentation or missing something?


(Martial) #10

Honestly, i use Cloudflare behind my forum and also have 200 OK under each /poll, see :


(Kai Liu) #11

Yes I agree with above posts from @Frez, @Philip_Colmer, @Chopper, Cloudflare don’t need any extra setup. I also confirmed this on my site.

Cloudflare works as an reverse + caching proxy, not just a dumb static file distribution network. It’s just like we put an external Nginx in front of Discourse docker for multi-site setup. In fact Cloudflare use the Openresty package for that purpose, which is built on top of Nginx. So theoretically it is not necessary to setup the long pool URL as for other static file CDN, and our own tests proved that.


(Jeff Atwood) #12

Did you test this to see if it works?


(Kai Liu) #13

Yes I did. I tested this a few times right after opened my site in 2 browsers, and also left them there for a couple hours then tested again, all worked.

In case someone wonder my Cloudflare setup detail, I’m currently on free plan, SSL mode full(strict), caching level standard, no Railgun(not part of free plan), Rocket Loader off.


(Jeff Atwood) #14

What do you think @sam?


(Sam Saffron) #15

sounds like CF supports long polling now,