Redundant Discourse Hosting on AWS


(Benjamin Boyle) #1

I initially intended to host my forum in a way that is protected against server hardware failure by having it hosted in AWS in the following way:

RDS for PostGres,
Elasticache for Redis,
S3 for user uploads
2 docker installations in different data centers
Load balancer

With S3 support deprecated, how can I achieve hardware failure-proof hosting now?


(Benjamin Boyle) #2

I know that I can use regular disk backups to protect against loss of data from hardware failure. My question is not about safety of the data, but being able to keep the forum alive when there is a hardware or datacenter issue.


(Tarak'ha) #3

Multiple VPSes connected via VPN threading them together to share data. You mentioned S3, I know AWS serves load balancing features.


(Dean Taylor) #4

If you are going down the entirely AWS route Amazon Elastic File System (EFS) is probably what you are after for storage, available across multiple Availability Zones (AZs) connected to multiple instances at the same time…

If you check out this map you can see that most / if not all AZs are in different data centres within the same region.


(Benjamin Boyle) #5

I will check out EFS thank you Dean :slight_smile:

Does anyone know how to configure discourse to store the uploads in a
custom location as would be required with the attached EFS?


(Dean Taylor) #6

It’s all about mounting file systems - standard linux stuff - nothing to configure in Discourse.


(Benjamin Boyle) #7

Yes, mounting the filesystem as an extra drive in Linux is fine. But discourse is hosted on the instance’s primary drive and naturally keeps user uploads there. I would need to tell it to keep the user uploads on the EFS drive, a non standard setting. Do you know how to do that?

Am i making the question clearer or proving i don’t understand!? :slight_smile:


(Dean Taylor) #8

Drives are not mounted as drives - you are thinking of Windows…

You mount to a folder say /my-lovely-new-storage or /my/usr/home/complicated/path etc - you probably need to do a bit more reading up on mounting file systems in *nix environments.

In addition to this symlinks come in handy read up on those too.


(Benjamin Boyle) #9

Great! That’s the second thing i learned from you today :slight_smile:

But… Still need to tell discourse to store the files there… Discourse itself cannot be installed on the EFS drive and run by several machines simultaneously, can it?


(Dean Taylor) #10

Your questions don’t need to be asked of Discourse… you should ask them of Discourses components, including nginx.

I think from what you have said you have a little mountain of a learning curve to join all the components together.

All I can say is: get your AWS calculator out and make sure it makes sense - the Official Discourse Hosting might be a better bet - once you factor in the cost of your time.


(Benjamin Boyle) #11

Are you saying to mount the EFS storage over the top of the folder discourse normally uses?


(Dean Taylor) #12

Perhaps not the whole folder, maybe just uploads - it depends entirely on how you wish configure your solution - you are designing it here.


(Dean Taylor) #13

@bboyle1234 if you haven’t read this thread already some part of it might be useful to you:

https://meta.discourse.org/t/set-up-discourse-on-amazon-aws/14853?u=deanmarktaylor&source_topic_id=28358

(Benjamin Boyle) #14

Thanks Dean, I have read it. That blog post has exactly the same problem. Now that S3 support is missing from Discourse, Natnahm also doesn’t know what to do!

I’ll be experimenting with hacking in EFS within a month or so from now.


(Manthan Mallikarjun) #15

Hey!

I think I found a solution, but truthfully I dont know how to set it up.

It would be great if you could check this out:

Thanks.


(Dean Taylor) #16

S3 support deprecated decision has been reversed.

The topic of an S3 replacement is perhaps now irrelevant.


(Benjamin Boyle) #17

Hoorayy!!!
Wonderful!!!
Thank you so much!!!

How did you do it … create new files/objects instead of changing existing files/objects?