The discussion revolves around setting up a stateless Discourse server using AWS RDS and S3. phil22 wants to ensure that their server can be terminated and rebuilt without losing data. They plan to move the database to Amazon RDS and store app.yml, uploads, and backups in S3.
phil22 asks if they can safely terminate the instance, clone Discourse, and rebuild the app using ./launcher rebuild app. pfaffman responds that this is mostly true, but they need to ensure that they don’t have two instances running at once and that they use the correct database version. The data in Redis is considered ephemeral and does not get backed up.
Poster_Nutbag shares their experience with a similar setup and suggests using an EBS volume to store the /shared folder contents. This allows for persisted data even when instances come and go. They also caution that Discourse releases often include major database upgrades, so it’s essential to track these upgrades and upgrade the external RDS accordingly.
phil22 thanks the respondents for their advice and decides to use RDS with a pinned commit of the discourse_docker repo. This will make upgrades more complex but ensures that the data is safe and can be quickly restored in case of a problem.
Hi, I’ve just got a new discourse installation setup - thanks for all the hard work making it so straightforwards!
Ideally I’d like to make my discourse server disposable, so that if e.g. someone accidentally kills it in the EC2 console, I don’t lose any data.
The built backup/restore functionality is great, and I’ve already done a few test runs with that. I’m storing the app.yml, uploads and backups in S3.
The next step would be to move the database to Amazon RDS, for which there’s a great guide here.
My question is - if I do this, should I in theory be safe to terminate the instance, then on a fresh server simply clone discourse, get my app.yml and run ./launcher rebuild app? OR is there other state beyond postgres/uploads/app.yml? Perhaps in Redis?
I think that’s mostly true. You’ll want to make sure that you don’t have two running at once. When you build the new container, you’ll migrate the database, potentially making it unusable for the other container (you can avoid this with skip_post_deployment_migrations). The stuff in Redis is considered ethereal and doesn’t get backed up. I’m not quite sure how or whether some of the stuff there gets restored, but you probably don’t care.
@phil22 - I’ve worked on a setup similar to what you are proposing and its more subtle that you think:
The Discourse team puts out multiple releases a month and so you need to stay with your original version when you clone to your new ec2. Normally its ok to blindly upgrade the app, but some releases include major database upgades (PG 12 → PG 13) and if you don’t track this and neglect to upgrade your external RDS, you might get stuck.
We found success in having the ec2 use an EBS volume which is also configured to be mounted inside your app container. Using this, we store all of the /shared folder contents. That way when your ec2 instances come and go, you have all of this shared folder content persisted on EBS. Further, sometimes you change your mind about storing files in s3. Having an alternative place to store uploaded files (like the /shared folder) is good in that case.
Following from your advice I think I’m going to go for RDS but to pin to a specific commit of the discourse_docker repo. Whilst it sounds like that will make upgrades more complex, I’m hoping it means that if we do have a problem with the server we’ll have everything safe on RDS and fairly swiftly be able to the same working state without a manual restore.
(We could achieve the same thing with EBS I think, but with volume encryption and unix magic to mount/dismount the volume between instances I’m a bit scared of the process of automating that)