The discussion explores reliable, basic strategies for ensuring Discourse forum recovery, focusing on backup redundancy and threat modeling. Canapin recommends automated backups synced to Google Drive via rclone, though manual verification is needed due to potential token expiration. Ed_S uses rsync over SSH and emphasizes the importance of verifying backup integrity and securing domain ownership to allow for recovery on a new domain if necessary.
Multiple users advocate for object storage solutions like S3 or S3-compatible services (e.g., Cloudflare R2) to avoid local disk filling and ensure offsite redundancy NateDhaliwal, pfaffman, Lilly. pfaffman warns about the risks of losing provider accounts and advises against relying on a single provider, a point reinforced by Ed_S. Jagster and nlv suggest varying backup frequencies (hourly vs. daily) based on data criticality, with nlv also mentioning restic for external disk backups.
Ed_S provides a comprehensive threat model, highlighting risks such as physical disasters, provider policy changes, loss of access to accounts/domains, and uncooperative admins. This underscores the need for diverse backup locations and shared control. satonotdead proposes a three-copy strategy: host, cloud, and offline media (optical/USB). Editor0732 uses Discourse’s built-in backup for databases and restic for images, uploading to a cloud drive, while noting the benefit of multi-provider redundancy. nlv suggests git-annex for decentralized storage options.
Over the years I’ve helped friends hosting their own forum.
I’ve made them rent a Hetzner server, set up automated backups on Discourse, and I sync them to my Google Drive with rclone and a cron task. Each admin of each forum has access to their own backups on my Drive. It’s the simplest and safest I could imagine as a non-sysadmin.
A few times, I noticed the tokens were not refreshed on rClone (at least on one instance) and I had to reconfigure it. Because I have no alerts whatsoever if a sync doesn’t work, I just manually look at my Drive from time to time and ensure the last backups are recents.
If you were to voluntarily help someone or an association host a forum, what solution would you set up to ensure there are always backups somewhere that can be safely recovered if the forum blows up for some reason)?
I pull backups using rsync over ssh, with a restricted account, from a cronjob. Well, I don’t, but one of my mods does. I take and download a backup when I do an update. So, in the one case we could lose a few days of forum activity, and in the other case we could lose a month or more.
I should probably set up a crontab of my own. I could also increase the frequency of backups - presently set to 2 days.
In both cases we have to be moderately competent admins of our own systems. We should only lose the forum if both of us lose our downloaded copies and our own backups as well as the server itself.
Edit: good point, one must have some positive check that the process is working, and some idea, however shallow, that the downloaded backups are good.
I haven’t taken special care of the domain: ideally that too would have dual administration and joint ownership of some sort. If we lost control, the forum would have to reappear on a new domain - it could send digests and notifications to the members by email, so with a bit of trust they could rejoin.
Well, I’m backupping the database every hour It is same principle than with e-commerce — one must consider what is the biggest timeframe what can and/or wants to restore.
Agreed. If you’ve got the YML file(s) with the S3 configuration and backups on S3, then all you need to restore the forum on a new server is the YML file(s). This has the added benefit of not filling your disk with backups.
CAVEAT: You have to remember to pay the bill for the S3 bucket as well as the VM. I think maybe cloudflare’s is the cheapest?
I have had someone somehow lose control of their AWS account and lost all of their images. So just remembering to pay the bill is a cost not to be underestimated.
I currently perform a backup once a day and upload it.
The database is backed up using Discourse’s built-in backup feature, while the original images are backed up incrementally every day using restic.
Everything is uploaded to a cloud storage service, but the current drawback is that it only uploads to a single provider. Ideally, we should upload to multiple cloud storage services in the future.
use offsite backups and object storage. i use cloudflare R2 (S3 compatible) and have restored to different servers a number of times. easy to manage and restore. also, use a dual container build.
It is same principle than with e-commerce — one must consider what is the biggest timeframe what can and/or wants to restore.