Worth noting here that Manjaro (a major Linux distribution being developed for computers and mobile phones) suffered catastrophic failure of their Discourse-based forum. Sounds like a botched update lost all uploads and lost other data which eventually caused them to decide to rebuild their massive Discourse forum from scratch. More an fyi post than anything, but interesting to at least let the meta* community know. Sorry I do not have more details; perhaps worth continuing discussion on their forum. 
7 Likes
That’s horrible. But if they lost all of /var/discourse/shared then they are just out of luck.
Pushing backups to S3 is a good idea. You can push up to 10GB to backblaze for free and after that it’s still cheap).
4 Likes
Yeah you definitely want to set up S3 (or S3 compatible) automatic backup uploads, which default to weekly.
4 Likes
Is it worth having some kind of dashboard warning to that effect which requires manual dismissal?
Your installation doesn’t appear to be configured to automatically ship backups off-server.
There seem to be an increasing number of sites who aren’t aware of basic DR and production release processes. I get that not everyone will go to the trouble of testing on a staging copy, but we can’t do anything to help site owners who have nothing to restore from.
3 Likes
I guess, but you could also manually download backups every so often via your web browser, or have a preferred local backup script that runs, etc.
It is a bit better now that our S3 support is “S3 compatible” with various services (thank you @falco) so people aren’t locked in to Amazon for automatic backups, though!
3 Likes
I think all this talk of cloud leads to people forgetting just how vulnerable a flat install can be.
It doesn’t matter if you’re on a $5 VPS, $500 of infrastructure or even $5000. If you aren’t shipping backups off-server and ideally using s3 to store images, then everything lives in /var/discourse/shared and it all stands to be lost.
7 Likes
If people don’t know that you need to have data on more than one device, I don’t think we can teach them. It took me quite a while to wrap my head around the possibility that someone developing an OS would have data on only one device & assumed that there was some way to fix it, but no one asked me.
On my list is to add support for Backblaze to my automated installs. For a small site, you can keep a reasonable number of backups (10GB) for free.
4 Likes
Problem is, not all images (e.g. avatars and others) are stored via S3, yet.
1 Like