Configure automatic backups for Discourse

So you’d like to automatically back up all your Discourse data every day?

Go to the /admin settings, backup, and set backup_frequency to 1.

Now backup will be taken every day.

Store backups on local server or Amazon S3

:warning: WARNING

Storing backups and regular uploads in the same bucket and folder is no longer supported and will not work.

By default backups are stored on the local server disk. In order to store backups on Amazon S3, you’ll need to create a unique, private S3 bucket and put it’s name in the s3_backup_bucket site setting.

:warning: The S3 bucket should only be used for backups. If you need to use a bucket that contains other files please make sure that you provide a prefix when you configure the s3_backup_bucket setting (example: my-awesome-bucket/backups) and make sure that files with that prefix are private.

You can follow most of the steps in Setting up file and image uploads to S3 if you need help in setting up a new bucket.

Next, set your S3 credentials under the Files section: s3_access_key_id, s3_secret_access_key, and s3_region.

Finally, you need to select “S3” as backup_location.

From now on all backups will be uploaded to S3 and not be stored locally anymore. Local storage will only be used for temporary files during backups and restores.

Go to the Backups tab in the admin dashboard to browse the backups – you can download them any time to do a manual offsite backup.


Instructions for Discourse versions 2.2 beta 3 and earlier

Up until v2.2.0.beta3 backups are always saved on the local server disk by default.

If you want to also automatically upload your backups to Amazon S3, check enable_s3_backups. You’ll need to create a unique private S3 bucket name in s3_backup_bucket to store your backups.

Next, set your S3 credentials under the Files section: s3_access_key_id, s3_secret_access_key, and s3_region.

Backups are always stored on local server disk. Go to the Backups tab in the admin dashboard to browse your local server backups – you can download them any time to do a manual offsite backup.

If you’ve enabled S3 backups, check your S3 bucket to find the uploaded backup files.

Note that you can also enable an automatic move to Glacier bucket lifecycle rule to keep your S3 backup costs low.

47 Likes

You are able to Archive Backups from your S3 Bucket to Glacier.
It is cheaper, but an Restore attemps more Time.

This Site will Help you to reduce Backup costs.:

8 Likes

Setting this up can be rather confusing. Here’s a simple guide to help you out.

  • Log into your Discourse admin panel
  • Configure daily backups
  • Set maximum backups to 7
  • Log into your Amazon Web Services account
  • Go in the S3 Dashboard
  • Open the bucket containing the backups
  • Click on the properties tab
  • Activate versioning
  • Open the Lifecycle menu
  • Add a rule for the whole bucket
  • Set current version to expire after 15 days
  • Set previous version to
  • Archive to Glacier after 1 days
  • expire after 91 days
  • Save and logout

How it works

Versioning will keep backups automaticly deleted by Discourse. One day after beeing deleted it will be moved to the Glacier storage. After 91 days it will be delete from the Glacier storage.

Warning

Amazon charge you for item stored in Glacier for 90 days even if you delete them before. Make sure your Glacier Lyfecicle keep your file at least 90 days.

10 Likes

I may have missed this, but how to I make sure a bucket for backups is private the proper way? Setting up file and image uploads to S3 doesn’t seem to be described here.

Looks like there is no such option anymore.

UPD: ah, now it’s split into 2 steps of this wizard.

So I guess it should be like this:

2 Likes

Does S3 backup functionality play nice without IAM access key and secret if using AWS roles assigned to the instance and s3 use iam profile is enabled in the settings menu?

Edit: The answer is YES! Just ensure you have region set appropriately.

1 Like

I had this problem too, which was resolved by adding a couple of lines to the policy document:

"Resource": [
        "arn:aws:s3:::your-uploads-bucket",
        "arn:aws:s3:::your-uploads-bucket/*",
        "arn:aws:s3:::your-backups-bucket",
        "arn:aws:s3:::your-backups-bucket/*"
      ]

Should this be added to the OP (which isn’t a wiki)? Or perhaps in the OP of Setting up file and image uploads to S3?

2 Likes

Is there a good reason not to make daily backups aren’t the default?

Fixes needed for OP

Add something about moving S3 backups to Glacier?
Delete the S3 stuff and link to Using Object Storage for Uploads (S3 & Clones)

It’d be overkill and expensive. Same reason we don’t recommend changing the oil in your car every 500 miles?

You back up your servers once a week?

I think your 500 mile oil change analogy would be a good answer for “why not keep 30 backups?”, but it doesn’t make sense to me here.

Do you not think that if you’re going to have 5 backups most people would rather have backups every day so that if they do have to revert to a backup they’ve lost no more than a day? The only time I remember an older backup being useful is when images fell out of tombstone.

If they have so little disk space that they can’t hold 5 backups, it would seem better to learn that in 5 days when they still have some memory of what they did than wait 4-5 weeks.

Yes, that is correct, for my self hosted servers.

If you prefer different settings, feel free to edit them from the defaults. What’s stopping you from doing so?

1 Like

Well, I’ll be!

I’m cleaning up topics like this one. If the defaults were changed, it wouldn’t be needed!

I’m convinced that the defaults won’t be changed, so I’ll move ahead. :wink

2 Likes

Once a week is a good starting point and a solid default.

1 Like

I had the very same problem!

i suggest to add a note regarding this on the initial post too!

3 Likes

Would be nice if this could be used to upload to a different S3 compatible provider, such as running MinIO.

If you want to use minio, see Using Object Storage for Uploads (S3 & Clones). If you want different services for backups and assets then you’re out of luck (though there are ways to set up triggers to copy from one bucket 8 another)

No, just want for backups, so they get automatically pushed to a different machine on my network.

Then the guide I linked is what you’re looking for.

1 Like