Backup location

A few days ago we added a new feature that allows you to select where backups are stored. The new site setting is called backup_location and currently supports “Local” and “Amazon S3”.

The setting defaults to “Local” unless you previously set enable_s3_backups to automatically upload backups to S3.

What has changed?

Up until now backups were always stored on the server’s local disk – uploading backups to S3 was just an additional, optional step at the end of the backup process. With the new setting you can choose to store backups either locally or on S3. Local storage will be used only for temporary files during backups and restores if you select S3.

Also, the Backups tab now works the same for every backup location.

There’s no need to login into Amazon S3 anymore in order to upload, download or delete a backup. You can even restore a backup directly from S3.

Plugins

Unfortunately the new system breaks existing plugins that automatically sync backups to Dropbox, Google Drive, Nextcloud or Box.

The good news is that it should be a lot easier to add new backup locations from within a plugin than it was before. Do you want to give it a try?

In the meantime you can take a look at Use rclone to sync backups to Dropbox or Google Drive.

Is there a Howto topic?

Yes, there is. Configure automatic backups for Discourse

15 Likes

Will the old way still work—local + S3? I like having a local backup along with the S3 for redundancy.

3 Likes

Local backups and sync to S3 is not possible anymore. You need to choose one backup location.

6 Likes

This sounds great! Does it really mean Amazon S3 or will it also work with S3 lookalikes like Digital Ocean Spaces and that thing that starts with M? (I think that I’d made backups push to Spaces a while ago.)

Is it related to this bug? Admin dashboard won’t load after upgrade to v2.2.0.beta3 +36

I haven’t tested it with anything other than Amazon S3. It might work on lookalikes too if they support the required APIs. DO might work – /reference/api/spaces-api/ looks promising.

Yes, it is.

4 Likes

I haven’t been able to get DigitalOcean Spaces to serve as my Discourse backup location. Might require extra setups than Amazon S3. Would appreciate it if someone can give me some pointers. Thanks!

I have this working on DO Spaces. Below are my settings for reference, be sure to take the bucket name out of the endpoint url.

6 Likes

Clarification on how this works when settings are changed:

  1. When once switches from S3 to Local or vice versa, do the existing backups get transferred or are they removed or are they left alone?
  2. When one see’s the list of backups in the Backup page is there a way to tell the location or are they all from the selected location in the setting?
  3. What is the meaning of this setting given that backups are no longer “synced” and either on S3 or Local? Disable the removal of backups from S3 when removed locally.
2 Likes

Existing backups won’t be touched. You need to delete them manually if you want to free up space.

Only from the selected backup location.

The description of the s3_disable_cleanup setting needs an update. It disables the removal of backups from S3 and makes the system ignore the value of maximum_backups when backup_location is set to S3.
I think we could actually remove the s3_disable_cleanup setting and allow disabling the cleanup by setting maximum_backups to 0. That would make it work for all backup locations the same way.

6 Likes

That makes sense to remove the option and use zero to disable removal of back ups. A follow up clarification on this, when one changes the settings, say from 120 days to 30 days, would that trigger an automatic cleanup of all the back ups?

maximum_backups has nothing to do with days…
And no, the cleanup only happens after successful backups. Changing settings doesn’t trigger anything.

4 Likes

Sorry, let me rephrase my question. When I change the maximum back ups from 120 to 30, the next time when it backs up, will it delete anything older than the last 30 back ups?

1 Like

Yes, it will delete the older backups except for the last 30.

4 Likes