How to move my backups from Amazon S3 to Wasabi?

Product Support

AI-generated summary

ariznaf seeks to migrate Discourse backups from Amazon S3 to Wasabi for cost savings, noting that backups — not uploads or images — are the target. While researching, they found guidance on configuring S3-compatible storage in app.yml but were unsure where to set environment variables. pfaffman clarifies that for backups alone, no CDN or upload settings are needed, and suggests alternatives like DigitalOcean Spaces if Wasabi proves problematic. ariznaf confirms success after adding Wasabi-specific env variables to app.yml, including DISCOURSE_USE_S3, DISCOURSE_S3_ENDPOINT, and DISCOURSE_S3_BACKUP_BUCKET, and notes region-specific endpoint adjustments are required. They explain the migration is necessary to avoid ongoing Amazon S3 charges for old backups, as Wasabi offers free storage up to 1TB. pfaffman adds that storing credentials in app.yml simplifies restoration on new servers and recommends tools like s3cmd for migrating existing data between S3 buckets — a task outside Discourse’s scope.

1

We currently are using Amazon S3 to make daily backups from within discourse and maintain them for several days.

We are storing there only backups not serving images or uploads from there (we prefer to store them in our server for now in order to keep it simple).

But Amazon S3 is not all that cheap. I would like to move from Amazon S3 to Wasabi that is cheaper for our needs.

I have read this post Using Object Storage for Uploads (S3 & Clones) - howto / admins - Discourse Meta and this other (wasabi specific) Discourse + Wasabi (S3-Compatible Storage) - support - Discourse Meta

But it is not clear to me how to do it.

In the first post it talks about changing some lines (adding them) in the app.yml.
But it seems it is needed to activate image and uploads stored in the S3 service, not for backups.

There they talk about some service specific envirrontment variables to change the CDN, and stores service.
But it is not clear to me where I have to store that environment variables.

Is there any config file where I should put that variables prior to using launcher?

Thank you for answers.

2

If you’re concerned only with backups then you don’t really need to move things to the new bucket. And if you do you can do it with whatever tools you’re comfortable with. Just change the settings for the new bucket.

You don’t care about cdn or any of that.

I had trouble with wasabi and gave up using it. Digital ocean spaces is cheap.

3

Thank you.
I will take that into account, will try wasabi for a while to see if I have troubles.

I had not realized that under the env section of app.yml I could create as many env variables as needed. :relaxed:

I need to move things to the new bucket because I maintain old backups there and amazon will continue charging for it until I delete it, and in wasabi I have no additional costs until 1 TB of data.

I have edited it and added these lines:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: eu-central-1
  DISCOURSE_S3_ENDPOINT: https://s3.eu-central-1.wasabisys.com
  DISCOURSE_S3_ACCESS_KEY_ID: {access key}
  DISCOURSE_S3_SECRET_ACCESS_KEY: {secret key }
  DISCOURSE_S3_BACKUP_BUCKET: {backup bucket name}
  DISCOURSE_BACKUP_LOCATION: s3

It works, now it saves backups to wasabi.
In case you use other regin you have to change it in the end point name too.

4

The good thing about having those set in the app.yml is that all you need to restore your site to a new server is that file; you can then install a fresh discourse and

./launcher enter app
discourse enable_restore
discourse restore

then copy/paste the backup that you want to restore.

That’s not a Discourse issue, then. There are a variety of tools you can use to do that, so search something like “move data s3 bucket”. I use s3cmd, for reasons I don’t recall.