Using S3: “Backup with Uploads” Settings
The discussion revolves around the “backup with uploads” setting in Discourse, particularly when using separate S3 buckets for uploads and backups. ghufransyed was unsure if enabling this setting would store uploads in the backup bucket. gerhard clarified that, by default, backups do not include S3 uploads, and the setting only includes local uploads if include_s3_uploads_in_backups is not enabled.
jammydodger confirmed this, stating that the “backup with uploads” setting only includes local uploads if include_s3_uploads_in_backups is disabled. alexmas shared concerns about the implications of enabling or disabling this setting when restoring or migrating backups. They suggested that enabling the setting might be a more resilient approach in case of problems.
The group discussed the potential benefits of adding explanatory notes to clarify the behavior of this setting. codinghorror requested a proposal for the explanatory note, which has not been provided yet.
I just followed the instructions to configure automatic backups and set up file and image uploads to s3
I am a little confused about the following setting:
backup with uploads include uploads in scheduled backups. Disabling this will only backup the database.
So if I have separate buckets for upload and backup, should I check this or not? I wasn’t sure if this would somehow try to store my uploads in the backup bucket…?
By default backups do not include uploads stored on S3 as it’s usually a waste of bandwidth a disk space. You need to enable the hidden include_s3_uploads_in_backups setting in order to change this behavior.
But if I understand your question correctly you don’t want your backups to contain uploads stored on S3, so you don’t have to change any settings.
Thanks! I wonder if it’s worth putting an explanatory note on that setting to that effect?
What explanatory note are you proposing? Can you write it out?
Hi
I have the same question, and I confirm it is not so obvious.
Now in 2023, the setting “backup with uploads” is selected by default.
I have two buckets on S3 for backups and uploads, and I wonder what happen if I restore or migrate a backup with that setting activated or not.
If not activated, will all my upload files correctly match with the uploads on S3 ?
If activated, will my uploads be overwrited or duplicate on S3 ?
Maybe a bit of answer in this file :
line 351 "# when the backup is restored to a site with s3 uploads, it will upload the items to the bucket" https://github.com/discourse/discourse/blob/main/script/bulk_import/discourse_merger.rb
Would you advice to always select “backup with uploads” as it a more resilient way if any problem occur ?
The hidden site setting include_s3_uploads_in_backups is still disabled by default.
I believe the setting backup with uploads will only include local uploads if you have it enabled but don’t have include_s3_uploads_in_backups enabled as well.