I am using Backblaze as S3 storage and have the clean up orphan uploads setting turned on. The problem is, instead of deleting the orphan upload, there created a duplicated file with 0 byte. see below for example. the (2) indicates the number of files with the same name. if expanded, you will see the original file still exists, also with an 0 bytes file. Has anyone had the similar issue? is it the problem of Backblaze or the setting? Thanks. [Screen Shot 2021-03-15 at 8.00.38 AM]

Discourse uses the S3 API, and since it works fine with AWS S3 it leaves this to being a Backblaze problem. Maybe contact their support? I will add a note about this in Using Object Storage for Uploads (S3 & Clones)

There’s an ellipsis which suggests the name of the 0-byte file is being truncated. What is the full name there? I’d wager the bucket has a file lifecycle configured and that file is a “hide marker”, as they call it. When the lifecycle expires both versions should disappear.

The truncated part is “hidden”. I think you are right. It looks like the duplicated files are all recently uploaded ones. I will wait some time and see if they are gone. Thank you.

Hi, Can you please update us if the issue has been resolved? More than a year passed since your last post, so hoping this is OK now… :slight_smile:

The issue is not resolved yet. I just now manually deleted all the orphan files from BackBlaze manually. I think I’ll move to S3 at this point because regularly cleaning up the storage is a task in itself.

Just an update with my experience… I have a free B2 account (i.e. the first 10gb) and I don’t have a payment method added. The backup files are rotated and the ‘hidden’ files are removed permanently after a few days. I never have more than 7 files at once (Discourse has max 5 daily backups rotation on). Since my backups are never more than ~500mb each I’ve never had to pay for anything or delete any ‘orphan’ files manually.

Has there been any change with this issue? Has anybody tried contacting Backblaze support about this?

But the issue seems to be with ‘Uploads’!! As the ‘Backups’ aren’t so many/too many files to manage. They are manageable, even manually.

It seems this issue continues. @Falco could you elaborate on what S3 operations Discourse performs to clean up orphans?

lib/s3_helper.rb 4b22e67c8 def remove(s3_filename, copy_to_tombstone = false) s3_filename = s3_filename.dup # copy the file in tombstone if copy_to_tombstone && @tombstone_prefix.present? self.copy(get_path_for_s3_upload(s3_filename), File.join(@tombstone_prefix, s3_filename)) end # delete the file s3_filename.prepend(multisite_upload_path) if Rails.configuration.multisite delete_object(get_path_for_s3_upload(s3_filename)) rescue Aws::S3::Errors::NoSuchKey, Aws::S3::Errors::NotFound end

בעיית Backblaze S3: העלאות כפולות לאחר מחיקה

תמיכה התקנה

Falco (Falco) 19 ביוני,‏ 2023,‏ 2:15pm 14

Nice find! Can you please edit it to the wiki?

נושא		תגובות	צפיות
Cleaning up uploads and purging uploads from S3 Site Management reference , file-management	2	1372	24 בדצמבר,‏ 2024
Duplicate Uploaded Files Bug	3	1523	20 ביוני,‏ 2016
Can't migrate uploads to S3 Product Support s3	8	453	12 ביוני,‏ 2025
Uploads not getting orphaned and purged Product Support	9	1889	19 באפריל,‏ 2024
Setting up backup and image uploads to Backblaze B2 Product Support	6	3240	24 ביולי,‏ 2020

בעיית Backblaze S3: העלאות כפולות לאחר מחיקה

נושאים קשורים