I am using Backblaze as S3 storage and have the clean up orphan uploads setting turned on. The problem is, instead of deleting the orphan upload, there created a duplicated file with 0 byte. see below for example. the (2) indicates the number of files with the same name. if expanded, you will see the original file still exists, also with an 0 bytes file. Has anyone had the similar issue? is it the problem of Backblaze or the setting? Thanks. [Screen Shot 2021-03-15 at 8.00.38 AM]

Discourse uses the S3 API, and since it works fine with AWS S3 it leaves this to being a Backblaze problem. Maybe contact their support? I will add a note about this in Using Object Storage for Uploads (S3 & Clones)

There’s an ellipsis which suggests the name of the 0-byte file is being truncated. What is the full name there? I’d wager the bucket has a file lifecycle configured and that file is a “hide marker”, as they call it. When the lifecycle expires both versions should disappear.

The truncated part is “hidden”. I think you are right. It looks like the duplicated files are all recently uploaded ones. I will wait some time and see if they are gone. Thank you.

Hi, Can you please update us if the issue has been resolved? More than a year passed since your last post, so hoping this is OK now… :slight_smile:

The issue is not resolved yet. I just now manually deleted all the orphan files from BackBlaze manually. I think I’ll move to S3 at this point because regularly cleaning up the storage is a task in itself.

Just an update with my experience… I have a free B2 account (i.e. the first 10gb) and I don’t have a payment method added. The backup files are rotated and the ‘hidden’ files are removed permanently after a few days. I never have more than 7 files at once (Discourse has max 5 daily backups rotation on). Since my backups are never more than ~500mb each I’ve never had to pay for anything or delete any ‘orphan’ files manually.

Has there been any change with this issue? Has anybody tried contacting Backblaze support about this?

But the issue seems to be with ‘Uploads’!! As the ‘Backups’ aren’t so many/too many files to manage. They are manageable, even manually.

It seems this issue continues. @Falco could you elaborate on what S3 operations Discourse performs to clean up orphans?

lib/s3_helper.rb 4b22e67c8 def remove(s3_filename, copy_to_tombstone = false) s3_filename = s3_filename.dup # copy the file in tombstone if copy_to_tombstone && @tombstone_prefix.present? self.copy(get_path_for_s3_upload(s3_filename), File.join(@tombstone_prefix, s3_filename)) end # delete the file s3_filename.prepend(multisite_upload_path) if Rails.configuration.multisite delete_object(get_path_for_s3_upload(s3_filename)) rescue Aws::S3::Errors::NoSuchKey, Aws::S3::Errors::NotFound end

Backblaze S3 issue: duplicated uploads after delete

Product Support Self-hosting

Falco (Falco) June 19, 2023, 2:15pm 14

Nice find! Can you please edit it to the wiki?

Topic		Replies	Views
Cleaning up uploads and purging uploads from S3 Site Management reference , file-management	2	1366	December 24, 2024
Duplicate Uploaded Files Bug	3	1523	June 20, 2016
Can't migrate uploads to S3 Product Support s3	8	450	June 12, 2025
Uploads not getting orphaned and purged Product Support	9	1884	April 19, 2024
Setting up backup and image uploads to Backblaze B2 Product Support	6	3234	July 24, 2020

Backblaze S3 issue: duplicated uploads after delete

Related topics