Remove local uploads after s3 migration

I’ve recently managed to move files to S3 (Contabo object storage) Configure an S3 compatible object storage provider for uploads - #474 by tumbano

but old local files haven’t been removed. From what I’ve read it should be done automatically (I’ve set Clean orphan uploads to 1 hours and Purge deleted uploads grace period to 1 day) but it isn’t. Also there’s this job “Jobs::CleanUpUploads” in sidekiq marked as FAILED.
Is there a way to safely delete the files or run the job from command line?

I too am experiencing a similar issue. All my uploads are in my bucket, but also on my server in the shared directory :thinking:

I just created a new site backup as a test, and it put it in both shared and the s3 bucket

Right, the backup still puts the local uploads folder into the archive, even if it says it doesn’t.

I remember there was an error at the end of the migration process, that some files were not transferred. I didn’t care if a few images out of 80000+ were missing, but who knows, maybe discourse did, so today I tried something that I’ve found on another topic and

cd /var/discourse
./launcher enter app
VERBOSE=1 rake posts:missing_uploads

The result was there were 2 posts with a missing image each and gave me the exact links so I went and edited the posts. After running the command again this time there were no missing files.
With a bit of hope I then tried running rake uploads:migrate_to_s3 again, with no luck. This is how it ends

Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
16163 posts were flagged for a rebake
rake aborted!
FileStore::ToS3MigrationError: 4 posts are not remapped to new S3 upload URL. S3 migration failed for db 'default'. (FileStore::ToS3MigrationError)
/var/www/discourse/lib/file_store/to_s3_migration.rb:132:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:81:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:383:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:59:in `migrate'
/var/www/discourse/lib/tasks/uploads.rake:126:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:106:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:36:in `each_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management.rb:21:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:104:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:100:in `block in <main>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => uploads:migrate_to_s3