So, I have a Discourse backup from our initial server and we’re trying to move the system to a new system. Unfortunately, there are two problems: During restore, it says that 71 of 1073 uploads are not migrated to S3, and therefore hard-fails during the restore process. Attempting to fix this on the main instance by rebaking, etc. the posts, it indicates some uploads are still not migrated to S3, even when I try and use the uploads:migrate_to_s3 rake mechanism. Is there any way to get verbose information from migrate_to_s3 as to which uploads are not migrated, so I can manually fix it? It’s possible also that they are referring to a dead/failed S3 repository from where we were initially uploading to, except that things exploded at that point and we simply switched S3 mechanisms (to MinIO of course), and there was still old stuff sitting in the AWS S3 side of things. Which I don’t think I can easily migrate to our MinIO instance. Alternatively, is there a way to disable the restore mechanism’s uploads on s3 checker because I just forced the migration of uploads myself already?

OK, so, figured it out after doing a deep dive in the DB. Looks like leftovers (71 uploads) from the original long-dead S3 environment (or rather, in this case, available but not primarily used S3 environment, because it was not going to be cost effective). Ended up doing this: From Origin Server: ./launcher enter app sudo -u postgres psql discourse SELECT url FROM uploads WHERE url NOT LIKE '%ExpectedS3DomainGoesHere% (replace the ExpectedS3DomainGoesHere with the actual URL you use for your S3 config) This will get URLs for you to work with, because we need to do some things. Where the URLs are from older buckets on different URLs, use the Amazon S3 client (or the client for your S3 storage backend), and: a. Sync the unexpected URLs’ buckets if available down to local storage. b. Sync the items from Local to the new Bucket. discourse remap OLD-URL-FROM-DB NEW-URL-FROM-DB While it was suggested here to use DbHelper.remap, the remap function from discourse worked fine. Make sure the data is migrated. rails uploads:migrate_to_s3 Rebake time! rails posts:rebake Backup Site again on ‘origin’ machine/server. Download that latest update. From New Destination Server: Set up Discourse as expected, copy app.yml and such from the origin server over to the new server in /var/discourse/containers/ to make sure the rebuild hits the proper plugins, etc. needed. Make sure that you comment out any DISCOURSE_BACKUP_LOCATION: s3 entries in the app.yml though, if you are working with local backup copies. Had some issues with S3 being weird with backup files being truncated so I took the local approach for a restore. Follow the steps in Restore a backup from the command line to get the backup into your server and restore it. Including the rebuild steps. This was painful for me to resolve, but it was resolved once I deep-dove into the uploads table in the DB. BUT, this seems to have worked, so…

Unable to migrate to S3, therefore unable to restore from backup

Product Support Self-hosting

michaeld (Michael - Communiteq) March 9, 2022, 5:43am 3

What often works in these kind of situations is temporarily disabling s3 uploads on the original instance before taking a backup. After restoring you can enable s3 again.

Topic		Replies	Views
Restore failed at "EXCEPTION: x of y uploads are not migrated to S3. S3 migration failed for db 'default'." Self-hosting	4	1270	June 10, 2022
Restore from backup fails due to S3 migration Product Support	8	1536	September 10, 2019
Restore fails due to s3 migration failure Product Support	2	168	August 12, 2026
Restore process cancelled at migrating uploads to S3 step Product Support	48	4026	January 24, 2026
Unable to restore from S3 backup - Fails Product Support	5	264	March 20, 2025

Unable to migrate to S3, therefore unable to restore from backup

Related topics