Restore failed at "EXCEPTION: x of y uploads are not migrated to S3. S3 migration failed for db 'default'."

Installation
1

This has been sitting as a draft on meta.discourse.org as I worked through the issue on my Discourse server. I’ve been able to get around the problem described below, but that I couldn’t do what was effectively a disaster restore (a restore without the benefit of being able to get into the old, broken server) is concerning. (The root cause was a failed Postgres 10-to-12 upgrade, but I was able to postpone the update in the Discourse app.yml file long enough to disable the enable_s3_uploads site setting before creating the backup. I’ve now done that backup, uploaded it to the new server, restored it, and have the Discourse site running again. Original message:

I am reconstructing a Discourse install from backup stored on S3. The old installation was very old—first installed in June 2016. I started the restore from a fresh installation because of a failed Postgres 10-to-12 upgrade. Rather than fix that…since this box was running an no longer supported version of Ubuntu…I thought I would just take the latest backup and load it onto the new server. The restore fails, however. In the logs, the point at which I’m running into a problem is here:

[2020-11-23 22:27:32] Reloading translations...
[2020-11-23 22:27:32] Remapping uploads...
[2020-11-23 22:27:32] Restoring uploads, this may take a while...
[2020-11-23 22:28:38] EXCEPTION: 465 of 751 uploads are not migrated to S3. S3 migration failed for db 'default'.
[2020-11-23 22:28:38] /var/www/discourse/lib/file_store/to_s3_migration.rb:131:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:78:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:357:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:65:in `migrate'
/var/www/discourse/lib/file_store/s3_store.rb:238:in `copy_from'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:48:in `restore_uploads'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:30:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:64:in `run'
/var/www/discourse/script/spawn_backup_restore.rb:23:in `restore'
/var/www/discourse/script/spawn_backup_restore.rb:36:in `block in <main>'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `fork'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `<main>'
[2020-11-23 22:28:38] Trying to rollback...

This sounds similar to Restore process cancelled at migrating uploads to S3 step - #5 by Jim.Morrison, but it didn’t offer any guidance on how to fix the issue.

The before-you-post message in the right pane of this compose window also suggested Failed to restore DB, but that didn’t have anything helpful either.

1 Like
3

I’m facing the same issue here but I haven’t been able to resolve it:

Could you explain in more details what exactly you did here? Was this settings disabled in the Admin → Settings GUI page or was it on in app.yml? So after the backup, you were able to restore it and get everything back? Did you turn on this setting again after the backup? Thanks

4

This was editing the app.yml file, and yes I was able to get everything back. Unfortunately, I didn’t create any notes from my work 2 years ago, and the details have been lost with time. I didn’t re-enable “include uploads in backup”, but I think I will now — hopefully whatever issue I had before is fixed.

1 Like
5

Did you find any broken links to attachments in the posts after the restore?

6

Not that I can recall. Good luck!