Restore Failed: Error/Problem in uploads shifting/migrating

I had an rather old discourse (working ok) installation (Ver 2.6.0.beta4 = 1ac010db3c )

And I’m trying to restore its backup on a new/fresh Discourse installation (on a fresh/separate server). I’ve tried installing the new installation with the same plugins as the old one, and/or without any plugins also.

I am getting the following error in restoration (I’ve only pasting the last few lines of restore log, which I think are relevant, because before these lines, all log lines seem to show ‘ok’ msgs):

[2021-04-13 13:15:04] Restoring uploads, this may take a while...
[2021-04-13 13:16:51] EXCEPTION: 855 of 1096 uploads are not migrated to S3. S3 migration failed for db 'default'.
[2021-04-13 13:16:51] /var/www/discourse/lib/file_store/to_s3_migration.rb:131:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:78:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:357:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:65:in `migrate'
/var/www/discourse/lib/file_store/s3_store.rb:240:in `copy_from'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:62:in `restore_uploads'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:44:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:62:in `run'
/var/www/discourse/script/spawn_backup_restore.rb:23:in `restore'
/var/www/discourse/script/spawn_backup_restore.rb:36:in `block in <main>'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `fork'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `<main>'
[2021-04-13 13:16:51] Trying to rollback...
[2021-04-13 13:16:51] Rolling back...
[2021-04-13 13:16:52] Cleaning stuff up...
[2021-04-13 13:16:52] Dropping functions from the discourse_functions schema...
[2021-04-13 13:16:52] Removing tmp '/var/www/discourse/tmp/restores/default/2021-04-13-131303' directory...
[2021-04-13 13:16:52] Marking restore as finished...
[2021-04-13 13:16:52] Notifying 'bathinda' of the end of the restore...

In the mean time, I’ll try to update the old disco instance to the new/latest discourse version, then backup and then try to restore that backup on the new instance.

Edit1: Before starting restore operation, I’m just doing one thing, i.e. I configure the uploads folder and backup folder on my Aws S3 instance (copying the same id/secret etc from the old instance to the newly instance settings, and then starting the restore).

Looks like some of your assets are on local storage and you need to get them all on S3. You might also try to migrate them back to local storage before you make your backup.

2 Likes

Much thanks.

My local /var/discourse/shared/web_only/uploads folder is fully empty.

My site is here.

You might try to configure S3 only the backup location, then restore, then immediately configure S3 for uploads.

2 Likes

Restore failed even when I configured ‘Uploads’ onto local folder only in source discourse installation as well as target before doing restore operation.

[2021-04-14 05:23:43] Restoring uploads, this may take a while...
[2021-04-14 05:25:32] EXCEPTION: 855 of 1096 uploads are not migrated to S3. S3 migration failed for db 'default'.
[2021-04-14 05:25:32] /var/www/discourse/lib/file_store/to_s3_migration.rb:131:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:78:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:357:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:65:in `migrate'
/var/www/discourse/lib/file_store/s3_store.rb:240:in `copy_from'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:62:in `restore_uploads'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:44:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:62:in `run'
/var/www/discourse/script/spawn_backup_restore.rb:23:in `restore'
/var/www/discourse/script/spawn_backup_restore.rb:36:in `block in <main>'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `fork'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `<main>'
[2021-04-14 05:25:32] Trying to rollback...
[2021-04-14 05:25:32] Rolling back...
[2021-04-14 05:25:32] Cleaning stuff up...
[2021-04-14 05:25:32] Dropping functions from the discourse_functions schema...
[2021-04-14 05:25:32] Removing tmp '/var/www/discourse/tmp/restores/default/2021-04-14-052145' directory...
[2021-04-14 05:25:33] Marking restore as finished...
[2021-04-14 05:25:33] Notifying 'bathinda' of the end of the restore...

Try restoring a backup that doesn’t contain any uploads if the hostname of your forum will stay the same and all your uploads are stored on S3.

1 Like
  1. The backup I’m creating on old instance doesn’t contain any ‘uploads’. Domain name is same. Still its giving these errors.

  2. Someone suggested that I should include the ‘uploads’ in backup forcefully by these commands (and then restore the same on new instance of the same domain):

cd /var/discourse
./launcher enter app
rails c
SiteSetting.include_s3_uploads_in_backups = true
It returned this: => true

exit (from rails) and container.

Then I tried to create backup with ‘Uploads’. But still the backup size (380 MB) indicated that backups weren’t included.

Kindly guide.

Edit1: Note that source Discourse instance and target disco instance, both are latest/current version (tests-passed).

I’m pretty sure your backup contains uploads. This doesn’t happen when you have a backup that doesn’t contain any uploads.

2 Likes

But as told above, my ‘uploads’ folder on my virtual ubuntu server is empty.


How do I go around it?

How can I correct this anomaly (of Discourse thinking that some uploads are lying locally, but in actuality, they’re not)