I setup a new host so I can have a ‘staging’ environment. I re-created the same discourse installation with what should be the same version. I’m running version: 2.8.2.
First comment, as of version 2.8.2 my back up size has dropped from 282MB to around 90MB. Not sure why but I’m going to just with some intelligence added in that I’m taking advantage of.
I downloaded the latest archive from my forum, and uploaded to local storage on the new staging environment.
The restore fails due to:
[2022-02-27 19:41:18] ALTER TABLE
[2022-02-27 19:41:18] ALTER TABLE
[2022-02-27 19:41:18] Migrating the database...
[2022-02-27 19:43:00]
[2022-02-27 19:43:00] Reconnecting to the database...
[2022-02-27 19:43:00] Reloading site settings...
[2022-02-27 19:43:00] Disabling outgoing emails for non-staff users...
[2022-02-27 19:43:02] Disabling readonly mode...
[2022-02-27 19:43:02] Clearing category cache...
[2022-02-27 19:43:02] Reloading translations...
[2022-02-27 19:43:02] Remapping uploads...
[2022-02-27 19:43:02] Remapping 'https://forum.geekbeacon.org' to 'https://forum-staging.geekbeacon.org'
[2022-02-27 19:43:08] Restoring uploads, this may take a while...
[2022-02-27 19:43:36] EXCEPTION: 8 posts are not remapped to new S3 upload URL. S3 migration failed for db 'default'.
[2022-02-27 19:43:36] /var/www/discourse/lib/file_store/to_s3_migration.rb:132:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:87:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:373:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:66:in `migrate'
/var/www/discourse/lib/file_store/s3_store.rb:317:in `copy_from'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:62:in `restore_uploads'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:44:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:61:in `run'
/var/www/discourse/script/spawn_backup_restore.rb:23:in `restore'
/var/www/discourse/script/spawn_backup_restore.rb:36:in `block in <main>'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `fork'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `<main>'
[2022-02-27 19:43:36] Trying to rollback...
[2022-02-27 19:43:36] Rolling back...
[2022-02-27 19:43:36] Cleaning stuff up...
[2022-02-27 19:43:36] Dropping functions from the discourse_functions schema...
[2022-02-27 19:43:36] Removing tmp '/var/www/discourse/tmp/restores/default/2022-02-27-194051' directory...
[2022-02-27 19:43:36] Unpausing sidekiq...
[2022-02-27 19:43:36] Marking restore as finished...
[2022-02-27 19:43:36] Notifying 'csgeek' of the end of the restore...
This is your issue. Maybe use the same S3 bucket and use the same bucket? Maybe see that the hidden setting that’s called something like "download S3 in backup " is turned on. You’ll need to search or look on on the source for the setting name.
Or maybe you need to see that the rest of your uploads on the production site are on S3.
I have no S3 configured I’m guessing maybe the old server has some old data that didn’t get mapped to a valid S3 location.
I went back to the older server and I have 2 buckets configured, 1 for backups and 1 for media. I followed the guide and setup AWS S3 including the CDN.
When I run rake uploads:migrate_to_s3 which failed, I follwed that with a posts:rebake and that seems to have reduced the number of errors, but it still fails:
Please note that migrating to S3 is currently not reversible!
[CTRL+c] to cancel, [ENTER] to continue
Migrating uploads to S3 for 'default'...
Uploading files to S3...
- Listing local files
=> 208 files
- Listing S3 files
. => 978 files
- Syncing files to S3
................................................................................................................................................................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
15 posts were flagged for a rebake
rake aborted!
FileStore::ToS3MigrationError: 1 posts are not remapped to new S3 upload URL. S3 migration failed for db 'default'.
/var/www/discourse/lib/file_store/to_s3_migration.rb:132:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:87:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:373:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:66:in `migrate'
/var/www/discourse/lib/tasks/uploads.rake:123:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:102:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-4.0.0/lib/rails_multisite/connection_management.rb:80:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-4.0.0/lib/rails_multisite/connection_management.rb:90:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:100:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:96:in `block in <main>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => uploads:migrate_to_s3
(See full trace by running task with --trace)
Is there a way to run migrate_to_s3 in a verbose mode to identify the post that’s the culprit?