Unable to migrate to S3, therefore unable to restore from backup

Installation
2

OK, so, figured it out after doing a deep dive in the DB. Looks like leftovers (71 uploads) from the original long-dead S3 environment (or rather, in this case, available but not primarily used S3 environment, because it was not going to be cost effective).

Ended up doing this:

From Origin Server:

  1. ./launcher enter app

  2. sudo -u postgres psql discourse

  3. SELECT url FROM uploads WHERE url NOT LIKE '%ExpectedS3DomainGoesHere%
    (replace the ExpectedS3DomainGoesHere with the actual URL you use for your S3 config)
    This will get URLs for you to work with, because we need to do some things.

  4. Where the URLs are from older buckets on different URLs, use the Amazon S3 client (or the client for your S3 storage backend), and:
    a. Sync the unexpected URLs’ buckets if available down to local storage.
    b. Sync the items from Local to the new Bucket.

  5. discourse remap OLD-URL-FROM-DB NEW-URL-FROM-DB
    While it was suggested here to use DbHelper.remap, the remap function from discourse worked fine.

  6. Make sure the data is migrated.
    rails uploads:migrate_to_s3

  7. Rebake time!
    rails posts:rebake

  8. Backup Site again on ‘origin’ machine/server. Download that latest update.

From New Destination Server:

  1. Set up Discourse as expected, copy app.yml and such from the origin server over to the new server in /var/discourse/containers/ to make sure the rebuild hits the proper plugins, etc. needed.
    Make sure that you comment out any DISCOURSE_BACKUP_LOCATION: s3 entries in the app.yml though, if you are working with local backup copies. Had some issues with S3 being weird with backup files being truncated so I took the local approach for a restore.

  2. Follow the steps in Restore a backup from the command line to get the backup into your server and restore it. Including the rebuild steps.

This was painful for me to resolve, but it was resolved once I deep-dove into the uploads table in the DB. BUT, this seems to have worked, so…

5 Likes