Migrate_from_s3 problems

I recently opened a new PR to allow migrating fewer posts at a time (such as testing one at a time) and added tests for the task.

https://github.com/discourse/discourse/pull/9933

It feels like it makes sense to start with posts, since you want to rebake posts to change the URL in the cooked post after each migration; migrating all the uploads and then starting the rebakes would leave the site very broken for potentially a long time for a large site (I need to migrate about 100GB from S3 to local, so I care). But what I wrote might help start writing a migrate_uploads_from_s3 task that would be run after migrate_from_s3 in order to clean up uploads that weren’t part of posts.

@zogstrip What’s the current status of “We’re working on…” — is this still in flux or is this migration now worth paying attention to?

5 Likes