# Migrate\_from\_s3 problems

**URL:** <https://meta.discourse.org/t/migrate-from-s3-problems/119064>\
**Category:** Bug\
**Created:** [30 במאי,‏ 2019,‏ 7:28am UTC](https://meta.discourse.org/t/migrate-from-s3-problems/119064 "2019-05-30T07:28:49Z")\
**Posts on this page:** 1\
**Showing post:** 20

<div class="post-metadata">

**Author:** ![mcdanlj](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mcdanlj/32/131829_2.png) [@mcdanlj](https://meta.discourse.org/u/mcdanlj)\
**Post date:** [17 בינואר,‏ 2021,‏ 3:44pm UTC](https://meta.discourse.org/t/migrate-from-s3-problems/119064/20 "2021-01-17T15:44:27Z")

</div>

As has been noted, I ran into some problems with the migration I did. This supports @cvx 's point that I wrote the migration completely wrong, and until someone does it right, the removal of the existing migration full of silent data corruption **really should be merged.** (It’s a really simple patch, review should be trivial. The sooner it is merged, the less likely someone is to destroy their site trying to walk backwards through a one-way door. I’m not sure how to plead more strongly for a merge; if I did I would. Edit: Thanks @cvx for the approval, merge, and followup to make sure that mass migration is marked as irreversible.)

Here are known problems I ran into:

- At least the discobot avatar didn’t show; all discobot participation showed the generic person avatar (light gray head-and-shoulders on white background).
- Some other people’s avatars were not migrated correctly. Many were fixed manually.
- When the site was being configured, the original administrator experimented with changes to “S3” configuration (in our case, digital ocean spaces, with and without CDN) leading to some orphaned optimized images that didn’t get fixed in the migration.

For anyone who tried using my branch and had some images go away like this, I ran a hack in the rails console and it fixed some things. **This is not the right way to do this, I just don’t know the model well enough to do it right.** My ruby-foo is very weak; I don’t touch ruby outside of my small contributions to Discourse.

But, such as it is, this terrible hack fixed broken images (from the initial configuration mistakes) and at least brought back the discobot avatar (and possibly others, but all of those of which I’d previously been aware had already been repaired).

What doesn’t show here is everything I did between steps, looking at each set as I went. This was part of a “explore what is there in the rails console” session and not actually a script I wrote and ran. This has no error handling, and I took a complete site backup before I started doing any of this work.

I don’t know how to say more strongly that this code is _not_ the idiomatic ruby on rails way of doing this repair. I just wanted to share how I repaired at least some of the damage I did to myself in my previous attempt to migrate my site off digital ocean spaces.

```ruby
require 'set'

uploadids = Set.new
optimizedimages = Set.new
OptimizedImage.where("url like '%UNIQUEPARTOFS3URL%'").each do |oi|
  uploadids.add(oi.upload_id)
  optimizedimages.add(oi)
end

postids = Set.new
uploadids.each do |u|
  PostUpload.where(upload_id: u).each do |pu|
    postids.add(pu.post_id)
  end
end

optimizedimages.each do |oi|
  oi.delete
end

postids.each do |pid|
  Post.where(id: pid).each do |p|
    p.rebake!
  end
end

```

---

_[View the full topic](https://meta.discourse.org/t/migrate-from-s3-problems/119064)._
