1 of 16551 uploads are not migrated to S3. S3 migration failed for db 'default'

[pawelorzech (Paweł Orzech)]

Ok, so this issue has been now fixed! Here’s how I did it (with immense help of Artur from our forum)

In to_s3_migration.rb, I’ve changed this:

count = Upload.by_users.where("url NOT LIKE '#{base_url}%'").count
      if count > 0
        error_message =
          "#{count} of #{Upload.count} uploads are not migrated to S3. #{failure_message}"
        raise_or_log(error_message, should_raise)
        success = false
      end

To this:

count = Upload.by_users.where("url NOT LIKE '#{base_url}%'").count
      if count > 0
        failed_uploads = Upload.by_users.where("url NOT LIKE '#{base_url}%'")
        failed_ids = failed_uploads.map(&:id)
        failed_id_list = failed_ids.join(", ")
        error_message =
          "#{count} of #{Upload.count} uploads are not migrated to S3. \n List of failed IDs: \n #{failed_id_list}; #{failure_message}"
        raise_or_log(error_message, should_raise)
        success = false
      end

And then in rails console just destroyed problematic upload. Now I’ve got issue with

FileStore::ToS3MigrationError: 3 posts are not remapped to new S3 upload URL. S3 migration failed for db 'default'.

But I think I’ve encountered a bigger issue - assets after being compilation are not being uploaded to S3 bucket:

There obviously should be asset folder but there are none. This is something that’s preventing me from putting CDN in front of it all, but I guess that’s another issue, for a different topic?

[pawelorzech (Paweł Orzech)]

For files changed:

count = Post.where("cooked LIKE '%#{cdn_path}%'").count
      if count > 0
        failed_posts = Post.where("cooked LIKE '%#{cdn_path}%'")
        id_list = failed_posts.map(&:id).join(", ")
        error_message = "#{count} posts are not remapped to new S3 upload URL. \n List of failing posts: \n #{id_list} \n #{failure_message}"
        raise_or_log(error_message, should_raise)
        success = false
      end

to

count = Post.where("cooked LIKE '%#{cdn_path}%'").count
      if count > 0
        failed_posts = Post.where("cooked LIKE '%#{cdn_path}%'")
        failed_ids = failed_posts.map(&:id).join(", ")
        failed_id_list = failed_ids.join(", ")
        error_message = "#{count} posts are not remapped to new S3 upload URL. \n List of failing posts: \n #{failed_id_list} \n #{failure_message}"
        raise_or_log(error_message, should_raise)
        success = false
      end

Got myself three ID’s and then went into https://example.com/p/ID and deleted it from UI. Then it went smoothly!