How can I recover missing avatars?

In the case I’m looking at, the missing avatar images were moved to the tombstone folder in s3 after a command line upgrade and a manual upgrade of postgres from 10 to 12. I’m still not sure why exactly.

Finding the uploads

@Jeremie_Leroy @Pad_Pors If you want to see if the same is true in your case, the way I checked for the missing avatars in my s3 tombstone folder is as follows

  1. Obtained the sha1 (the 16 character string in the upload url) of an avatar upload I knew was broken after the migration. I did this by taking the upload id from the broken avatar upload url (it’s the first part of the file name, e.g. for 6254_2.png the id is 6254, and then using that id to look up the upload’s sha1 in recent database dump. If you’re uncomfortable with the command line you can visualise your data in a dump using a postgres GUI like Postico 2.

  2. Did a search for the sha1 in the tomstone folder of the relevant s3 bucket using the AWS CLI.

    aws s3api list-objects --bucket <bucket_name> --query "Contents[?contains(Key, <sha1>)]" --prefix "tombstone"
    

    (you have to change <bucket_name> and <sha1>)

If it works, you will get a list of results that looks like this

{
  "Key": "tombstone/original/2X/d/d7b553ff276fca054c7090e859ef5339fd1f936e.jpg",
  "LastModified": "2020-05-16T11:45:03+00:00",
  "ETag": ## string of letters and numbers,
  "Size": 64580,
  "StorageClass": "STANDARD",
  "Owner": {
     "ID": ## string of letters and numbers
   }
}

Note, 5/16 was when I performed the upgrade. All incorrectly tombstoned avatars have timestamps from around then. I suspect the issue took a while to show up in production, and did so intermittently, due to caching.

Recovery

As far as I understand, UploadRecovery (and it’s associated rake task) is specific to post uploads, and doesn’t handle tombstoned avatars.

I’m currently looking adding (patching) a new method to UploadRecovery that uses recover_from_s3 to recover the avatar uploads.

If anyone knows of an easier way to recover incorrectly tombstoned avatars in s3, I’m all ears.

@tgxworld any ideas?