How to restore missing S3 images from post

[Nacho_Caballero]

One of my posts has randomly lost its images. I ran PostCustomField.where(name: Post::MISSING_UPLOADS) in the rails console and got this:

[#<PostCustomField:0x00005636227c20a0
  id: 7335,
  post_id: 36,
  name: "missing uploads",
  value:
   "[\"upload://ljcw6gjRoPMD0IB5Vcc6K23VXIA.png\",
     \"upload://v6caNgjm6wDrHzN8MQdXYnlNJEs.png\",
     \"upload://bn2lC9z0Xp4Fit00p6KepGMZZ2h.png\",
     \"upload://omhuL50n5VfAxY8Reyf2rQFoEWb.jpeg\",
     \"upload://yJCRzx9SogObwuj1En0QciZdntF.jpeg\",
     \"upload://wOeXgaPEBBia8oS0xxbOr5NKNWo.gif\",
     \"upload://fuXMNACp8WoEHEr3ayciiVJjmNl.png\",
     \"upload://MTIX7g8Wu11qaJasGg5yGsFvJA.jpeg\",
     \"upload://oKle1GsOlPD85zeHiKRdUGJCCEM.jpeg\",
     \"upload://zY43AR6Xpga92HNzdEnxQRIVpYE.gif\",
     \"upload://ufigIPfGEPbDInHP4co0I0YDT51.png\",
     \"upload://h7A1BJjNwZZwuUTYWXCeASk7UAh.png\",
     \"upload://gmFGijsek8NmtfqBwAguIgkzDFP.png\",
     \"upload://qPFQBe9zvnTmOnqMhpofQF3oqCh.png\"]",
  created_at: Tue, 18 Feb 2020 17:44:40 UTC +00:00,
  updated_at: Tue, 18 Feb 2020 17:44:40 UTC +00:00>,

Since it’s only 14 images in a single post, I’m okay with fixing it manually but I have no idea how to find the original images in my S3 bucket. I’ve tried searching for the filenames here, but didn’t get any results:

[maiki]

@Nacho_Caballero

I’ve tried searching for the filenames here, but didn’t get any results

I don’t use S3 per se, but wouldn’t your images not being there cause them to be missing?

Legit: are there logs you can check for those file names? Maybe something weird deleted them programmatically?

[Nacho_Caballero]

@maiki

I don’t use S3 per se , but wouldn’t your images not being there cause them to be missing?

The problem is that the S3 web console only seems to search in the current folder. If I drill down to a specific folder and search for a file that is inside it, it finds it. If I go up to the parent folder, it doesn’t find it.

[michaeld (Michael - Communiteq)]

You need to convert the upload:// urls into actual file names first.

Converting short upload URLs to full URLs

I may be barking up the wrong tree here, so apologies if I am - but any pointers very much welcome! There are some threads in our Discourse site which are displayed in part on our main website. Because the cooked version of a post contains all the HTML for the lightbox, which we don’t want on the main site, I’m working with the raw version of a post. One thing that’s tripping me up is the file upload URLs. How can I convert an upload:// URL to a full URL? I’ve tried searching and come across S…

[Nacho_Caballero]

Thanks, Michael. That would be useful, but I can’t figure out how to get the full path. In the rails console, I can do this but it doesn’t give me the actual filename:

Base62.decode("ljcw6gjRoPMD0IB5Vcc6K23VXIA").to_s(16)
=> "955952cf3952d2fad7c24528c08cfb79301a6260"

[michaeld (Michael - Communiteq)]

It depends on a lot of things but since the file is reportedly missing it might be in the wrong place anyway. Now that you know the filename it’s not much work to browse through s3 and see if you find a matching file.

You could also try to find an Upload in the database where the sha1 matches this string, and the upload record has a path. But again: if everything was alright then we wouldn’t be discussing this issue.