I’m migrating a load of content, about 15k worth of news stories to Discourse via the API and these topics have links to images in. The images have just been uploaded to blob storage and are fully accessible publicly. Most topics have the images shown correctly and get hotlink downloaded and uploaded into Discourse as you’d expect. Some though either do not get past the point of showing as links in the topic, or end up getting shown as broken images for some reason. i.e. Just staying as links: [image] Getting broken: [image] In the broken case, the topic shows as this when editing: [image] Config is: Virgin Discourse install just for testing the migration S3 storage of images I’ve had no problems just creating posts as a user and uploading issues, it seems to only occur when I migrating posts via the API, i.e. in bulk and in quick succession (am using the skip_validations=true parameter). So far I’ve only migrated a maximum of 1,000 topics before clearing down and starting again. Any ideas for what might be causing this? Or any tips on how to diagnose? All the Hotlink Sidekiq jobs seem to schedule and run as you’d expect. rake posts:rebake doesn’t help

How about the rebuilt html option with the little admin wrench? [image]

Your file has a double extension - .jpeg.jpg - is that true of all of the files which have been rejected? Have any double-extension files been processed successfully?

Thanks Robert. This is working for the first few I’ve tried. It’s going back to square one, i.e. rendering the hotlinked image, and then downloading/uploading it. I’ll have to see if this can be automated, though it doesn’t get to the root cause of why they’re failing to start with.

No, the double extension images are the minority and do not make up all of the failures. The Rebuild HTML option fixes these cases as well though.

Okay, I fixed all migrated posts via the API using /posts/{num}/rebake. Oddly the rake posts:rebake command in the container didn’t work. After automating the rebake API call, it turned out there’s about an 80% image upload failure rate. That’s pretty crazy.

In the end this turned out not to be reliable, it would work for a few posts but not for all. I’ll have to have another go at trying to upload the images again. I had issues with this before with requests just hanging. I’m not sure if I tried the skip_validations parameter on those requests though.

[image] Amethi: Oddly the rake posts:rebake command in the container didn’t work. [image] Amethi: In the end this turned out not to be reliable, it would work for a few posts but not for all. There are a couple of suggestions in this post that may be helpful for fixing broken images: Rebake with rails command or rake task doesn't work, but rebuilding HTML does. Why? - #12 by RGJ.

Some linked images not displaying/show as broken

Product Support

anon48433008 February 22, 2020, 12:43pm 6

That’s fantastic and congratulations, fortunately I learned that from other helpful people here when I’ve had similar issues and I’m happy I could pass it along to you

Topic		Replies	Views
Fixing Broken Images -- Summary of Steps? Product Support	4	1125	April 25, 2022
Rebake with rails command or rake task doesn't work, but rebuilding HTML does. Why? Self-hosting	9	2203	March 27, 2022
Old image uploads become broken images Product Support	63	9656	May 24, 2021
Rebaking posts does not fix broken locally downloaded linked images Product Support	5	1284	January 2, 2019
Images suddenly broken, even after re-uploading Product Support uploads	5	401	June 17, 2024

Some linked images not displaying/show as broken

Related topics