S3 Images broken after upgrade to V2.2.0beta7 + 119

After upgrading to V2.2.0beta7 + 119, our forum has gotten a report of a handful of broken images (so far just three). After some searching, I found that the images were in our S3 tombstone folder (orphaned files). I’m not exactly sure why they would be there as there is a reference to the images in post.

I tried a couple things to get it working:

  • rake uploads:recover_from_tombstone (doesn’t work - only for files hosted on same server)
  • rake posts:rebake (didn’t fix the problem)
  • tried to find out how many images might be broken
    • rake uploads:list_broken_posts (rake aborted! : looks like its not a rake task rake --tasks)
    • rake uploads:list_posts_with_broken_images (same as above)
    • DRY_RUN=1 rake uploads:recover / rake uploads:recover (no results)
    • rails c UploadRecovery.new(dry_run: true).recover(Post.where(“baked_at >= ?”,
      SiteSetting.purge_deleted_uploads_grace_period_days))
      (ActiveRecord::StatementInvalid: PG::UndefinedFunction: ERROR: operator does not exist: timestamp
      without time zone >= integer LINE 1: …HERE (“posts”.“deleted_at” IS NULL) AND (baked_at >= 30)
      AND…)
    • UploadRecovery.new(dry_run: true).recover() (ran but no results)
    • rake uploads:missing (SUCCESS!!! - says im missing alot of files… :frowning: but didn’t fix anything)

After having no luck with the previously mentioned commands, I decided to just copy the files from the tombstone bucket into the the correct folder. After copying the file, I rebaked the post with the broken image and BOOM! it appeared…but only until I repeated the process with the other 2 images that were missing. After rebaking the other 2 posts all images failed to load. I can currently see the image in the edit box when editing,
but the image will not show up on the original post. Any suggestions would be greatly appreciated!

Also, not sure if this helps but im not getting an actual broken image, instead im getting span with a child SVG that says “This image is broken”.

If it’s only 3 images, just re-upload them through the Discourse user interface.

Very hard to provide any guidance without info about the problem posts, what is raw for the sections with the images? Is it a proper upload url?

1 Like

The raw html (when using the edit interface on the post) for the image, was pointing to a file that didn’t exist in the S3 bucket. After some searching, I found the correct file in the tombstone folder (same file structure) and copied it to the correct location. So currently, the raw html for the image is pointing to the correct location of the image in the s3 bucket. Whats strange is that the preview post (window to the right when editing a post) shows the image, but the actual post does not.

I thought of fixing it this way, but what concerns me is that the “rake uploads:missing” command is returning “3182 of 6161 uploads are missing”. We have only received reports of the three so far, but this definitely concerns me. Is it possible that this command is returning inaccurate numbers?