Oneboxing large images fails with no visual cue

Richie · March 25, 2023, 9:25am

Running Discourse 3.1.0.beta3 (7c6a8f1c74)

We’re seeing some very odd behaviour with Onebox and PNG / JPG images.

Here’s an image that I’d like to onebox: https://dronescene.co.uk/images/uploads/fullsize/20211216142548_DJI_0205_HDR_Pano.png

It should embed nicely like so:

https://dronescene.co.uk/images/uploads/fullsize/20211216142548_DJI_0205_HDR_Pano.png(image larger than 10 MB)

But instead the image just displays as a link in the post.

Browser console is showing a 404:

Some other examples:

Small PNG:

https://dronescene.co.uk/images/uploads/fullsize/20211106095005_vlcsnap_2021_05_04_17h10m24s149.png

Large JPG:

https://dronescene.co.uk/images/uploads/fullsize/20230227161106_DJI_0087_1.jpg

https://dronescene.co.uk/images/uploads/fullsize/20230227161106_DJI_0087_1.jpg(image larger than 10 MB)

Large PNG:

https://dronescene.co.uk/images/uploads/fullsize/20220103215207_IMG_5782.png

https://dronescene.co.uk/images/uploads/fullsize/20220103215207_IMG_5782.png(image larger than 10 MB)

Richie · March 25, 2023, 9:27am

Hmm, it works here.

But it fails on try.discourse.org

https://try.discourse.org/t/lorem-lorem-ipsum-dolor-sit-amet-consectetur-adipiscing-elit/1425/2

See how the large one isn’t displaying?

Richie · March 25, 2023, 9:32am

Actually, I don’t think it does, the large image hasn’t appeared?

Richie · March 31, 2023, 7:36am

I first thought this bug might be due to the file size, our members often post 40MB+ images but I can replicate this issue on try.discourse.org using just a 5MB png image.

The image appears in the composer window when composing my post but does not appear in the post itself when submitted.

It also appears in the composer window if I then edit the post:

Here’s a onebox url image to try:

https://dronescene.co.uk/images/uploads/fullsize/20230329003455_Screenshot_2023_03_29_at_00.23.31.png

And to test it here on meta, it should appear here:

https://dronescene.co.uk/images/uploads/fullsize/20230329003455_Screenshot_2023_03_29_at_00.23.31.png(image larger than 10 MB)

Richie · March 31, 2023, 7:38am

This bug is present on try and on meta.

The image did appear briefly when I submitted that post but then it disappeared again a few seconds later, I should have done a screen recording. Is it being lost/broken in the baking process?

Richie · April 11, 2023, 3:55pm

I’ve been poking this bug a bit more.

If I look at a post where the image has failed to be one boxed / pulled down a local copy, when I Edit the post I see this 404 Not Found error in the browser console:

The URL it’s trying to fetch is:

https://greyarro.ws/onebox?url=https%3A%2F%2Fdronescene.co.uk%2Fimages%2Fuploads%2Ffullsize%2F20230411071449_8719D92D_726A_46BF_9437_A06DD0507D77.jpeg&refresh=true&category_id=17&topic_id=61425

Unsure why it’s throwing a 404, when the image does exist

https://dronescene.co.uk/images/uploads/fullsize/20230411071449_8719D92D_726A_46BF_9437_A06DD0507D77.jpeg

https://dronescene.co.uk/images/uploads/fullsize/20230411071449_8719D92D_726A_46BF_9437_A06DD0507D77.jpeg(image larger than 10 MB)

This has worked fine for the last five years, it’s only broken in the last three or four weeks.

JammyDodger · April 11, 2023, 4:01pm

I’ve been having another run at it today, but I’m afraid I am no closer to working out what’s happening.

When I try and add the https://dronescene.co.uk/images/uploads/fullsize/20211216142548_DJI_0205_HDR_Pano.png it shows in the preview, and in the post for a second, and then is just blank.

I don’t see any 404s. I did think it was down to the size, as it was loading line by line in the preview like it was the 90s But you said you tried with a smaller one and got the same. Which one was that, and I’ll try that one too.

Richie · April 11, 2023, 4:03pm

Thanks @JammyDodger

Here’s just a small 5mb image that has the same problem: https://dronescene.co.uk/images/uploads/fullsize/20230329003455_Screenshot_2023_03_29_at_00.23.31.png

And it should appear here:

https://dronescene.co.uk/images/uploads/fullsize/20230329003455_Screenshot_2023_03_29_at_00.23.31.png(image larger than 10 MB)

Richie · April 11, 2023, 4:03pm

Which as you say, it does for a second, then it vanishes again.

Yet it’s visible in the composer if I edit the post

Arkshine · April 13, 2023, 10:59pm

I believe it’s removed because the file is too large. By default, the max image size kb is 4096kb, and this specific file is > 5000 kb.

github.com/discourse/discourse

lib/cooked_post_processor.rb

main


      
          # frozen_string_literal: true
          
          # Post processing that we can do after a post has already been cooked.
          # For example, inserting the onebox content, or image sizes/thumbnails.
          
          class CookedPostProcessor
            include CookedProcessorMixin
          
            attr_reader :cooking_options, :doc
          
            def initialize(post, opts = {})
              @dirty = false
              @opts = opts
              @post = post
              @model = post
              @previous_cooked = (@post.cooked || "").dup
              # NOTE: we re-cook the post here in order to prevent timing issues with edits
              # cf. https://meta.discourse.org/t/edit-of-rebaked-post-doesnt-show-in-html-only-in-raw/33815/6
              @cooking_options = post.cooking_options || opts[:cooking_options] || {}
              @cooking_options[:topic_id] = post.topic_id

This file has been truncated. show original

From my understanding, if you upload an image, it’s automatically resized/compressed before sending. You can upload large photos because of the client-side browser optimization.

With download remote images to local setting, Discourse will try to download the image as it is, and if it’s too large, it will be removed from post content.

Richie · April 14, 2023, 5:24am

Hi @Arkshine

Has this default value been changed in the last couple of months?

We have over 1,000 images that are 30-40mb, or higher, which have been pulled remotely for many many years without an issue

Richie · April 14, 2023, 7:12am

Sorry @Arkshine I’ve looked in to this again this morning as I may have misunderstood your message earlier.

We changed our setting a number of years ago to allow images of up to 100MB in size.

I’ve been trying to trace exactly when this broke and I can narrow it down to roughly some point after 3rd March 2023.

Here’s a 35MB image that was working on 3rd March 2023:

https://dronescene.co.uk/images/uploads/fullsize/20230303120013_DJI_0013.jpg

And a 27.3MB example from the same date in March:

https://dronescene.co.uk/images/uploads/fullsize/20230303120713_DJI_0055.jpg

And a 11mb example from 27th February:

I might have a wade through the commits made in the last six weeks or so and see if any of the commit messages jump out at me

Arkshine · April 14, 2023, 1:04pm

The raw content for this specific post (and the one below) on your forum shows that it is an uploaded image, not an external link. Because of Client side image optimization enabled by default, you will be able to upload large images.

Your initial issue is about inserting an external image link.
That said, I’m unsure if Discourse behavior is expected not checking the image size when pasting an image link.

Can you share the value of upload_size in your app.yml?

Canapin · April 14, 2023, 1:06pm

Hey Richie!

I don’t know about Discourse being able to download very large remote images.

However, I remember that previously, when you hotlinked a large image, it displayed a message like “This image is too large to be displayed in the forum”, with a link to the external picture.

It seems it doesn’t happen anymore.

Pasting an external, large image link (https://upload.wikimedia.org/wikipedia/commons/0/0b/New_cyclic_large.gif - 11 MB GIF) , I can repro your issue:

It looks very much like a bug and possibly a regression.

Richie · April 14, 2023, 1:49pm

I think you may have misunderstood the issue, or maybe you are seeing the cause of the bug

I did not upload that image to Discourse, I just pasted a link, exactly like @Canapin 's screen recording below

It’s been pulling large images for years now @Canapin mate, many years, it’s only stopped working in the last six or eight weeks.

Thanks for testing

I can replicate it on our forum, on try.discourse.org and on meta.discourse.org

I’ve placed a bounty on the issue:

Thanks again to everyone who has looked at this issue so far

Lilly · April 14, 2023, 2:20pm

i was able to reproduce this issue as well. i also went through all my image related settings to make sure nothing there was somehow causing the effect. i notice when i first make the post with large image link, it flashes for a minute like it’s going to load from the top of the photo, then just disappears (this despite the image being very visible in the post preview window).

Richie · April 14, 2023, 2:21pm

Thanks @Lilly that’s exactly the issue we’re seeing too.

Lilly · April 14, 2023, 2:24pm

i just tested in private messages and the image shows up fine as expected and links to the original. so the issue is strictly in topic posting views it appears.

@Richie cool photos btw

JammyDodger · April 14, 2023, 2:31pm

FWIW I’ve just given that a test, but mine disappeared in the PM as well.

Lilly · April 14, 2023, 2:35pm

weird

here is my pm view:

getting same issue in desktop and mobile views too.

Topic		Replies	Views
Incorrect or failing oneboxes for links to other discourse instances Bug	16	2412	August 17, 2017
Link image resizing issue (setting) Support	6	766	June 3, 2021
Add setting to disable hotlinking of large image files Feature	58	7245	March 8, 2018
Fetching images from newspaper websites is failing Support	2	623	August 16, 2022
Missing images at Meta.discourse.org Site feedback	57	3363	June 20, 2026

Oneboxing large images fails with no visual cue

AI-generated summary

Related topics