Oneboxing large images fails with no visual cue

Arkshine · April 13, 2023, 10:59pm

I believe it’s removed because the file is too large. By default, the max image size kb is 4096kb, and this specific file is > 5000 kb.

github.com

discourse/discourse/blob/main/lib/cooked_post_processor.rb#L395-L400


      
          if info&.too_large?
            if img.ancestors(".onebox, .onebox-body").blank?
              add_large_image_placeholder!(img)
            else
              img.remove
            end

From my understanding, if you upload an image, it’s automatically resized/compressed before sending. You can upload large photos because of the client-side browser optimization.

With download remote images to local setting, Discourse will try to download the image as it is, and if it’s too large, it will be removed from post content.

Richie · April 14, 2023, 5:24am

Hi @Arkshine

Has this default value been changed in the last couple of months?

We have over 1,000 images that are 30-40mb, or higher, which have been pulled remotely for many many years without an issue

Richie · April 14, 2023, 7:12am

Sorry @Arkshine I’ve looked in to this again this morning as I may have misunderstood your message earlier.

We changed our setting a number of years ago to allow images of up to 100MB in size.

I’ve been trying to trace exactly when this broke and I can narrow it down to roughly some point after 3rd March 2023.

Here’s a 35MB image that was working on 3rd March 2023:

https://dronescene.co.uk/images/uploads/fullsize/20230303120013_DJI_0013.jpg

And a 27.3MB example from the same date in March:

https://dronescene.co.uk/images/uploads/fullsize/20230303120713_DJI_0055.jpg

And a 11mb example from 27th February:

I might have a wade through the commits made in the last six weeks or so and see if any of the commit messages jump out at me

Arkshine · April 14, 2023, 1:04pm

The raw content for this specific post (and the one below) on your forum shows that it is an uploaded image, not an external link. Because of Client side image optimization enabled by default, you will be able to upload large images.

Your initial issue is about inserting an external image link.
That said, I’m unsure if Discourse behavior is expected not checking the image size when pasting an image link.

Can you share the value of upload_size in your app.yml?

Canapin · April 14, 2023, 1:06pm

Hey Richie!

I don’t know about Discourse being able to download very large remote images.

However, I remember that previously, when you hotlinked a large image, it displayed a message like “This image is too large to be displayed in the forum”, with a link to the external picture.

It seems it doesn’t happen anymore.

Pasting an external, large image link (https://upload.wikimedia.org/wikipedia/commons/0/0b/New_cyclic_large.gif - 11 MB GIF) , I can repro your issue:

It looks very much like a bug and possibly a regression.

Richie · April 14, 2023, 1:49pm

I think you may have misunderstood the issue, or maybe you are seeing the cause of the bug

I did not upload that image to Discourse, I just pasted a link, exactly like @Canapin 's screen recording below

It’s been pulling large images for years now @Canapin mate, many years, it’s only stopped working in the last six or eight weeks.

Thanks for testing

I can replicate it on our forum, on try.discourse.org and on meta.discourse.org

I’ve placed a bounty on the issue:

Thanks again to everyone who has looked at this issue so far

Lilly · April 14, 2023, 2:20pm

i was able to reproduce this issue as well. i also went through all my image related settings to make sure nothing there was somehow causing the effect. i notice when i first make the post with large image link, it flashes for a minute like it’s going to load from the top of the photo, then just disappears (this despite the image being very visible in the post preview window).

Richie · April 14, 2023, 2:21pm

Thanks @Lilly that’s exactly the issue we’re seeing too.

Lilly · April 14, 2023, 2:24pm

i just tested in private messages and the image shows up fine as expected and links to the original. so the issue is strictly in topic posting views it appears.

@Richie cool photos btw

JammyDodger · April 14, 2023, 2:31pm

FWIW I’ve just given that a test, but mine disappeared in the PM as well.

Lilly · April 14, 2023, 2:35pm

weird

here is my pm view:

getting same issue in desktop and mobile views too.

Lilly · April 14, 2023, 2:52pm

i’m wondering if the download_remote_images_to_local setting might be worth testing here

edit: nope, i just tried and that didn’t affect the issue.

Arkshine · April 14, 2023, 2:54pm

Yes, my bad! The link will eventually be replaced by the uploaded version in the post. It just takes time and I overlooked that. (I expected the original link to be kept in the raw content)

That’s the expected behavior. If an image is too large, it’s removed.

It’s less expected that we don’t have a warning when you first post the link in the composer.
Seeing the image disappears suddenly is confusing, for sure.

Lilly · April 14, 2023, 3:06pm

i thought the expected behavior for external linked images was for it to shrink it to fit and make it linked to the original? that is exactly what it is doing in my pm views.

Arkshine · April 14, 2023, 4:25pm

The previous behavior always displayed a placeholder with text saying the image was too large.
Now, because the onebox contains a link, it’s always removed.

Related to:

Canapin · April 14, 2023, 4:45pm

Good catch.

I reverted this change on my local install and here’s the result, which is what we expect:

Richie · April 14, 2023, 4:58pm

@Arkshine that may be the case, but the issue being discussed here is that the image doesn’t display - when it should display.

The images we are all testing with are not dead images, so they should display.

They have always displayed, regardless of the size, for years and years (and years).

And now, in the last eight weeks, they don’t display anymore.

Canapin · April 14, 2023, 6:04pm

Can you check in your staff action logs if the download remote images to local setting was changed?
Just append this to your site URL: /admin/logs/staff_action_logs?filters=%7B%22action_name%22%3A%22change_site_setting%22%2C%22action_id%22%3A3%2C%22subject%22%3A%22download_remote_images_to_local%22%7D

From what I understand, when hotlinking an external large image,

Old code displayed the “image larger than X MB” prepended with the image link
New code completely removes the image (which seems wrong)

However, I am only able to have a displayed large external image in my post if I disable download remote images to local:

Arkshine · April 14, 2023, 6:17pm

You’re right; we strayed away from the initial issue. the problem I described concerns why the [too-large] images disappear (which is a valid point and a separate issue )

It doesn’t seem to be your situation.

I can reproduce the failing onebox request failing and return 404 on your forum.
It’s unclear why some of your URLs (of the same domain) in the first post fail.
It might be a host issue at some point, and the status got cached (should be 1h).

I tried to post other large images (> 20Mb) on your forum – it works well (lightbox works too)

60 MB here:

So far, considering others’ images work, it sounds like dronescene.co.uk had temporary issues.

Lilly · April 14, 2023, 6:42pm

ok yes i too have now verified that the issue is the images from that drone site. i have successfully loaded a number of externally linked high res nasa images without issue into a test topic post and they show up fine with links to the originals.