CNNのワンボクシングの失敗

tgxworld · 2024 年 5 月 8 日午前 5:33

This is fixed by

github.com/discourse/discourse

FIX: Attempt to onebox even if response body exceeds `max_download_kb`

discourse:main ← discourse:fix_dont_throw_away_response_body_when_max_download_exceeded

opened 05:28AM - 08 May 24 UTC

tgxworld

+46 -5

In 95a82d608d6377faf68a0e2c5d9640b043557852, we lowered the default for `Onebox….options.max_download_kb` from 10mb to 2mb for security hardening purposes. However, this resulted in multiple bug reports where seemingly nomral URLs stopped being oneboxed. It turns out that lowering `Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised more often for more URLs in `Onebox::Helpers.fetch_response` which `Onebox::Helpers.fetch_html_doc` relies on. When `Onebox::Helpers::DownloadTooLarge` is raised in `Onebox::Helpers.fetch_response`, we throw away whatever response body which we have already downloaded at that point. This is not ideal because Nokogiri can parse incomplete HTML documents and there is a really high chance that the incomplete HTML document still contains the information which we need for oneboxing. Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not throw away the response body when the size of the response body exceeds `Onebox.options.max_download_size`. Instead, we just take whatever response which we have and get Nokogiri to parse it.

Locally, the “problematic” URLs reported in this topic no longer displays an error when we try to onebox.

トピック		返信	表示
Oneboxing large images fails with no visual cue Bug onebox	47	1532	2023 年 5 月 4 日
Onebox links result in large image downloads Feature	2	473	2022 年 12 月 14 日
Can't override size of image oneboxes Feature	3	799	2017 年 6 月 18 日
Onebox issue with a specific site Support onebox	14	1617	2019 年 3 月 2 日
Onebox settings being ignored on edits Support	4	553	2022 年 3 月 24 日

CNNのワンボクシングの失敗

関連トピック