CNN 一次性包装失败

tgxworld · 2024 年5 月 8 日 05:33

This is fixed by

github.com/discourse/discourse

FIX: Attempt to onebox even if response body exceeds `max_download_kb`

discourse:main ← discourse:fix_dont_throw_away_response_body_when_max_download_exceeded

opened 05:28AM - 08 May 24 UTC

tgxworld

+46 -5

In 95a82d608d6377faf68a0e2c5d9640b043557852, we lowered the default for `Onebox….options.max_download_kb` from 10mb to 2mb for security hardening purposes. However, this resulted in multiple bug reports where seemingly nomral URLs stopped being oneboxed. It turns out that lowering `Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised more often for more URLs in `Onebox::Helpers.fetch_response` which `Onebox::Helpers.fetch_html_doc` relies on. When `Onebox::Helpers::DownloadTooLarge` is raised in `Onebox::Helpers.fetch_response`, we throw away whatever response body which we have already downloaded at that point. This is not ideal because Nokogiri can parse incomplete HTML documents and there is a really high chance that the incomplete HTML document still contains the information which we need for oneboxing. Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not throw away the response body when the size of the response body exceeds `Onebox.options.max_download_size`. Instead, we just take whatever response which we have and get Nokogiri to parse it.

Locally, the “problematic” URLs reported in this topic no longer displays an error when we try to onebox.

话题		回复	浏览量
Oneboxing large images fails with no visual cue Bug onebox	47	1532	2023 年5 月 4 日
Onebox links result in large image downloads Feature	2	473	2022 年12 月 14 日
Can't override size of image oneboxes Feature	3	799	2017 年6 月 18 日
Onebox issue with a specific site Support onebox	14	1617	2019 年3 月 2 日
Onebox settings being ignored on edits Support	4	554	2022 年3 月 24 日

CNN 一次性包装失败

相关话题