Провал одностраничного вывода CNN

tgxworld · 08.Май.2024 05:33:07

Это исправлено в

github.com/discourse/discourse

FIX: Attempt to onebox even if response body exceeds `max_download_kb` (#26929)

main ← fix_dont_throw_away_response_body_when_max_download_exceeded

merged 11:00PM - 08 May 24 UTC

tgxworld

+45 -4

In 95a82d608d6377faf68a0e2c5d9640b043557852, we lowered the default for `Onebox….options.max_download_kb` from 10mb to 2mb for security hardening purposes. However, this resulted in multiple bug reports where seemingly nomral URLs stopped being oneboxed. It turns out that lowering `Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised more often for more URLs in `Onebox::Helpers.fetch_response` which `Onebox::Helpers.fetch_html_doc` relies on. When `Onebox::Helpers::DownloadTooLarge` is raised in `Onebox::Helpers.fetch_response`, we throw away whatever response body which we have already downloaded at that point. This is not ideal because Nokogiri can parse incomplete HTML documents and there is a really high chance that the incomplete HTML document contains the information which we need for generating a reasonable onebox. Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not throw away the response body when the size of the response body exceeds `Onebox.options.max_download_size`. Instead, we just take whatever we have and get Nokogiri to parse it.

Локально «проблемные» URL-адреса, о которых сообщалось в этой теме, больше не вызывают ошибку при попытке onebox.

Тема		Ответов	Просм.
Issue with Onebox not working for videos larger than 5MB Support	0	51	20.12.2025
Oneboxing large images fails with no visual cue Bug onebox	47	1693	04.05.2023
Onebox links result in large image downloads Feature	2	494	14.12.2022
Can't override size of image oneboxes Feature	3	829	18.06.2017
Onebox issue with a specific site Support onebox	14	1673	02.03.2019

Провал одностраничного вывода CNN

Связанные темы