main ← fix-crawler-hooks-binary-response
merged 04:02PM - 28 Apr 26 UTC
**Previously**, the `CrawlerHooks` middleware parsed any 200 response carrying t…he `X-Discourse-Crawler-View` header as HTML, which crashed Nokogiri with `Document tree depth limit exceeded` when crawlers requested binary file downloads (e.g. `/uploads/short-url/file.zip?tl=fr`) — `set_crawler_header` runs as a `before_action` and gets set regardless of the eventual response type.
**In this update**, the middleware additionally requires a `text/html` `Content-Type` before attempting to parse and rewrite links, leaving binary and non-HTML responses untouched.