Depends on the onebox engine. Some onebox engines like Amazon scrapes for certain information like the price of the item from within the <body>. For opengraph engines, we should in theory only need certain tags like <meta> in <head>.
The most efficient way will be to parse the response for the necessary information as we stream the response but changing all our engines to do this is likely going to take a considerable amount of effort and is a much more complex solution.
Actually, Nokogiri which we use to parse the HTML response is capable of parsing incomplete HTML text so there is no need to throw away the entire response when it is too large. I think we can just continue to limit the response body to 2mb and if the response size exceeds that, we just try to parse the first 2mb.