Broken crawler view when you disable Javascript

Ran into this error while looking at something else but basically if you disable JS in chrome

Screen Shot 2022-05-06 at 9.57.17 PM

You get this

Instead of this, which is what you get if you change the user agent to Google smartphone bot

Google bot was crawling my instance and lately seems to have turned of JS in their crawler for me.

I think it has to do with me using prerender for about a month… so its adjusting to JS files instead of all html . Every 10 or so tests Ill see the screenshot that it renders correctly.

But most of the time it gives the format you see in the first screenshot. Time will tell.

This is by design, we have a mobile optimisation.

IF the useragent is modern AND you are browsing from mobile THEN we omit the text payload to optimise transit for mobile phones.

Not configurable, possibly we could do this longer term if there is enough demand for it.

3 Likes