Broken crawler view when you disable Javascript

Contribute Feature

AI-generated summary

The discussion revolves around the issue of a broken crawler view when JavaScript is disabled. stance455 encountered this error and discovered that Google’s crawler appears to have turned off JavaScript, likely due to the use of prerender. They shared screenshots comparing the broken view to the correct view when the user agent is changed to Google smartphone bot.

sam responded that this is a deliberate design choice for mobile optimization. When a modern user agent is detected and the user is browsing from a mobile device, the text payload is omitted to optimize transit for mobile phones. This behavior is not currently configurable but may be considered for future implementation if there is sufficient demand.

1

Ran into this error while looking at something else but basically if you disable JS in chrome

Screen Shot 2022-05-06 at 9.57.17 PM
Screen Shot 2022-05-06 at 9.57.17 PM510×284 14.2 KB

You get this

Screen Shot 2022-05-06 at 9.56.55 PM
Screen Shot 2022-05-06 at 9.56.55 PM988×1308 52.1 KB

Instead of this, which is what you get if you change the user agent to Google smartphone bot

Screen Shot 2022-05-06 at 9.56.15 PM
Screen Shot 2022-05-06 at 9.56.15 PM950×1346 115 KB

Google bot was crawling my instance and lately seems to have turned of JS in their crawler for me.

I think it has to do with me using prerender for about a month… so its adjusting to JS files instead of all html . Every 10 or so tests Ill see the screenshot that it renders correctly.

Screen Shot 2022-05-05 at 10.27.50 PM
Screen Shot 2022-05-05 at 10.27.50 PM818×1146 110 KB

But most of the time it gives the format you see in the first screenshot. Time will tell.

2

This is by design, we have a mobile optimisation.

IF the useragent is modern AND you are browsing from mobile THEN we omit the text payload to optimise transit for mobile phones.

Not configurable, possibly we could do this longer term if there is enough demand for it.

3 Likes
Disable or bypass feature detect for Googlebot (while serving JS app to crawlers)