Canonical structure for /u/* causing many urls to be indexed

[justin (Justin DiRose)]

Is this for your custom indexing setup discussed here?

Need "window.prerenderReady" implemented on my instance [PAID GIG]

I’m using prerender.io to serve the “application/JS” version of the site to crawlers ( my instance serves crawlers the JS version via a hidden setting ) It’s working well but seems discourse might fall under this category … but some web pages use some custom loading flows or constant polling that may trick Prerender’s logic; thus, it fails to make a decision on the ready-ness of the page. The first time prerender accesses any discourse URL, it will timeout ( 20 seconds set by prerender ) T…

If so, it might be helpful to note this so those looking at the job know what they’re getting into.

By default, /u/ URLs are not indexed, which is set both in robots.txt and passed in the initial header request on the page.