Google changed how they process robots.txt in Discourse?

codinghorror (Jeff Atwood) 22 Diciembre, 2020 06:02 21

We should probably backport this.

2 Me gusta

Tema		Respuestas	Vistas
Pages listed in the robots.txt are crawled and indexed by Google Support	19	3277	30 Julio 2019
Google notification to remove "noindex" statements from robots.txt Support	8	2438	30 Julio 2019
Google complaining – Indexed, though blocked by robots.txt Support	24	2496	28 Septiembre 2023
Issues Google Search Console is throwing at me for wrong discourse structure (or some for wrong administration of my site) Support	18	179	18 Diciembre 2024
Google search indexing not working? Support seo	0	53	23 Marzo 2025