Google changed how they process robots.txt in Discourse?

codinghorror (Jeff Atwood) 22 Dicembre 2020, 6:02am 21

We should probably backport this.

2 Mi Piace

Argomento		Risposte	Visualizzazioni
Pages listed in the robots.txt are crawled and indexed by Google Support	19	3270	Luglio 30, 2019
Google notification to remove "noindex" statements from robots.txt Support	8	2434	Luglio 30, 2019
Google complaining – Indexed, though blocked by robots.txt Support	24	2492	Settembre 28, 2023
Issues Google Search Console is throwing at me for wrong discourse structure (or some for wrong administration of my site) Support	18	160	Dicembre 18, 2024
Google search indexing not working? Support seo	0	47	Marzo 23, 2025