Google changed how they process robots.txt in Discourse?

codinghorror (Jeff Atwood) 2020 年12 月 22 日 06:02 21

We should probably backport this.

2 个赞

话题		回复	浏览量
Pages listed in the robots.txt are crawled and indexed by Google Support	19	3247	2019 年7 月 30 日
Google notification to remove "noindex" statements from robots.txt Support	8	2431	2019 年7 月 30 日
Google complaining – Indexed, though blocked by robots.txt Support	24	2488	2023 年9 月 28 日
Issues Google Search Console is throwing at me for wrong discourse structure (or some for wrong administration of my site) Support	18	146	2024 年12 月 18 日
Google search indexing not working? Support seo	0	46	2025 年3 月 23 日