Discourse Meta
Google changed how they process robots.txt in Discourse?
Ondersteuning
codinghorror
(Jeff Atwood)
22 december 2020 om 06:02
21
We should probably backport this.
← vorige pagina
Gerelateerde topics
Topic
Antwoorden
Weergaven
Activiteit
Pages listed in the robots.txt are crawled and indexed by Google
Product Support
18
3643
30 juli 2019
Google notification to remove "noindex" statements from robots.txt
Product Support
7
2640
30 juli 2019
Google complaining – Indexed, though blocked by robots.txt
Product Support
22
2856
13 december 2019
Issues Google Search Console is throwing at me for wrong discourse structure (or some for wrong administration of my site)
Product Support
18
441
18 december 2024
Generic rules in "robots.txt" not picked up by Googlebot
Product Support
5
1130
3 maart 2022