Discourse Meta
Needing to edit robots.txt file - where is it?
Support
Osama
July 11, 2019, 8:44pm
40
I’ve created a PR for this:
https://github.com/discourse/discourse/pull/7884
Screenshots:
image
2057×1477 234 KB
image
2015×781 51.5 KB
17 Likes
Google notification to remove "noindex" statements from robots.txt
show post in topic
Related topics
Topic
Replies
Views
Activity
Why there are lots of Disallow rule in robots.txt?
Support
34
4423
December 22, 2020
Pages listed in the robots.txt are crawled and indexed by Google
Support
19
3201
July 30, 2019
Google changed how they process robots.txt in Discourse?
Support
20
1577
December 22, 2020
Generic rules in "robots.txt" not picked up by Googlebot
Support
6
965
April 2, 2022
Google complaining – Indexed, though blocked by robots.txt
Support
24
2407
September 28, 2023