Discourse Meta
How to disable indexing by crawlers
支持
codinghorror
(Jeff Atwood)
2016 年2 月 9 日 11:14
3
There is a boolean site setting for this, search for “index” or “robots”.
7 个赞
MegaIndex bot did about 4,000 pageviews on one day
在话题中显示帖子
相关话题
话题
回复
浏览量
活动
Robots.txt to completely block all indexing of private site
Support
5
845
2020 年4 月 5 日
Why there are lots of Disallow rule in robots.txt?
Support
34
4527
2020 年12 月 22 日
Temporarily block Google crawler while setting up live site?
Support
4
464
2020 年2 月 5 日
Pages listed in the robots.txt are crawled and indexed by Google
Support
19
3251
2019 年7 月 30 日
Option to index topics
Support
8
538
2022 年12 月 31 日