# Disable these pages in robots.txt by default

**URL:** <https://meta.discourse.org/t/disable-these-pages-in-robots-txt-by-default/58871>\
**Category:** Feature\
**Created:** [3월 11, 2017, 7:17오전 UTC](https://meta.discourse.org/t/disable-these-pages-in-robots-txt-by-default/58871 "2017-03-11T07:17:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![bitsw](https://avatars.discourse-cdn.com/v4/letter/b/b5e925/32.png) [@bitsw](https://meta.discourse.org/u/bitsw)\
**Post date:** [3월 11, 2017, 7:17오전 UTC](https://meta.discourse.org/t/disable-these-pages-in-robots-txt-by-default/58871/1 "2017-03-11T07:17:30Z")

</div>

I believe indexing of these pages

- /guidelines
- /privacy
- /tos  
should closed by default, since they are duplicated at many websites. So there is chance forum with default settings receive small penalty at launch. And there is “Duplicate meta descriptions” at single site.

---

<div class="post-metadata">

**Author:** ![Mittineague](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mittineague/32/114259_2.png) [@Mittineague](https://meta.discourse.org/u/Mittineague)\
**Post date:** [3월 11, 2017, 9:22오전 UTC](https://meta.discourse.org/t/disable-these-pages-in-robots-txt-by-default/58871/2 "2017-03-11T09:22:55Z")

</div>

> [@bitsw](#):
>
> there is chance forum with default settings receive small penalty at launch.

Do you have a Google or Bing source reference to support that? That isn’t [what I’m seeing](https://www.google.com/search?sclient=tablet-gws&client=safari&channel=ipad_bm&site=&source=hp&q=The+conversations+we+have+here+set+the+tone+for+everyone.+Help+us+influence+the+future+of+this+community+by+choosing+to+engage+in+discussions+that+make+this+forum+an+interesting+place+to+be+%E2%80%94+and+avoiding+those+that+do+not.&oq=The+conversations+we+have+here+set+the+tone+for+everyone.+Help+us+influence+the+future+of+this+community+by+choosing+to+engage+in+discussions+that+make+this+forum+an+interesting+place+to+be+%E2%80%94+and+avoiding+those+that+do+not.&gs_l=tablet-gws.12...14880.14880.0.18273.1.1.0.0.0.0.0.0..0.0....0...1c.2.64.tablet-gws..1.0.0.lBOWPdl6Qms)