# Excluding user profiles in robots.txt (or allow edit of file)

**URL:** <https://meta.discourse.org/t/excluding-user-profiles-in-robots-txt-or-allow-edit-of-file/15886>\
**Category:** Feature\
**Created:** [2014年五月22日 22:30 UTC](https://meta.discourse.org/t/excluding-user-profiles-in-robots-txt-or-allow-edit-of-file/15886 "2014-05-22T22:30:37Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![neil](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/neil/32/102150_2.png) [@neil](https://meta.discourse.org/u/neil)\
**Post date:** [2014年五月22日 22:55 UTC](https://meta.discourse.org/t/excluding-user-profiles-in-robots-txt-or-allow-edit-of-file/15886/2 "2014-05-22T22:55:36Z")

</div>

Good catch. Web crawlers will see almost nothing there. My page as crawlers see it:

![](https://global.discourse-cdn.com/meta/original/2X/4/44621224f12bf81d3301c51c041d0522072515df.png)

Do we even need a setting for this? We could just add a rule to the robots.txt to disallow indexing of user pages. I don’t think any Discourse install would want them to be indexed.

---

_[View the full topic](https://meta.discourse.org/t/excluding-user-profiles-in-robots-txt-or-allow-edit-of-file/15886)._