# Google indexing issue (robots.txt)
**URL:** https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148
**Category:** Support
**Created:** [April 15, 2020, 6:23pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148 "2020-04-15T18:23:45Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@VoskCoin](https://meta.discourse.org/u/VoskCoin)
#### Post date: [April 15, 2020, 6:23pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/1 "2020-04-15T18:23:46Z")
Google informed me of this issue with my forum, any input on fixing this?
Also Discourse has no sitemap, I found a sitemap plugin but my question on that is – does the sitemap constantly update or would I need to re-initiate the plugin frequently in order to “update” the sitemap?
referenced forum
> **[VoskCoinTalk](https://voskcointalk.com/)**
>
> Official VoskCoin forum for discussing Bitcoin, Ethereum, and all of the other cryptocurrency blockchains.
---
### Author:  [@Overgrow](https://meta.discourse.org/u/Overgrow)
#### Post date: [April 15, 2020, 6:46pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/2 "2020-04-15T18:46:52Z")
How many pages / urls are affected by this issue?
---
### Author:  [@VoskCoin](https://meta.discourse.org/u/VoskCoin)
#### Post date: [April 15, 2020, 6:55pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/3 "2020-04-15T18:55:39Z")
7
> **[Ruth](https://voskcointalk.com/u/ruth)**
>
> Official VoskCoin forum for discussing Bitcoin, Ethereum, and all of the other cryptocurrency blockchains.
Apr 10, 2020
[https://voskcointalk.com/u/greer](https://voskcointalk.com/u/greer)
Apr 10, 2020
> **[WhaleTailMining](https://voskcointalk.com/u/WhaleTailMining)**
>
> Just a dude who works in construction and mines crypto in the background.
Apr 10, 2020
> **[MeatyMouse](https://voskcointalk.com/u/MeatyMouse)**
>
> \#QANON #WWG1WGA
> New to mining and the crypto space, not trying to make a fortune just trying to get some skin in the game and learn more about it all. I currently am running 3 L3+ overclocked on LTC. If you wanna try HiveOS here is my referral link:...
Apr 10, 2020
> **[Cmiles7888](https://voskcointalk.com/u/Cmiles7888)**
>
> Official VoskCoin forum for discussing Bitcoin, Ethereum, and all of the other cryptocurrency blockchains.
Apr 10, 2020
> **[Search results for '{search\_term\_string}' - VoskCoinTalk](https://voskcointalk.com/search?q=%7Bsearch_term_string%7D)**
>
> Official VoskCoin forum for discussing Bitcoin, Ethereum, and all of the other cryptocurrency blockchains.
Apr 9, 2020
> **[VoskCoin](https://voskcointalk.com/u/VoskCoin)**
>
> Official VoskCoin forum for discussing Bitcoin, Ethereum, and all of the other cryptocurrency blockchains.
Apr 9, 2020
---
### Author:  [@Overgrow](https://meta.discourse.org/u/Overgrow)
#### Post date: [April 15, 2020, 7:29pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/4 "2020-04-15T19:29:10Z")
Yes, `/u/` pages are excluded in robots.txt by default. Do you think you are missing something by not indexing user profiles?
If the answer is yes, you can override your robots.txt file in
[https://voskcointalk.com/admin/customize/robots](https://voskcointalk.com/admin/customize/robots)
---
### Author:  [@VoskCoin](https://meta.discourse.org/u/VoskCoin)
#### Post date: [April 15, 2020, 7:32pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/5 "2020-04-15T19:32:44Z")
Is there a reason they are excluded by default? what if someone wanted to search for a specific user on a forum easily through google ex.
greer voskcointalk, if his profile isnt being indexed, then that would not be possible other than google just link to a thread of his, then having the user navigate directly to it?
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [April 15, 2020, 7:35pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/6 "2020-04-15T19:35:03Z")
There’s no content on the crawler view for profile pages except the bio, and spammers love putting garbage in their bio. It’s better all around to block crawling to stop the garbage from being associated with the site.
---
### Author:  [@VoskCoin](https://meta.discourse.org/u/VoskCoin)
#### Post date: [April 16, 2020, 1:24pm UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/7 "2020-04-16T13:24:51Z")
> [@VoskCoin](#):
>
> Also Discourse has no sitemap, I found a sitemap plugin but my question on that is – does the sitemap constantly update or would I need to re-initiate the plugin frequently in order to “update” the sitemap?
Also Discourse has no sitemap, I found a sitemap plugin but my question on that is – does the sitemap constantly update or would I need to re-initiate the plugin frequently in order to “update” the sitemap?
Are you familiar with this information? thanks for the above reply
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [April 18, 2020, 3:30am UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/8 "2020-04-18T03:30:11Z")
It updates automatically.
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [May 23, 2024, 4:03am UTC](https://meta.discourse.org/t/google-indexing-issue-robots-txt/148148/9 "2024-05-23T04:03:54Z")