# Forum not being updated on search engines
**URL:** https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990
**Category:** Support
**Created:** [27 במרץ, 2017, 9:38pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990 "2017-03-27T21:38:54Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@john3](https://meta.discourse.org/u/john3)
#### Post date: [27 במרץ, 2017, 9:38pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/1 "2017-03-27T21:38:54Z")
Hi,
My forum is hosted on a subdomain and recently has not been updated on google. I made some changes and do check but its been days and its not getting updated. I changed the description even thats not updated its been 3 days now.
Any idea about this issue?
robots.txt file is the default one.
```
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
User-agent: *
Disallow: /auth/cas
Disallow: /auth/facebook/callback
Disallow: /auth/twitter/callback
Disallow: /auth/google/callback
Disallow: /auth/yahoo/callback
Disallow: /auth/github/callback
Disallow: /auth/cas/callback
Disallow: /assets/browser-update*.js
Disallow: /users/
Disallow: /badges/
Disallow: /search
Disallow: /search/
Disallow: /tags
Disallow: /tags/
```
Before posting here, I have tried other ways such as adding my domain to google search console. advertised the domain too. but still the content is like 1 month old.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [27 במרץ, 2017, 9:47pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/2 "2017-03-27T21:47:09Z")
If you post your domain, people could have a look.
You might try the [sitemap plugin](https://github.com/communiteq/discourse-sitemap). I’ve seen people make very convincing arguments that it’s absolutely necessary and that it’s [absolutely unnecessary](https://meta.discourse.org/t/how-to-create-xml-sitemap-is-there-any-plugin/33654/4). One client I have claimed that it made their site (with 230K imported posts) get indexed very quickly after waiting for some weeks.
---
### Author:  [@john3](https://meta.discourse.org/u/john3)
#### Post date: [27 במרץ, 2017, 10:03pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/3 "2017-03-27T22:03:10Z")
I tried to fetch the hoe page from google crawl sites. and found out that its unreachable. However, domain is working fine its reachable.
`Network unreachable: robots.txt unreachableWe were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.`
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [27 במרץ, 2017, 10:06pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/4 "2017-03-27T22:06:50Z")
> [@john3](#):
>
> I tried to fetch the hoe page from google crawl sites. and found out that its unreachable. However, domain is working fine its reachable.
This is really bad, it should be reachable. Did it give any error?
---
### Author:  [@john3](https://meta.discourse.org/u/john3)
#### Post date: [27 במרץ, 2017, 10:13pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/5 "2017-03-27T22:13:47Z")
yes.. network unreachable. quite confusing when the domain is working fine.
---
### Author:  [@Mittineague](https://meta.discourse.org/u/Mittineague)
#### Post date: [27 במרץ, 2017, 10:17pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/6 "2017-03-27T22:17:45Z")
If you use your browser to go to
`example.com/robots.txt`
do you see it OK?
---
### Author:  [@john3](https://meta.discourse.org/u/john3)
#### Post date: [27 במרץ, 2017, 10:30pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/7 "2017-03-27T22:30:39Z")
Hi,
It was actually a stupid error.
I was redirecting discourse via proxy to https. however, http was also working on the top level domain. hence because of the confusion between https & http, I think google was unable to fetch robots.txt file.
I just forced http to use https completely. and now everything looks fine.
thanks everyone for helping.
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [28 במרץ, 2017, 1:50pm UTC](https://meta.discourse.org/t/forum-not-being-updated-on-search-engines/59990/8 "2017-03-28T13:50:00Z")