# Just switched from private to public, blocked by robots.txt

**URL:** https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473
**Category:** Support
**Created:** [24 maart 2020 om 23:07 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473 "2020-03-24T23:07:34Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![jord8on](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jord8on/32/166809_2.png) [@jord8on](https://meta.discourse.org/u/jord8on)
#### Post date: [24 maart 2020 om 23:07 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/1 "2020-03-24T23:07:34Z")

</div>

I just opened our forums to be discovered by Google and used the search console to index our site. (I’m not sure if there are other things that I should be doing ¯\_(ツ)\_/¯ (any suggestions?)

Here’s the feedback I got from Google Search Console:

 ![image](https://global.discourse-cdn.com/meta/original/3X/5/7/57eccb01298c0f93594ce292afed7fc23f6d70fe.png)

---

<div class="post-metadata">

### Author: ![Stephen](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/stephen/32/95011_2.png) [@Stephen](https://meta.discourse.org/u/Stephen)
#### Post date: [24 maart 2020 om 23:12 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/2 "2020-03-24T23:12:10Z")

</div>

Have you checked your robots.txt?

---

<div class="post-metadata">

### Author: ![jord8on](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jord8on/32/166809_2.png) [@jord8on](https://meta.discourse.org/u/jord8on)
#### Post date: [24 maart 2020 om 23:15 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/3 "2020-03-24T23:15:21Z")

</div>

I’m not sure what to look at in my robots.txt or where to find it.

I did make sure this box was checked, in my admin settings…

 ![image](https://global.discourse-cdn.com/meta/original/3X/3/7/371f762180898544fffeda9972af0cf7553ebf0e.png)

---

<div class="post-metadata">

### Author: ![jord8on](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jord8on/32/166809_2.png) [@jord8on](https://meta.discourse.org/u/jord8on)
#### Post date: [24 maart 2020 om 23:17 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/4 "2020-03-24T23:17:36Z")

</div>

I don’t know if this makes a difference in the case of my question but one other thing to note is that i’m doing SSO and using Wordpress as my provider. would that make a difference, here?

My site is [pickleballist.com](http://pickleballist.com)  
My forum is [forums.pickleballist.com](http://forums.pickleballist.com)

---

<div class="post-metadata">

### Author: ![Stephen](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/stephen/32/95011_2.png) [@Stephen](https://meta.discourse.org/u/Stephen)
#### Post date: [25 maart 2020 om 00:06 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/5 "2020-03-25T00:06:48Z")

</div>

Your `robots.txt` lives in the root of each site, so:

```
https://pickleballist.com/robots.txt
https://forums.pickleballist.com/robots.txt

```

Nothing in those files is blocking Google, you can check this for yourself using third party robots.txt scanners, for example [this tool by ryte](https://en.ryte.com/free-tools/robots-txt/?refresh=1&url=https%3A%2F%2Fforums.pickleballist.com%2Frobots.txt&useragent=Googlebot&submit=Evaluate).

Which confirms the robots.txt doesn’t block the Googlebot:

 ![RYTE Robots Test](https://global.discourse-cdn.com/meta/original/3X/2/3/23c827b73714cf4fab4af8445edc8bf55ae5d504.png)

Google also has a tester [within the search console](https://support.google.com/webmasters/answer/6062598?hl=en).

---

<div class="post-metadata">

### Author: ![discovery1368](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/discovery1368/32/174095_2.png) [@discovery1368](https://meta.discourse.org/u/discovery1368)
#### Post date: [25 maart 2020 om 02:54 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/6 "2020-03-25T02:54:23Z")

</div>

User-agent: \*  
Disallow:

Delete the link to block in the disable file robots.txt

---

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [25 maart 2020 om 04:49 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/7 "2020-03-25T04:49:13Z")

</div>

Because your forum has been login\_required and/or blocking indexing for so long, it’ll be a while before the crawler rechecks your robots.txt file and sees that it’s been unblocked. There’s probably a button to request a recrawl somewhere in there, or you can just wait.

---

<div class="post-metadata">

### Author: ![jord8on](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jord8on/32/166809_2.png) [@jord8on](https://meta.discourse.org/u/jord8on)
#### Post date: [25 maart 2020 om 18:58 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/8 "2020-03-25T18:58:22Z")

</div>

Yes you are right that it has been “login\_required” for many months. Hopefully we’ll get indexed soon!

---

<div class="post-metadata">

### Author: ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)
#### Post date: [24 april 2020 om 19:00 UTC](https://meta.discourse.org/t/just-switched-from-private-to-public-blocked-by-robots-txt/145473/9 "2020-04-24T19:00:01Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
