I just opened our forums to be discovered by Google and used the search console to index our site. (I’m not sure if there are other things that I should be doing ¯\_(ツ)_/¯ (any suggestions?)
Here’s the feedback I got from Google Search Console:
AI-generated summary
jord8on recently switched their forums from private to public and submitted them to Google Search Console for indexing. However, they received a message indicating they were blocked by their robots.txt file. After checking, Stephen found that the robots.txt file isn’t blocking Google, and discovery1368 suggested that the file might be empty or incorrectly formatted. riking noted that since the forum was previously blocked, it might take some time for Google to recheck the robots.txt file and crawl the site. jord8on confirmed this and hopes to get indexed soon.
Have you checked your robots.txt?
I don’t know if this makes a difference in the case of my question but one other thing to note is that i’m doing SSO and using Wordpress as my provider. would that make a difference, here?
My site is pickleballist.com
My forum is forums.pickleballist.com
Your robots.txt lives in the root of each site, so:
https://pickleballist.com/robots.txt
https://forums.pickleballist.com/robots.txt
Nothing in those files is blocking Google, you can check this for yourself using third party robots.txt scanners, for example this tool by ryte.
Which confirms the robots.txt doesn’t block the Googlebot:
Google also has a tester within the search console.
User-agent: *
Disallow:
Delete the link to block in the disable file robots.txt
Because your forum has been login_required and/or blocking indexing for so long, it’ll be a while before the crawler rechecks your robots.txt file and sees that it’s been unblocked. There’s probably a button to request a recrawl somewhere in there, or you can just wait.
Yes you are right that it has been “login_required” for many months. Hopefully we’ll get indexed soon!