Frankz reported that their Discourse-hosted forum isn’t appearing in Google or Bing search results. NateDhaliwal asked if the site was recently launched and suggested checking crawler settings and submitting the site to Google Search Console. Frankz confirmed the site was new and noted that WordPress sites indexed faster. NateDhaliwal followed up asking if crawler user agents were modified and linked to a helpful guide. tobiaseigen encouraged checking settings and contacting support if unresolved, noting that new sites take time to be indexed. JammyDodger suggested verifying Allow index and robots.txt settings. Frankz confirmed the site is on Discourse and asked about the setting. tobiaseigen guided them to Admin > All site settings > search “allow index” and offered further support. Frankz reported success after adding www. before the domain. tobiaseigen explained that common domain words may confuse search engines and recommended using https:// or including topic keywords in searches. supermathie quoted Frankz’s original post and reminded them to search using site:yourdomain.com to verify indexing status. The issue was resolved by adjusting domain formatting and confirming proper indexing settings.
Hi, my whole website is a forum built by Disourse. I searched the domain of my website, both Google and Bing can not display my website domain. AI says perhaps some settings of crawlers on Discourse security block the google search. Is it right? And how can I make google search access my website?
Thanks,
Frank
Hi there, did you launch the forum recently? Is there enough high-quality content? IIRC that’s a key factor in ensuring your pages are indexed by search engines.
Yes, I uploaded the website just for a few weeks, but when I built other websites with Wordpress, google search would display the domain almost at once. Is it really I don’t need to change some settings to make google search find my website?
Did you change whitelisted crawler user agents or blacklisted crawler user agents?
Did you add your site to Google Search Console?
You may also want to see:
There’s a lot of helpful stuff in there.
@Frankz were you able to figure this out? Is your site showing up in internet searches now? I think Nate’s suggestions are correct. It takes a while for search engines to notice a new site. Do check your settings!
If this is a site hosted by us, reach out to team@discourse.org and our support team can help you verify your settings are correct. The defaults should be fine.
Possibly also worth double-checking Allow index in robots TXT is still enabled as well.
My website is hosted on Discourse Forum. And where is the setting of Allow index in robot TXT?
Hi Frankz! On your site, go to Admin, then select “All site settings” on the sidebar, then search for “allow index”. You will see the setting then. Is it selected as it should be?
After this if you still have trouble please write to team@discourse.org. Our support team will happily help you.
I tried to add www. before my domain name, then I can see my website on google search results. Thanks, Frank
Nice! The domain name for your forum also contains some common words, so I am not surprised google didn’t recognize it when you put it by itself. You can also put https:// in front of it to make sure google recognizes it as a website URL. I also searched for a few topic titles and see that your forum does appear at the top of search results, so I think you are in good shape here. 
Search using: site:hostname, e.g. site:meta.discourse.org
That’ll show you what it has indexed for your site.