Google Search Console started sending me error reports about some Discourse pages. URL is on Google, but has issues It can appear in Google Search results (if not subject to a manual action or removal request). However, some issues prevent it from being eligible for all enhancements. [Learn more …

People on here keep asking for MORE SEO JUICE and INTERLINKING every so often and I honestly can’t think of anything better along those lines than linking to the most-liked posts + recent posts you’ve made. Also, a forum user page is a valid search result for someone who wants to see your recent po…

We did the “please index the user page” conversation so many times though… in fact, to be honest, I think even mentioning this automatically raises the blood temperature in Jeff’s head automatically by 10 degrees. Let’s keep this focused on just properly excluding the user page so we can stop deali…

Hmm we could make it so TL2 and higher get indexed user pages @sam . That would be consistent with nofollow rules in posts too.

[image] sam: mentioning this automatically raises the blood temperature in Jeff’s head automatically by 10 degrees. It raised mine by more than a few mm Hg As someone that has been a moderator for several years and seen countless examples of what I call “profile spam” I beg, please no. To t…

I do like the idea of allowing this to TL2, agree it is consistent with the nofollow stuff.

If you are adding the per-page tag it is almost the same code anyway, and we’d need to remove the robots.txt user path block anyhow.

Using the noindex tag is the right thing anyway, as far as I understand: The discussion is about whether we want Google to index the user pages, but robots.txt does not prevent indexing : You should not use robots.txt as a means to hide your web pages from Google Search results. This is because ot…

Yeah, I don’t know if there’s any practical difference or not. I just think that it’s not a great experience if Discourse forum admins get emails from the Google Search Console that their site has failed validation (I assume others are experiencing this as well, since I didn’t make any special chang…

For my specific forum it would be great to have user pages indexed, it is a paid site so we don’t have the spammer problem. We have a group of photographers whom we actually want to help promote their work. All of our users automatically go to TL2 so that would work for us, at least having the optio…

This just started happing on my discourse install (self hosted). It reporting this for all the rss links e.g <link rel="alternate" type="application/rss+xml" title="RSS feed of 'Is there a performance penalty in calling an external FB script?'" href="https://forums.finalbuilder.com/t/is-th…

Google complaining – Indexed, though blocked by robots.txt

Support

codinghorror (Jeff Atwood) October 31, 2019, 3:16am 21

Then update to latest beta, where this is addressed.

Topic		Replies	Views
Why there are lots of Disallow rule in robots.txt? Support	34	4932	December 22, 2020
Issues Google Search Console is throwing at me for wrong discourse structure (or some for wrong administration of my site) Support	18	384	December 18, 2024
Pages listed in the robots.txt are crawled and indexed by Google Support	18	3539	July 30, 2019
Why isn't Google Indexing Discourse? SEO concerns Support seo	31	6303	June 1, 2024
Google indexing issue (robots.txt) Support	7	821	April 18, 2020

Google complaining – Indexed, though blocked by robots.txt

Related topics