I’ve seen: [image] Pages listed in the robots.txt are crawled and indexed by Google Support Pages meant to be hidden from Google are in the robots.txt However, Google attempts to crawl them anyway. Since they are accessible through links on web pages, th…

The “upshot” or outcome of that discussion appears to be very clear: a new feature was added to add noindex for pages that shouldn’t be crawled that new feature had to be removed/reverted because Google didn’t support it and was then reporting robots.txt as invalid.

Yes, Google stopped supporting noindex in robots.txt. But their recommendation now appears to be putting a noindex meta tag on all pages that you want to be entirely eliminated from their index. That’s what I’m trying to accomplish.

Sorry, I should have been more explicit that the first part of the topic post should be removed. You are probably better to refer to the Google announcement because that has a link to the page HTML option that pfaffman suggests above: Block search indexing with ‘noindex’

I haven’t done much theme work, so this is exactly what I needed to know. Thanks! And it’s showing up in the head correctly (added to “head_tag”). For the record, the Google recommended tag if you want to ask all robots not to index, is: <meta name="robots" content="noindex"> I’ve now disabled (w…

[image] L30110: I haven’t done much theme work, so this is exactly what I needed to know. Glad I could help! I’m still not very good at themes and know nothing about SEO, so I didn’t know what the meta tag actually needed to look like. Glad I got you close enough for you to solve it.

How to noindex all pages to keep them out of Google indexes

Product Support

pfaffman (Jay Pfaffman) March 7, 2020, 6:54pm 4

Can’t you just make a theme component that will add the noindex meta tag to all pages?

Just add

<meta name="noindex">

to /admin/ customize/ themes/ common/ head_tag

Right?

Topic		Replies	Views
Pages listed in the robots.txt are crawled and indexed by Google Product Support	18	3584	July 30, 2019
Google notification to remove "noindex" statements from robots.txt Product Support	7	2622	July 30, 2019
Add "noindex" on tag pages Product Support	4	855	February 9, 2021
I have indexed my site to google and get error “No: ‘noindex’ detected in ‘X-Robots-Tag’”. How to remove this header tag? Data & reporting analytics	4	1305	April 24, 2025
Delete indexes Product Support	0	120	June 21, 2024

How to noindex all pages to keep them out of Google indexes

Related topics