Noindex - Follow in Tags Meta robots - Avoid duplicate content

(Veer) #1

Tags are a big necessity for my forum, i want tags to either not indexed in robots.txt, or it will be best if it have Noindex - follow in its Header meta robots.
How can i achieve it ?
It can be very very benefitial to avoid any Google panda or penguin penalty.
@denharsh

  • Also how to put sitemap link in Robots.txt ?
The «Tagging» core functionality adds multiple pages with the same human content (but different to Google) to SERP
The «Tagging» core functionality adds multiple pages with the same human content (but different to Google) to SERP
(Sam Saffron) #2

robots.txt is not really customizable without a plugin. However, I agree the various /tags/ routes should probably not be indexed by google, seems like an inefficient way to reach content.

we should look into this, also I am not totally against adding sitemap support at some point.