I wonder what’s the best place to disable indexing (set noindex) on the entire site, and I mean across the board.
Is it enough to simply insert NOINDEX in Discourse header (I wonder if that would apply to all pages/URLs generated by the site)?
P.S. And for misbehaving crawlers such as some search engines, where’s the best place to block countries, is it before Docker NAT or after, inside of Docker? I’d like to block entire countries. It seems it’s cheapest to drop connections with iptables before they hit the container, but I am not sure if that’s a good practice.