Too many Crawlers, is that a problem?

Data & reporting
4

Well, that depends on how large your site is and the user agent for that crawler. You don’t want to block Googlebot, for example.

Have a look here

3 Likes