I’m right. Theirs documentation is… overpositive. You can google the right answer really quickly — good starting point is what googlebot itself does or not. And after that basics of robots.txt.
But you don’t need trust on me or anybody else. You can try it. First you should check out all visited bots. And after that how many of those have looked robots.txt. You will be suprised
You have plenty of too active bots and you can use robots.txt to slow down them. After a couple of days you’ll get the answer.
That is more or less waste of time, though. Issues of robots.txt are widely known.