There are a few topics about this already. The simplest explanation that I have seen is in this post:
A detailed explanation of how Discourse counts crawler traffic is in this topic: