Hi Jeff,
FYI, it is fairly common knowledge, not my personal opinion, that bot traffic exceeds human traffic on the Internet.
It is also common knowledge, not my personal opinion, that a large percentage of bot traffic is by bots which do not respect robots.txt. Some estimate at least half, my experience is that it is “site and subject dependent”.
I’m glad you have a different experience at the companies you have founded and built and very happy for you.
On the other hand, the fact on the Internet is that bot traffic in 2020 is about between 55 and 60 percent of all traffic; and out of that bot traffic, around perhaps half is from bot traffic which does not respect robots.txt. Some research will. put the figure of “bad bots” to as low as 35 percent of all traffic, some higher, depending on the research. I’m not making this up, it is well documented.
If you have research papers or statistics outside of your experience hosting at Discourse or at your prior experience; showing “bad bot traffic” is minuscule in nature, I would be very pleased to read it; because personally, I have never seen any research paper or referenced article which states that “bad bot” traffic is so minuscule like you are stating here.
I apologize if not agreeing with you displeases you. I have provided references and can provide more references (not my opinion), if you are open to the facts about Internet traffic.
Otherwise, I’ll stop posting on this topic and so not to annoy you 
as I don’t want to be disagreeable with you over something you have a strong opinion about on a forum which I have no admin power
Happy Holiday Season!