Note… I have seen a misguided sentiment out there of:
Just block it
This is a one way relationship
I feel this is missing one important point. Having OpenAI crawl meta.discourse.org has been highly beneficial for CDCK. When you ask GPT 4 Discourse questions it has at least a fighting chance of answering them.
It is a two way relationship:
You give Open AI access to data
Open AI burn forests training the LLM on your data, which can result in value for you.
Also related: How to prevent community content from being used to train LLMs like ChatGPT?
We see some GPTBot access across our fleets, maybe 20-40x less traffic than we see from Googlebot
Anyone uncomfortable with it can block in the Discourse UI direct, but the bot appears to be very well behaved compared to some bad ones we have seen.