I’d love to hear the experiences of staff of forums who don’t allow AI/LLM generated content. How have you been able to communicate this to users? How do you detect it? How do you approach users who post it anyway?
All thoughts are welcome.
Note: Personally, I’m only interested in the human side of things on the front end interactions on the site. I’m assuming that blocking crawlers is a lost cause.
Our forum is a spiritual/religious discussion forum. We ban any and all AI generated content.
Most, if not all, AI text can be easily detected just be reading it. Google’s SynthID is cool tech for detecting AI images and it claims to even be able to detect text probably only written by Gemini but OpenAI also supports the standard. Being able to personally detect the text myself is probably an acquired skill but I appreciate the work being done to respond to the current crisis we have of not being able to detect AI imagery or text.
Muting/suspensions are still the right way to go for this in my opinion, especially if the account is new. if there is a random new account that joins your site and instantly posts an AI generated topic I see no reason why you shouldn’t just suspend the account and block it.
As for the entire scraping dilemma: My site is for internal communication & documentation within a small company at the moment and I’m planning on using it as a backend for blogging eventually. It was not hard to set up a honeypot to deter the crawlers that opt to ignore the robotstxt files on my domains.
Just this one tactic alone has led to somewhere in the neighborhood of 6 MILLION requests over the span of two weeks (about 6 reqs/s to the domain):
Whenever an AI crawler visits said site, they are led to an infinite maze of spam using the lovely iocaine project self-hosted with a dataset of roughly ~7000 made up words, some gibberish HTML, random words, and fake news made by 8B Llama) .
Obviously this is a nuclear “go away” tactic and is not for everybody but it has been great for me in my goal of stopping LLMs from taking my code or text content. I remember reading a case study Anthropic did about LLM poisoning but I can’t find the article anymore so it won’t be attached here, but surely at some point they need to block my domain when they realize the bot has sent a cool 5 million requests to my domain as recently.
(I notice we’re setting aside the question of crawler load, crawlers taking content for training, and the social and economic consequences of the current rapid developments. That’s good.)
For myself, on a low volume hobby site,
A suggested form of guidance might look like this:
(In our hobby environment, there’s an extra angle, which is use of LLMs within the hobby, which covers a spectrum of possibilities and has both its enthusiasts and its detractors.)
This example does not involve Discourse, but the policy and governance issues would certainly overlap.
Both the English and German versions of Wikipedia now have what I would describe as extreme AI-purity policies. As an editor with 16 years experience, I have been wrongly accused of adding AI‑generated content to an article that I predominantly wrote. And the process of seeking a remedy has been bruising. That includes filing formal complaints to the Wikipedia EN ArbCom committee and the Wikimedia Foundation Trust and Safety Team (WMF TST) — dismissed summarily or rejected without reason, respectively, thus far.
I don’t want to recount my experiences here, partly because they are ongoing. However, I would encourage readers here to review this extraordinary exchange regarding the Wikipedia EN article on Fern Cave, a site of cultural significance to the Modoc people of California:
The author in question has to deny using AI four times. Their indigenous knowledge is also discounted in the process. The enforcing editor concludes with a remark that resembles concern but that can equally be read as a veiled threat to sanction if they fail to yield.
Psychological tactics like these are regularly deployed by enforcing editors and I have evidence of other examples.
If you look at the source markup in that same exchange, you will find a font-size: 0pt honeypot trap set to try and detect AI usage by the accused author when formulating replies. This trap was not sprung.
There are credible reports of several editors abandoning Wikipedia DE after being “unfairly attacked” over their alleged use of AI tooling.
To answer the original question posed by this topic, I would argue that Wikipedia EN and DE are failing badly in their governance in multiple ways in their attempts to deal with AI content. I documented three case studies in my recent 17 page report to the WMF TST team. I may well make that PDF public in due course?
I strongly disagree, given my experiences on Wikipedia EN (detailed elsewhere on this topic). I suggest this approach is mostly pseudoscience mixed with hubris. More significantly, such detection is not possible to refute — and indeed the enforcing editors on Wikipedia, often working in concert, simply dig their toes in under challenge.
The accused editor is left with a reverse burden of proof and accusations that are simply not falsifiable — even given the most compelling evidence to the contrary. Natural justice evaporates. Not the kind of community governance that usually works out.
I know. I have been subject to this process first hand. It’s ugly. There has to be a better way.
Relevant reading, especially the second article was eye-opening for me when I first read it. I think that partly also applies to you @robbie.morrison . Some people “just write that way”.
And I have come to this thesis statement: I don’t write like ChatGPT. ChatGPT, in its strange, disembodied, globally-sourced way, writes like me. Or, more accurately, it writes like the millions of us who were pushed through a very particular educational and societal pipeline, a pipeline deliberately designed to sandpaper away ambiguity, and forge our thoughts into a very specific, very formal, and very impressive shape.
Yes, bias against non-native English speakers is one of my biggest concerns.
Also not a Discourse community, but this is an interesting read: Rules Roundtable XXI.V: Plagiarism, Integrity, and Generative AI. It’s from the AskHistorians subreddit. For those who aren’t familiar, this is probably one of the most strictly moderated communities on Reddit. They basically only allow academic level responses to questions.
The AskHistorians reddit still has a “shoot first” bias, followed by a reverse burden of proof for the accused (but certainly more due process than Wikipedia/Wikimedia):
If you do not say you used AI to generate an answer and we suspect that you may be using AI in a rule-violating way, you will receive a ban. However, everyone banned for AI-use is told how to submit an appeal. It feels harsh, but part of our rationale is to bring people to modmail so we can have a discussion and avoid a public call-out that can lead to pile-ons.
Perhaps that is the only workable strategy at present? But hardly natural justice.
PS: that reddit posting in full:
Yeah, I guess that sub is not at all about “community”. They are ruthless with what they accept, AI or not.
But the whole accusing someone of using AI is fraught.
I’ve been using the – since forever, and now I can’t anymore because people will think it’s AI. RIP long dash.
Yeah, really anyone who takes advantage of Markdown and composes a well written post could be suspect.
same! I’ve been overusing the emdash for years before AI rolled around and stole it 
I make too many mistakes to be wrongly identified as an AI 
Strangely, I noticed I’m more inclined than before to voluntarily not correct small mistakes I notice in my own posts (even in French, my native language). I feel like leaving those bits of imperfection makes what I write more authentic and have more identity. People could potentially identify a message as mine because of those mistakes or weirdly written sentences. It’s also less mentally tiring not trying to make everything I write perfect (whatever it means).
