# Are you experiencing AI based spam?
**URL:** https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707
**Category:** Community Building
**Tags:** ai
**Created:** [January 24, 2024, 1:26am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707 "2024-01-24T01:26:13Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Saif](https://meta.discourse.org/u/Saif)
#### Post date: [January 24, 2024, 1:26am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/1 "2024-01-24T01:26:13Z")
Im curious to hear from community members whether they are experiencing any or an uptake in AI powered spam
This would be specifically seeing answers to questions that look like they are ChatGPT based and seem either non-human like or have hallucinations (a common problem with LLMs)
## I am experiencing AI based spam
_Poll ([view on site](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/1))_
If the answer is **yes** Im curious to hear…
- How often this is happening?
- How much of a problem is this creating within your community?
- What are you currently doing about it?
If the answer is **no** Im curious to know…
- How are you preventing this from happening?
- Are there reasons as to why your community inherently doesn’t face this issue?
---
### Author:  [@wellbing-go](https://meta.discourse.org/u/wellbing-go)
#### Post date: [January 24, 2024, 2:18am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/2 "2024-01-24T02:18:24Z")
We just use AI as a tool to seek knowledge, maybe a little causal chat.
Perhaps our community is small, and has a _common sense_ that hallucinations are **BAD**
---
### Author:  [@maiki](https://meta.discourse.org/u/maiki)
#### Post date: [January 24, 2024, 3:39am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/3 "2024-01-24T03:39:07Z")
> [@Saif](#):
>
> Are there reasons as to why your community inherently doesn’t face this issue?
Private community (login required, invite only).
---
### Author:  [@Jagster](https://meta.discourse.org/u/Jagster)
#### Post date: [January 24, 2024, 6:53am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/4 "2024-01-24T06:53:45Z")
I reckon the most effective way to stop anykind spamming is being member of very small and difficult language. It stops those clowns who are doing manual labour.
Well, we all know spammers aren’t that smart and automatic traffic doesn’t care of language, genre or even size. So, there must be another reason why some forums or sites are like honey pots for anykind trash and others live without drama.
For the reason why spammers can sign in there and can’t somewhere else when system and setup should be identical I don’t have answer. But one thing is sure: admin’s or other background force’s need to increase fast growth from global audience will lead to bot ans spam problems.
---
### Author:  [@cbrandtbuffalo](https://meta.discourse.org/u/cbrandtbuffalo)
#### Post date: [March 26, 2024, 1:26pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/5 "2024-03-26T13:26:15Z")
> [@Saif](#):
>
> - How often this is happening?
In the last two weeks or so, we have seen a spike on our site. We’re seeing typical spam with hidden links on new replies from new accounts. When we increased the reputation for creating new posts, we saw AI-generated responses increase, and it seemed the bots were trying to slowly increase their reputation on bogus accounts. These responses don’t have obvious bogus links, they just have generic AI text that doesn’t contribute to answering the question.
> [@Saif](#):
>
> - How much of a problem is this creating within your community?
We got hit over a weekend with a large spike in spam posts, enough that someone created a new topic saying there was too much spam on our forum. Since then, admins need to check the site every day to clean up bogus AI posts. We’re also seeing AI posts on accounts that were created in the past and had no activity, which makes it seem like some spam bots had been seeding accounts for a while and letting them sit with no activity. Now they are trying to slowly get past the engagement limits so they can post new topics.
> [@Saif](#):
>
> - What are you currently doing about it?
As noted above, we increased the trust levels for posting new topics. We also enabled akismet. But this hasn’t stopped the AI spam posts. Currently we need an admin/moderator to check the forum every day to review flagged posts and clean up. Some are challenging and look like they might be a person, so two people need to check.
We encouraged our users to help out and flag posts that look like AI and that has helped.
Our forum is fairly low volume and has run for years with very low admin clean-up and maintenance, but it seems the AI bots have found us. I’m thinking AI may be needed to stop AI?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 26, 2024, 11:11pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/6 "2024-03-26T23:11:42Z")
> [@cbrandtbuffalo](#):
>
> I’m thinking AI may be needed to stop AI?
Yeah, sadly. Either that or you temporarily just vet all new users and slow down the time from “when a user signs up” till the post.
We do have:
> [@Discourse AI - AI triage](https://meta.discourse.org/t/discourse-ai-post-classifier-automation-rule/281227):
>
> AI triage is designed to enhance the management and moderation of forum posts by automating the process of classifying posts. Please note that this feature requires both the [discourse-automation](https://meta.discourse.org/t/discourse-automation/195773) and [discourse-ai](https://meta.discourse.org/t/discourse-ai/259214) plugins to function. Use Cases Automated Post Categorization: AI triage can automatically categorize topics based on their content. This is particularly beneficial for large forums where manual categorization can be time-consuming. You can apply the rules to a subset of topics (first…
It also supports flagging, so you could use that today.
---
### Author:  [@Saif](https://meta.discourse.org/u/Saif)
#### Post date: [March 31, 2024, 10:38am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/7 "2024-03-31T10:38:35Z")
On that note we just published a guide on this!
> [@Setting up spam detection in your community](https://meta.discourse.org/t/setting-up-spam-detection-in-your-community/300427):
>
> bookmark This is a #how-to guide for setting up spam detection in your community using the Discourse AI - AI triage. person_raising_hand Required user level: Administrator warning Discourse AI now ships an [efficient spam scanner that requires minimal setup](https://meta.discourse.org/t/discourse-ai-spam-detection/343541). For custom or complex use cases, we recommend following this guide Overview Spam detection is an essential feature for maintaining the quality of discussions in your community. This guide will help you set up spam detection using…
---
### Author:  [@Saif](https://meta.discourse.org/u/Saif)
#### Post date: [May 27, 2024, 3:13pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/8 "2024-05-27T15:13:09Z")
> [@Saif](#):
>
> On that note we just published a guide on this!
>
> > [@](#):
> >
> > Overview In this topic we are going to use [Discourse AI Post Classifier](https://meta.discourse.org/t/discourse-ai-post-classifier-automation-rule/281227) to detect spam. Note that the instructions here can be customized for your preference. See an example setup here… [[CleanShot 2024-03-21 at 15.19.11@2x]](https://global.discourse-cdn.com/meta/original/4X/9/a/b/9ab3bb5e0ed0c6764bbd6e35f3b75d2e4d330f2a.jpeg) Why should I use this? If you’ve tried Akismet and other anti-spam tools and were not happy with its results for detecting spam in your community Pre-requisites In order for this to work you will need the following enabled [Discourse AI](https://meta.discourse.org/t/discourse-ai/259214) [Discourse-Automation](https://meta.discourse.org/t/discourse-automation/195773) LLM (Large …
Following up on this, has anyone had a chance to try this out? I would love to get your feedback
---
### Author:  [@j127](https://meta.discourse.org/u/j127)
#### Post date: [May 27, 2024, 6:19pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/9 "2024-05-27T18:19:13Z")
It haven’t seen a lot of it yet, but my forum holds the first few posts in moderation, and I can usually tell if someone might be a spammer by certain clues. I lock the suspicious ones at TL0 until they post something that is clearly on topic.
It isn’t a “chat about random things” forum, so it’s usually possible to tell whether someone is faking interest by the first post.
---
### Author:  [@j127](https://meta.discourse.org/u/j127)
#### Post date: [May 27, 2024, 8:46pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/10 "2024-05-27T20:46:08Z")
Actually, I just stumbled on a user who slipped by and is posting with ChatGPT or other AI. There might be more spam accounts that I’ve missed.
Some ideas on how to fight it:
- Make a database of VPN providers. This one’s IP address is from “M247 Europe SRL” which is a VPN service provider. I’ve always wanted some kind of notification that a new account is using a VPN. I have to do it manually at the moment.
- Keep track of read time, days visited, topics/posts read. This user spent 8 minutes reading the site but posted 6 comments, and only visited 3 times on the day of their registration. The user is actually still TL0 naturally, because they haven’t really done anything except post comments.
- I wrote more ideas in comments on [this page](https://meta.discourse.org/t/block-googles-help-me-write/301295).
I wonder if it’s possible to roughly classify users by the ratio of time spent on the site vs. number of words written, plus other signals like VPN, pasted content, injected content, etc. Suspect accounts could be marked for review.
Edit: this quick [Data Explorer](https://meta.discourse.org/t/32566?silent=true) query turned up a few more, though some of them were already suspended.
```sql
SELECT
u.id,
u.created_at,
u.username,
u.trust_level,
us.time_read,
us.days_visited,
us.topics_entered,
us.post_count,
us.topic_count
FROM users u
LEFT JOIN user_stats us
ON us.user_id = u.id
WHERE u.trust_level < 1
AND u.created_at > '2023-01-01'
AND us.time_read < 1000 -- seconds
AND us.post_count > 1
```
---
### Author:  [@Saif](https://meta.discourse.org/u/Saif)
#### Post date: [May 27, 2024, 10:14pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/11 "2024-05-27T22:14:10Z")
> [@j127](#):
>
> Keep track of read time, days visited, topics/posts read. This user spent 8 minutes reading the site but posted 6 comments, and only visited 3 times on the day of their registration. The user is actually still TL0 naturally, because they haven’t really done anything except post comments.
This is an interesting take to weed out people who might “fake activity” in a single day to upgrade to a higher TL
I like the recommendation here to use additional ways to classify users, something to look into!
---
### Author:  [@JammyDodger](https://meta.discourse.org/u/JammyDodger)
#### Post date: [July 3, 2024, 8:55pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/12 "2024-07-03T20:55:20Z")
5 posts were split to a new topic: [Blocking recent wave of spam](https://meta.discourse.org/t/blocking-recent-wave-of-spam/314867)
---
### Author:  [@guidoleenders](https://meta.discourse.org/u/guidoleenders)
#### Post date: [September 20, 2024, 7:06am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/13 "2024-09-20T07:06:35Z")
One per day. Pattern on forums with 2000 or so users (500 per year new):
- new user signs up with email domain from category “temporary domain” such as “[cetnob.com](http://cetnob.com)”.
- within a number of hours either creates a new topic or replies to an existing topic
- creates text in English sections only (bilingual site, with over 90% NOT in English)
- sometimes includes a URL in the text, sometimes not
- text seems like an actual question or remark, using words uniquely used on the forums and relevant material
- but text feels somewhat off-topic, but very good still: an inexperienced support engineer could not detect it
It highly resembles in pattern the pattern described in:
> [@Approving a new user posts that contains links](https://meta.discourse.org/t/approving-a-new-user-posts-that-contains-links/314597):
>
> OK. So now we are seeing an increase in spam posts esp. using AI to generate plausible looking posts. Is there a way to require a review of posts which: Are made by TL0; and Contains a link (http://)? Currently the first x posts are reviewed, but esp. with AI, spammers post x amount of innocuous posts before posting the spam links.
We are blocking already hotmail, gmail and other large consumer oriented domains using an explicit list, but there are at least 10.000 domains known to us that are used for this type of approach. In our own software we have an explicit list plus a real-time check on UserCheck (we use the free variant and only check on sign up on our own apps and cache, so 5000 lookups per month is sufficient).
From what I have seen, this specific behaviour can be tackled by automatically blocking temporary / spam email domains.
Blocking TL0 from using links is not really considered more viable than moderating all new requests, since many users post directly after first sign up, the site being a support portal.
There is a plug-in for this, but it seems no longer maintained (see [Plugin to detect & reject disposable emails on signup](https://meta.discourse.org/t/plugin-to-detect-reject-disposable-emails-on-signup/162569)).
I am not sure whether this approach will solve all issues for instance for larger forums or forums that accept consumer email addresses.
---
### Author:  [@j127](https://meta.discourse.org/u/j127)
#### Post date: [October 10, 2024, 5:07pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/14 "2024-10-10T17:07:41Z")
I’m getting a lot of AI spammers lately, and it’s time-consuming to go through them.
With the current spammer I’m looking at, the text is written in perfect English, it’s a VPN, the email address is in StopForumSpam, and I can tell the content was copied/pasted because the dash character that was used doesn’t exist on keyboards. I had to check all of that manually though and still have several more to look at this morning.
Brainstorming another idea:
When a post is saved, Discourse could record extra data in a JSONB field on that post:
- IP address
- is\_vpn? — a lookup in maxmind to find the org and see if it’s a VPN (e.g., PacketHub S.A.)
- a quick lookup for the email address in StopForumSpam
- A comparison of number of characters output into the editor vs. number of output-producing characters typed (excluding arrow keys, ctrl, etc.). For example, the user output 1,000 characters in the raw content, but only pressed output-producing keys 10 times (suggesting that the content was pasted and the user then might have edited a word).
- Number of times content was copied or cut using keyboard shortcuts or right-click.
- Number of times content was pasted using keyboard shortcuts or right-click. The difference in the copy/paste numbers would provide another clue.
Moderators could view that data on posts in a small table. Unusual values could be highlighted so suspicious posts would stand out.
There probably isn’t a perfect method to automate the detection, but having more information would speed up the moderation process.
---
### Author:  [@Saif](https://meta.discourse.org/u/Saif)
#### Post date: [October 11, 2024, 2:38pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/15 "2024-10-11T14:38:37Z")
I’m curious if any of the following guides have helped since they the spam has only gotten more intricate over time
- [Setting up spam detection in your community](https://meta.discourse.org/t/setting-up-spam-detection-in-your-community/300427)
- [Setting up NSFW detection in your community](https://meta.discourse.org/t/setting-up-nsfw-detection-in-your-community/330196)
---
### Author:  [@j127](https://meta.discourse.org/u/j127)
#### Post date: [October 11, 2024, 4:55pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/16 "2024-10-11T16:55:37Z")
I haven’t used AI in my forums because it’s expensive. I’m also not convinced AI would be able to solve this problem, because the content looks normal. I’d still have to manually investigate every suspicious post.
I haven’t had a problem with NSFW content.
My problem isn’t that there is anything wrong with the content. The only suspicious things about it are that new users don’t write posts like that within a few minutes of registering, and the content is also somewhat vague. My main forum has a very specific topic, and if a first post doesn’t say something specific about the person’s relationship with that topic, I start the investigation process. Otherwise I might not notice their posts.
Here are a couple of examples. The content is just vague enough for me to start the investigation process, but it’s time consuming, because I have to do it manually.
I can’t ban this user based on content alone. It’s the other clues that tell me it’s a spammer.
This IP address was a VPN in Norway, and the content was too vague. I was able to confirm it because the email address was in StopForumSpam with an IP address of Germany:
I’m just brainstorming out loud here, but it would be faster to moderate these users if there were a small table on posts that said something like:
| | |
| --- | --- |
| location | Oslo, Norway [from maxmind] |
| organization | PacketHub S.A. [from maxmind] |
| is\_vpn | true |
| email | whatever@example [sometimes this provides clues] |
| stopforumspam | true [link] |
| characters\_output | 1,234 |
| characters\_output\_pressed | 10 [this doesn’t match the number of chars in the post, so it’s a clue] |
| num\_cut\_or\_copy | 0 [didn’t copy text from editor] |
| num\_paste | 1 [did make one paste] |
| seconds\_editor\_open | 20 [suspicious for a post of that length] |
Maybe the table could be collapsed unless there is a suspicious value and/or moderators could flag a specific user as “probably not spam” which would then collapse the table on all their posts or stop future lookups for that user. Or the user could automatically be marked as safe when they reach TL2.
It’s a combination of things:
- VPN or IP address tends to be in a small number of countries where there are a lot of SEO companies (India, Pakistan, Ukraine, Vietnam, Bangladesh).
- Sometimes the email addresses are in StopForumSpam.
- A lot of the content is pasted in to the editor, but probably not all of it.
- The email addresses often don’t match the username. E.g., the username will be “Bob Smith” and the email address will be something different like stevenjohnee1234@example.
- The raw content sometimes uses formatted punctuation characters like smart quotes or mdash, suggesting that the content wasn’t written in the Discourse editor.
---
### Author:  [@j127](https://meta.discourse.org/u/j127)
#### Post date: [October 13, 2024, 4:57pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/17 "2024-10-13T16:57:27Z")
Take a look at post `1622105` here in this forum. It was posted 3 minutes after registering, hand-edited to change the link from Quora to stackexchange, and the English is good, but it’s talking about technology that isn’t relevant to Discourse. I don’t want to link to it because it would notify the poster.
That’s the kind of post where it would be useful to see the data I mentioned above right in the post.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [October 14, 2024, 12:27am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/18 "2024-10-14T00:27:39Z")
A lot of this feels like staff experience vs AI. Will call the right people into the topic.
I do think it sound compelling to try to bulk up some of the “fast typing” detection we already have. Having SFS integrated into core may also be compelling.
There is of course the deep philosophical question:
> Is it spam if it adds value to the forum (even if it is AI generated)
> Should it be removed from the forum if it adds no value to the forum (even it if is human generated)
Neither of these have super clear answers.
---
### Author:  [@Ed\_S](https://meta.discourse.org/u/Ed_S)
#### Post date: [October 14, 2024, 1:00pm UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/20 "2024-10-14T13:00:46Z")
I think the suggestion to make more metadata available to the mods is a good one. Independent of improving the automatic features.
---
### Author:  [@j127](https://meta.discourse.org/u/j127)
#### Post date: [October 15, 2024, 1:17am UTC](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707/21 "2024-10-15T01:17:50Z")
> [@sam](#):
>
> > Is it spam if it adds value to the forum (even if it is AI generated)
>
> > Should it be removed from the forum if it adds no value to the forum (even it if is human generated)
>
> Neither of these have super clear answers.
It’s possible that it varies by forum.
I leave a few spammer posts online when they provoke discussion, but most of them get deleted. The quality is very low, and it’s often easy to tell when something is written by AI. If I feel like something I’m reading is AI, I start losing trust in the source. I’m not an AI Luddite, but I don’t want to read AI-generated content unless I know it’s generated by AI.
If I see that someone is using AI in the forum, I immediately put a stop to it because trustworthy content is one of the forum’s most important assets.
Also, what looks passable to humans in 2024 might be easy for people to detect as AI in 2034, kind of like how movie effects that once looked realistic decades ago are now immediately detectable as fake. I think AI-generated content from 2024 is going to look dated eventually.
[Next page](https://meta.discourse.org/t/are-you-experiencing-ai-based-spam/292707.md?page=2)