How do you guys deal with users that use AI to automatically reply to threads so they can gain trust level?

Gemeenschap
1

This question is for all the forum owners/moderators, what are your thoughts on users who use ai to automatically browse the forum, automatically read the past few comments, and the ai would automatically write a response to the thread and post it. They do this so they dont have to actually participate in the forum, because they want to gain higher trust level to access trust-gated sections of the forum.

Do you guys actively police these users? how do you tell if something is written by ai? do you turn a blind eye?

3 likes
2

It can get even more dishonest. I’ve seen posts where one sock puppet account posts one new topic with AI then a shortish while later a new user (presumably owned by same actor) posts a reply agreeing with and complimenting the OP

4 likes
3

Haven’t seen this behaviour, but it would be a ban from me. If they were members in good standing, it would be a warning, but this sounds like they never contributed.

It helps, I think, to have an explicit policy on LLM contributions.

6 likes
4

well how you should handle this issue my question would be how much do you value your trust-gated content and how much you value access to said content.

If it is content you don’t want people ‘gaming the system’ to gain access, I’d figure out how to try to stop such behavior.

Reason being such behavior is common place. See Robert’s example:

ah… this goes on here… every day

Anyhow

a lot of times it is pretty obvious to a human moderator by simply reading it but that takes moderator time. Some admins are employing AI to catch the AI (funny huh?)

see similar discussion here:

4 likes
5

If you are able to isolate the user-agents used, or if they honour control standards, you could consider blocking them, or rate limiting them or using llms.txt.

These only work if the automations behave like gentleman.

If particular regions are an issue there is always geo-blocking as well.

I use an ai detecting agent to screen new/low trust levels. It has proven to be around 95+% accurate.

It does depend how clever your bad actors are, but with some work and testing you can be cleverer :wink:

4 likes
6

We’ve seen this quite a bit, too. Of course I cannot speak to motivations, but in my experience it’s far less interesting than trying to “gain trust.” I think it’s just spam.

Spam now also includes AI training datasets and GEO — where it’s beneficial for them to just mention a product or service or company all over the internet.

So sometimes it’s to farm enough trust to drop a link, but I suspect links are secondary now.

So far the tells have been fairly obvious — my “honest take” is that the “load-bearing seams” show through clearly. I put the topic on hold and send the user a friendly (but canned) DM saying welcome and to just reply to me to bring it back.

Nobody has ever responded to me.

4 likes
7

As someone else suggested, it sounds like you might need to reconsider the value of your gated content and the relative value of user contributions to the forum. Do people who don’t have access to the gated content have the knowledge or skills needed to contribute meaningfully? Maybe change the requirements for getting to a new trust level in such a way that posting garbage won’t help and somehow make it clear so that people won’t be tempted to do that.

Before Ai were new users able to contribute meaningfully and get to the next trust level?

4 likes
8

Surprisingly or unsurprisingly, I’ve used the AI helper on posts here to determine if a post was AI (partly to test, partly for fun), after determining myself if it truly was. And lo! It is surprisingly accurate to determine it as AI/human written.

If it was blatantly obvious that the user was using AI to literally game the forum, I would suggest a warning or 2, followed by a suspension of perhaps a few days.

1 like
9

I would go back to first principles—is the content they are producing valuable to the wider community? If it is, does it matter how it was created? If it isn’t, deal with that problem. If someone is sockpuppeting and creating a whole lot of crap, the consequences should be the same whether it is AI or human, should they not?

6 likes
10

Our community is on standby and I thought a lot about this, because in the times we are living almost nobody respect policies or principles. Thus I implemented a custom trust level / karma system where users need to do specific (human) quests to gain access.

We will start with a bare low, but if I see the behavior you named here, I will raise it to make it boring, annoying or almost impossible to automate and/or accomplish using LLMs.

Our community is based on human interaction, we left Discord because their centralized system is not aligned with our philosophy. I also expect to see AI generated live cameras someday, need to plan what to do with them.

I moderated and managed a couple of forums in my Internet life and I learned the hard way that coercive or punitive actions usually have counter-effects. They need to be the last ratio, always.

You know, “do not feed the troll” applies everywhere.

3 likes
11

TBH I’m concerned if the TL system is starting to fail because it is fundamental to our layered defence. I am not seeing it happen here on meta—have you customised your AI spam detection or TL settings?

3 likes
12

Being honest and letting my inner <90s movie buff continue writing, I’m re-starting a niche community from Latam that will be opened to the world, because I think forums need to survive and in a couple of years we —humans— will be the 0.5-1% of the world.

Digg was gone, Stack Overflow, gone. Reddit is not human-driven anymore. I was there and opted out when I saw the actual monster coming. Limits on APIs, banning alternative clients, cookies everywhere, captchas that ask a human ‘to prove not being a bot’, fingerprints on every interaction, and so on.

Re-opening our community will open us to spam bots, but I expect more attacks on our infrastructure because we (I) like to always self-host, so I chose to forget everything about SEO, limit everything I can, keep the interactions between ‘us’, learned how to encrypt, distribute in decentralized networks and to be on Tor where Discourse can be used through Dumbcourse.

That’s the last defense line and it is not necessary today, but my dystopian (perhaps lunatic from who listens or reads) side is always on standby, you know. We see the movies: agents, spiders, full control and the illusion over almost everyone.

Turning back into Discourse, I do not expect spam abuse because we’re doing something unusual and outside what the majority follows, but I do expect many users to try to exploit the AI in their interactions to make themselves sound better than they are. Not only is that impossible, but it’s also the worst decision anyone can make wanting to improve.

So we will move from an emotional crowd following a leader to a trustable group of moderators keeping the agents out. I think the trust level system of Discourse was great, but it is not encouraged by the world as it is today; people are trained to get everything instantly and get almost nothing valuable in return.

My decision to make an old-school system instead of the common badge/trust one would cut all the noise and I think I chose the best platform for a forum led by the best team on the world. We will see where everything is going on.

I read your article and the one about mitosis or toxic fragmentation. It is always beautiful to read people writing from what they really live and own. I’m pleasantly surprised that you’re familiar with the “dead internet” theory. It is now.

1 like
13

Re spam. 2-ish years ago we moved to discourse from a dated and dying platform that was awful in every respect. Spam was impossible to keep ahead of.

Discourse pulled the handbrake on that, when AI detection was enabled it was a gamechanger.

I can’t recall a topic that has gotten through. There are a handful of false positives for new members but nothing major and that cost is nothing compared to before.

We were able to block a handful of geos that were creating vast amounts of accounts, which dropped it to a level that isn’t a burden nor worth worrying about.

What I enjoy about discourse is the flexibility you have to deal with problems, the constant evolution and the willingness of the team to help. It is a refreshing change.

2 likes
14

I do not have all the experience building on the Internet that the overall Meta community has, because I always am on selected/small/niche places. I was like a decade for the foundations of peer to peer systems, and also understood English from a couple of years ago, so I was almost out of the big show.

Meanwhile I received a spam attack on Ghost CMS and I needed to disable registration because the system was not prepared to handle it. It was crazy, the SMTP server resources used in a month what it used like in two years.

Maintaining selected communities allows for human review of accounts; I think it is the only reliable method to keep crawlers/bots away. Nowadays there is no system they can’t surpass; the compute they manage is evolving and gets stronger every day.

Good tools will help like always, but the real defense is on us.

15

I think it does. In some cases, we’ve had spammers ask AI-generated evocative questions for which our community has collectively spent a good amount of time… only for the “OP” to come back and drop a spam link or retroactively edit the question to include one. They’re not obviously troll-y before the turn… but they do often have tells of AI generation.

Folks will only get burned like that so many times. It destroys the goodwill of the community.

2 likes
16

I would guess your forum isn’t the only target and with that in mind, obviously the entity that is doing this probably does not what to spend time authoring the original topic. As such an AI generated post saves time in the overall scheme. Therefore vigilance toward blocking AI authored posts seems possibly worthwhile method of reducing spam attacks it would seem.