Improved AI content labelling in the composer for post content

סיכום מבוסס בינה מלאכותית

martin announced an expansion to AI content labeling features, allowing users to tag code, quotes, and details blocks as AI-generated to foster trust. New methods include a composer submenu and specific syntax: adding an ai attribute to details/code blocks (e.g., [details="Summary" ai]), and prefixing quotes with [!AI]. southpaw provided the GitHub PR for the implementation, which is part of Discourse core. david noted that hosted sites on the ‘latest’ channel typically receive such updates within 24 hours.

Feedback included requests for automatic AI detection, which awesomerobot and martin discussed as currently unreliable due to sophisticated AI usage. efuetatem1 suggested nuanced labeling options (e.g., “Human-written, edited by AI”), but satonotdead argued that such templates are rarely used effectively. martin clarified the goal is to prevent unlabelled dumping of AI content, suggesting that heavily edited posts may not require labeling.

per1234 identified several issues: the composer hardcoded javascript as the language for AI code blocks, ignoring forum defaults; BBCode quote tags lacked AI support; and the cursor placement after inserting AI blocks was inconvenient compared to standard blocks. martin fixed the language identifier issue via PR 46430 and added BBCode support via PR 46439. Tris20 proposed a

We are living in a world now where forum members may have the need to share content in their posts that is partially or wholly AI generated content.

We already had an AI indicator that the AI plugin used for quotes, but it’s time we expanded this capability and put this labelling in the hands of regular members, it’s an important distinction to make to increase trust between people in online communities.

Code, quotes, and details blocks can be labelled as AI content. There are a few different ways you can now label AI content in posts. The easiest way is to use this new composer submenu that will insert a placeholder with the AI indicator already applied:

Details

Simply add an ai attribute, which works with and without the details summary:

[details="Summary here" ai]
This is AI content in a details block
[/details]
Summary hereAI

This is AI content in a details block

Quotes

This one is a little odd, but still simple. Add [!AI] to the start of the quote block:

> [!AI]
> This is a quote made by AI

This is a quote made by AI

Code

Similar to details, you can add an ai attribute, with or without the language specified:

```javascript ai
console.log("This code is made by AI");
```
console.log("This code is made by AI");

Feedback is welcome in this topic!

17 לייקים

Nice feature!

I would like to add this to the Discourse-based forum I manage. Can you provide information about that possibility? Is this part of the Discourse core, or implemented as a separate component?

If it is in the core, it would be great if you would provide a link to the GitHub PR or commit where it was implemented. That would allow me to more easily identify when my forum is updated to the Discourse version that has the feature.

4 לייקים

Hi @per1234,

Here’s the PR:

6 לייקים

How do hosted sites get this? Thanks

3 לייקים

For hosted sites on the ‘latest’ channel, changes typically go out within 24h. But we can expedite it on a case-by-case basis. (I’ll drop you a PM about it now)

3 לייקים

This is nice! However, at least in my forum, what I see is that some users create posts by copypasting walls of text from their AI agent’s output. I see this is as a low-effort practice, and I assume these are “low-effort users”, so I would guess they’re the least likely to go out of their way to tag this AI content text.

In a case like this, a feature that I would find more interesting would be automatic labeling of AI content. I guess this could be done today with some custom workflow, and I understand that this might be outside of your scope, but yeah, these are just my 2 cents. Thanks for this feature though, it’s nice to have it regardless!

8 לייקים

This would be great and many people would appreciate it, but I don’t think anyone’s able to reliably detect it at this point. Some providers like Anthropic are working on APIs to allow detection of their own watermarks, and Google has one for media (SynthID)… so maybe it will be at least partially possible soon?

3 לייקים

This is interesting, but I don’t know if people use AI content that way.

Personally, I usually write the post first, then ask AI to proofread it, or sometimes give it basic ideas or guidelines, then refine its output with a few back-and-forths between the AI and me until the final version.

Did you consider an option where the user can manually identify their post with options such as:

  • Human-written, edited by AI
  • AI-written, edited by human
  • AI written
  • Etc.

Human-written, edited by AI :blush:

לייק 1

We tried a template years ago in our community and it was very similar to this one you posted. Almost nobody used it.

Algorithms can not prevent algorithms, it is a philosophical contradiction. I think about all of it like Matrix descriptions: agents looks like people and they are everywhere. One can not stop them but stop feeding them.

Isolation is the only way, and it is not relatable to Discourse. Hardening the signups and being really creative about what we ask about, can improve the overall experience but there is no way to prevent them from the scratch. They are on the money and money decides how is Internet today.

This is interesting, kind of another level up in the ladder to identify the whole post as AI assisted. The change in the OP was mainly done to avoid people dumping giant screeds of AI content without properly labelling it.

I would posit that if you have written it yourself and edited with AI or vice-versa, depending on the degree of your involvement labelling may not even be required.

For communities, I think if people are posting purely AI generated posts, and people are tolerating or even apathetic about it, you have a big problem with the health of your community.

Yes would be nice but it’s a tricky problem…as humans, how easy it is to detect it depends on how well someone uses AI to write. If they leave all the claudisms in there, you can tell instantly, but if someone has actually used AI really well then it’s very hard to tell as a human let alone using AI to detect it.

לייק 1

I see that the “post menu > Add AI Content… > Code” feature adds the javascript language identifier to the inserted code fences:

Having this language identifier hardcoded is not very friendly to forums where code blocks are used for things other than JavaScript code.

Please remove the hardcoded language identifier, so that the feature instead adds code fences without a language identifier:

``` ai
// AI content
```

This will allow each forum’s Default code lang site setting to apply to the code block (and of course the user can still specify a different language if needed).

לייק 1

Oh my bad, sorry! Will fix now.

Edit: UX: Generic AI code content placeholder - Pull Request #46430 - discourse/discourse - GitHub

לייק 1

Thanks @martin!


Some additional feedback: it would be nice if support was added for labeling via BBCode quote tags, in addition to the current support for labeling quotes using Markdown.

If the convention established for the details tag was followed, it would look something like this:

[quote ai]
This is a quote made by AI
[/quote]

I love Markdown, but I do end up using BBCode quote tags for larger content because I find it faster and easier than adding the > to the start of each line (I always use the Markdown editor and manually type markup).

3 לייקים

Didn’t even realise this existed, this would have been handy to know :laughing: Will add it!

לייק 1
2 לייקים

I wonder if we could prompt the user at the point of pasting, and perform a few checks so the prompt only appears in particular cases? If that were a feature it could also encourage better source sharing in general, such as “what website/document did you get this from”?

e.g. Is the user pasting more than 100 characters? If yes, modal window: “What is the source for this content?”

You can play with this mock up here: Claude Artifact


Edit: There is now a Feature Topic for this idea to make it bit easier to discuss should there be interest: Source/Citation Needed Modal for Pasted Content

2 לייקים

There is a UX regression as compared to the equivalent standard features without AI labeling. When using the standard features, the block content is selected in the inserted content. This is convenient for the standard workflow:

  1. Add block formatting.
  2. Add content into block.

When the content is added via the “Add AI content..” menu item, the editor cursor is instead placed after the block. This adds an extra step to the workflow:

  1. Add block formatting.
  2. Select block.
  3. Add content into block.

On my forum, users frequently struggle to correctly use code blocks, so it is important to make it as easy to use as possible.


In case my text description above isn’t clear, I’ll provide some illustrations. Here you can see the behavior of the standard code block feature:

Screencast of code block usage

and here the behavior of the AI-labeled code block feature:

Screencast of AI code block usage


The same applies to the quote and details features.

לייק 1