What's next for Toxicity detection in Discourse AI

Announcements
, ,
1

This image features a robot in a scenic outdoor setting, holding a sign that reads "RIP Toxicity." (Captioned by AI)
This image features a robot in a scenic outdoor setting, holding a sign that reads "RIP Toxicity." (Captioned by AI)1024×1024 324 KB

Today we are announcing good bye to the Discourse AI - Toxicity module :wave: in favor of Discourse AI Post Classifier - Automation rules, leveraging the power of Large Languge Models (LLMs) to provide a superior experience.

:information_source: This will be a beta experience so expect changes to occur to the feature.

Why are we doing this?

Previously using the Toxicity module meant…

  • You were stuck using a single pre-defined model
  • No customization for your community-specific needs
  • Confusing threshold metrics
  • Subpar performance

LLMs have come a long way and can now provide a better performing and customizable experience.

Whats new?

Discourse AI Post Classifier - Automation rule can be used to triage posts for Toxicity (amongst other ways) and enforce communities to specific code of conducts. This means…

  • Multiple LLMs supported for different performance requirements
  • Easy to define what and how content should be treated
  • Customizable prompts for community-specific needs
  • Flag content for review

and much more.

To assist with the transition we have already written out 2 guides

What happens to Toxicity?

This announcement should be considered very early, until we are ready to decommission you can continue to use Toxicity. When we do it, we will be decommissioning the module and removing all code from the Discourse AI plugin and associated services from our servers.

6 Likes
2

Is there a cut-off date for the Toxicity module?

3 Likes
3

We haven’t picked a specific date but are currently working on getting this done soon

3 Likes
4

This is extremely relevant to an article I’m writing for Community Leaders Institute right now, to my Ph.D on toxicity, and more. Would you all be willing to do a behind-the-scenes interview on this system for my YouTube channel as part of my series on inoculating online communities from toxicity? It is also doubling as an academic endeavor for my Ph.D.

You up for that?

2 Likes
6

Thank you for thinking about Discourse for your YouTube channel! Could you please email more details to mae@discourse.org about what exactly the behind the scenes interview would entail and what’s needed from our end?

2 Likes
7

FYI about the following change

2 Likes
8

Thanks Mae! I’ve followed up!

1 Like
9

As a heads-up, we are now hiding site settings for enabling/disabling Toxicity and NSFW. This is our ongoing effort as we continue to deprecate the features.

If you have these features turned on, it will still operate as normal. We have NOT fully deprecated the features yet.

If you have it turned off and would like to turn it on, you will now be unable to do so.

1 Like