Prompting users to avoid harmful language

AI-generated summary

A discussion on prompting users to avoid harmful language in Discourse has been initiated. dfabulich referenced Twitter’s improved reply prompts and wondered if Discourse has a similar feature. awesomerobot mentioned that a Google Perspective API plugin was experimented with, but it struggled with nuanced language and false positives.

codinghorror suggested using the “Watched Words” feature, which allows admins to block, censor, or require approval for certain words. However, dfabulich thought a “prompt” or “nag” option would be more effective, as it would allow users to ignore the prompt if they intended to use the word appropriately.

codinghorror proposed a “warn” option that would pop up a dialog when a target phrase or regex is detected in a post. dfabulich agreed and suggested making the dialog configurable. codinghorror noted that all text in Discourse is editable, but unique text per word might be too much.

dfabulich suggested explaining why a word is suspect, possibly by linking to a page that explains the issue. eviltrout liked the idea, but noted that implementing it would require medium to large effort, particularly in handling multiple warned words at once.

Does Discourse have anything like this? It would be cool if it did.

We experimented with Discourse Google Perspective API, but ran into similar issues as described in the article you linked… it struggled on more nuanced language, and would sometimes pick up on some non-offensive language as offensive… and these false positives can be offensive in their own right!

However, Twitter’s early tests ran into some problems. It found its systems and algorithms sometimes struggled to understand the nuance that occurs in many conversations. For example, it couldn’t always differentiate between offensive replies and sarcasm or, sometimes, even friendly banter. It also struggled to account for those situations in which language is being reclaimed by underrepresented communities, and then used in nonharmful ways.

9 Likes

Not prompt, but you can certainly try

Admin → Customize → Watched Words

From there you have

  • block
  • censor
  • require approval
  • flag
  • link
  • replace
  • silence

which is very flexible, use to your tastes. I suppose we could add a “prompt” or “nag” option where it’d say

:loudspeaker: Are you sure you want to use the word “moist” in your post? This word has been known to cause problems in our community, so we urge you to consider alternatives such as “damp”.

2 Likes

I absolutely think a “prompt”/“nag” option would do a lot of good. The nice thing about it is that even if it misses context, users can just ignore the prompt and use the word appropriately, and if users get suspended, they can’t say “I had no idea!”

@sam @eviltrout how do you feel about a “warn” option here? basically it’s just popping a dialog if the target phrase/regex appears in the post when you press the submit button. Literally this:

:loudspeaker: Are you sure you want to use the word “moist” in your post? This word has been known to cause problems in our community, so we urge you to consider alternatives.

3 Likes

It’d be cool if we could configure the dialog, too, though I think this would be a great start.

All text is editable in Discourse already, via Admin, Customize, Text. You would not get unique text per word though, that feels like a bit much..

1 Like

I just think it might be worth saying why the word in question is suspect. (But maybe I’d just link all prompted words to the same page that would explain it…?)

1 Like

I like it. We might have some issues with them using multiple warned words at once, and we have to add the whole new type of watched word. It’s somewhere between medium and large effort, probably closer to medium.

2 Likes