As suggested by @erlend_sh, I have created a development log for a plugin I’ve begun developing for Discourse.
I have started work on building what I call the pre-emptive striker plugin (if anyone has a better suggestion for the name, throw it at me!). It was proposed at the beginning of the summer and I’m now officially developing the plugin. Link to original proposal is here.
Basically, the plugin checks on the user as they are writing for toxicity using Google’s perspective API. and send JIT notifications if it detects particularly toxic language.
I would like some help, however, in understanding how this plugin will interact with the rest of Discourse. I created a quick Draw.io to illustrate the architecture of the plugin as I understand it in the context of Discourse. Discourse is pretty big to dive into as my first ruby app. I’ve gone through all of the Discourse beginner guides to familiarize my self a bit… Link to the file Please download and make edits or point out anything that looks wrong! This is a fairly high-level diagram.
basic user flow goes like this:
- user begins composing.
- They type “You’re a dumb person for making that comment, why do you even exist?”
- As they type, Perspective API is sent requests to analyze the comments. It will return a high toxicity score.
- The high toxicity score is over a threshold (probably a threshold set in the plugin setting admin panel?) and thus triggers a JIT notification.
- The user sees the warning, but ignores it and posts anyways! (or they reflect on what they just wrote and proceed to cry because they’ve realized what a terrible person they are…)
- If they post it, they will be automatically flagged for moderation follow-up.
- A moderator will review the post and do whatever moderators do best.
Is there documentation on the API’s for JIT notifications and moderation flagging?
Any pointers/guidance there would be greatly appreciated and would speed development up.
I wrote a quick node app that interfaces with PAPI. Next steps are to begin writing it in ruby.
The plugin repo is here for anyone who wants to have a look. It’s pretty bare bones at the moment.