Please forgive - I’m not familiar with the abilities of the Discourse AI yet. I have been wanting to consider this feature for content moderation, but have not yet looked deeply into its positioning or abilities.
We have an extensive list of prohibited language but AI moderation as a front-end to this would eliminate a ton of regex and keywords.
I don’t know if the Discourse AI can pre-process content, to trigger a function where we can determine what to do based on findings. Consider the moderation features in Discourse for blocking terms versus holding content for moderation.
I would like to get to a place where each of us can use the tools available to fine-tune how we manage these concepts:
- What is acceptable content?
- What happens when specific kinds of content are detected? (Trigger functions)
- What API functions can we use to react to specific kinds of content triggers? Examples include: Warn user, or block user, or block based on number of warnings, or forward to moderator, or put in moderation queue, or combinations of these actions.
As a developer who is familiar with the current state of LLMs and related tooling, I lack two things related to all of this:
- I lack a firm understanding of exactly where the Discourse AI fits, the specific model being used, and what coding/API options we have available for integration and management.
- I lack an understanding of API options available that would allow someone like me to code fine-tuned solutions, rather than relying on the fine team here to consider and provide solutions that would hopefully be acceptable to the widest possible audience.
So “How do you use Discourse AI?” Well, this is how I’d like to use it. As to “make it even better”, I’m hoping that detailed documentation and an API will allow for site-level configurations that will eliminate a lot of team development here. That will allow the team to focus on things that we on the outside cannot do. I consider that a strong win-win for everyone.