Hi everyone! I’m a machine learning PhD and I’ve been thinking about building a tool to let anyone train a custom AI to help with content moderation on the discourse forums. Is this something that people here would want? I know that there is the Google perspective API but its very generic. My idea is that you’d have a plugin where you can moderate posts and it automatically trains a text classifier to learn from your moderation. For example, if you mark a post as having toxic language or being off-topic it learns from you and can flag similar posts or even pre-moderate them. What do you think?

[תמונה] Razcle: What do you think? Text classifiers (used in forums, for example) have been around for a long time. Typically this type of text classification and scoring is performed with Bayesian classifiers. If you do a Google search with keywords: bayesian classifier ruby and /or bayesian classifier javascript You will find myriad libraries and examples of text classification using Bayesian classifiers. We have used various Bayesian classifiers for forum post moderation, spam detection, and more over the years; and have implemented custom code to train the classifier when moderators perform mod actions. Hope this helps.

One potential issue here is that many moderation tasks aren’t a simple punishment, but are more complex such as “close the topic for 12 hours” or “this needs to be a wiki post” or “this needs to be moved to a different category”. Good luck!

סינון תוכן מותאם אישית באמצעות בינה מלאכותית

מפתח

Terrapop 9 באוקטובר,‏ 2020,‏ 3:34pm 2

This or auto-categorization of uncategorized topics based on a custom trained model would be nice, too.

נושא		תגובות	צפיות
AI Forum Moderation: Seeking Insights and Experiences Development ai	8	2236	27 בספטמבר,‏ 2025
AI integration for moderation Product Support	2	293	25 בינואר,‏ 2026
Discourse AI to make spam filter smarter? Feature completed , ai	1	573	18 במאי,‏ 2024
Pre-emptively warning a contributor about the toxicity of their post Feature	18	3782	27 בספטמבר,‏ 2017
Experiments with AI based moderation on Discourse Meta Community Building moderation , ai	11	1846	26 במאי,‏ 2025

סינון תוכן מותאם אישית באמצעות בינה מלאכותית

נושאים קשורים