# Discourse Disorder
**URL:** https://meta.discourse.org/t/discourse-disorder/246587
**Category:** Plugin
**Tags:** deprecated
**Created:** [December 16, 2022, 9:10pm UTC](https://meta.discourse.org/t/discourse-disorder/246587 "2022-12-16T21:10:01Z")
**Posts on this page:** 20
**Page:** 2
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 1, 2023, 12:45pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/25 "2023-03-01T12:45:10Z")
I have also modified your query to display scoring in a more convenient way using [Data Explorer](https://meta.discourse.org/t/32566?silent=true).
Credits go to ChatGPT and [PostgreSQL clues by Leonardo](https://meta.discourse.org/t/discourse-data-explorer/32566/263):
```sql
SELECT
json_extract_path_text(pcf.value::json, 'classification', 'toxicity') AS toxicity,
json_extract_path_text(pcf.value::json, 'classification', 'severe_toxicity') AS severe_toxicity,
json_extract_path_text(pcf.value::json, 'classification', 'obscene') AS obscene,
json_extract_path_text(pcf.value::json, 'classification', 'identity_attack') AS identity_attack,
json_extract_path_text(pcf.value::json, 'classification', 'insult') AS insult,
json_extract_path_text(pcf.value::json, 'classification', 'threat') AS threat,
json_extract_path_text(pcf.value::json, 'classification', 'sexual_explicit') AS sexual_explicit,
json_extract_path_text(pcf.value::json, 'model') AS model,
pcf.created_at,
p.raw
FROM
post_custom_fields AS pcf
INNER JOIN
posts AS p ON p.id = pcf.post_id
INNER JOIN
topics AS t ON t.id = p.topic_id
WHERE
pcf.name = 'disorder'
AND t.archetype = 'regular'
ORDER BY created_at DESC
```
> **And this modification will return those rows, where any of classification values is bigger than 50 (or whatever you set)**
>
> ```sql
> -- [params]
> -- int :threshold = 50
> SELECT DISTINCT ON (p.id, pcf.created_at)
> json_extract_path_text(pcf.value::json, 'classification', 'toxicity') AS toxicity,
> json_extract_path_text(pcf.value::json, 'classification', 'severe_toxicity') AS severe_toxicity,
> json_extract_path_text(pcf.value::json, 'classification', 'obscene') AS obscene,
> json_extract_path_text(pcf.value::json, 'classification', 'identity_attack') AS identity_attack,
> json_extract_path_text(pcf.value::json, 'classification', 'insult') AS insult,
> json_extract_path_text(pcf.value::json, 'classification', 'threat') AS threat,
> json_extract_path_text(pcf.value::json, 'classification', 'sexual_explicit') AS sexual_explicit,
> json_extract_path_text(pcf.value::json, 'model') AS model,
> p.id as post_id,
> pcf.created_at,
> p.raw
> FROM
> post_custom_fields AS pcf
> INNER JOIN
> posts AS p ON p.id = pcf.post_id
> INNER JOIN
> topics AS t ON t.id = p.topic_id
> WHERE
> pcf.name = 'disorder'
> AND t.archetype = 'regular'
> GROUP BY p.id, pcf.value, pcf.created_at
> HAVING
> CAST(json_extract_path_text(pcf.value::json, 'classification', 'toxicity') AS FLOAT) > :threshold
> OR CAST(json_extract_path_text(pcf.value::json, 'classification', 'severe_toxicity') AS FLOAT) > :threshold
> OR CAST(json_extract_path_text(pcf.value::json, 'classification', 'obscene') AS FLOAT) > :threshold
> OR CAST(json_extract_path_text(pcf.value::json, 'classification', 'identity_attack') AS FLOAT) > :threshold
> OR CAST(json_extract_path_text(pcf.value::json, 'classification', 'insult') AS FLOAT) > :threshold
> OR CAST(json_extract_path_text(pcf.value::json, 'classification', 'threat') AS FLOAT) > :threshold
> OR CAST(json_extract_path_text(pcf.value::json, 'classification', 'sexual_explicit') AS FLOAT) > :threshold
> ORDER BY pcf.created_at DESC, p.id
>
> ```
>
> You can also modify it by introducing several more parameters to be able to set different thresholds to report on using [Data explorer](https://meta.discourse.org/t/32566?silent=true).
Please note: this will return Public posts only, without accessing private messages.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 1, 2023, 4:33pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/26 "2023-03-01T16:33:04Z")
> [@kinetiksoft](#):
>
> but may I suggest adding those to Reviewable Scoring table for each flagged post?
We are working on this exact feature right now!
> [@kinetiksoft](#):
>
> If needed, I can provide you with SQL Query results. We are far from finishing reviewing Flag Queue…
> We are using multilingual model and haven’t tried others. Decided it would be a good to start with considering we have some users who prefer posting using their original language.
We are also planning on using the false positive / negative rates to run an optimizer that can suggest you the best thresholds for each option, so keep that information as it will be useful in the near future.
---
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 1, 2023, 4:47pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/27 "2023-03-01T16:47:08Z")
Sounds great. Glad to hear that.
So far, I tend to decline/ignore all the flags Disorderbot makes, even having thresholds raised up to maximum of 90-100. But, due to the nature of the forum we’re testing it on (NSFW), AI is confused easily if communication is really toxic or not. As long as it is not that reliable to our use case, we will continue using it, but will use it’s reports only to “re-inforce” other reports to really toxic posts.
As soon as we find some better thresholds to use for a long-term, we will be able to enable precautionary warnings when user tries to post something really toxic.
---
### Author:  [@satonotdead](https://meta.discourse.org/u/satonotdead)
#### Post date: [March 1, 2023, 5:01pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/28 "2023-03-01T17:01:56Z")
That’s what I suspect when AI becomes mainstream. It will allow censorship and limit genuine status-quo questioning that’s neccessary for the healty of every community on the world.
Not limit or ban, educate and discuss. Perhaps there is a way to use the tools without the _side-effect (as my concerns that’s the wanted effect)_ but I see that’s not possible at the moment.
Thanks for your feedback, it has value for me. And of course, thanks to the team for keeping Discourse updated and improving like always 🙂
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 1, 2023, 5:50pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/29 "2023-03-01T17:50:28Z")
> [@kinetiksoft](#):
>
> But, due to the nature of the forum we’re testing it on (NSFW), AI is confused easily if communication is really toxic or not.
Setting all thresholds to 100 and relying only on the more extreme ones, like “severe toxicity” and “threat”, is something that I can see being adopted in communities like that.
---
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 1, 2023, 5:56pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/30 "2023-03-01T17:56:12Z")
Thanks. It is currently set like this, and is still too sensitive. I will raise some even further and see how it goes
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 1, 2023, 5:57pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/31 "2023-03-01T17:57:08Z")
Would have to see the raw classifications, but I’d increase the insult one first too.
---
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 1, 2023, 6:09pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/32 "2023-03-01T18:09:12Z")
I’d better keep you away from reading those 😃 Those may be really NSFW, even in text form
I’ve raised the first threshold to 100 too, will see how it goes now 😃
---
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 1, 2023, 7:00pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/33 "2023-03-01T19:00:40Z")
I really hope to make it possible for Disorder not to check (or not to report) on private messages in the future versions. We do not access them and feel like AI checking private conversations is highly unethical.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 1, 2023, 7:11pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/34 "2023-03-01T19:11:32Z")
Yeah, that is the same thing @davidkingham asked, we will put it in our roadmap.
---
### Author:  [@ganncamp](https://meta.discourse.org/u/ganncamp)
#### Post date: [March 1, 2023, 7:28pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/35 "2023-03-01T19:28:52Z")
> [@Falco](#):
>
> **multilingual** : A model that can classify Italian, French, Russian, Portuguese, Spanish and Turkish.
…and English? 😅
Also, I’m wondering to what degree this can replace Akismet. We’re at a 97% disagree rate on Akismet’s flags right now. It seems to simply react to posts with a lot of digits in them, so if you’re posting job logs, where every line starts with a timestamp…
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [March 1, 2023, 7:35pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/36 "2023-03-01T19:35:21Z")
> [@ganncamp](#):
>
> Also, I’m wondering to what degree this can replace Akismet. We’re at a 97% disagree rate on Akismet’s flags right now
The arms war between spam and spam detection just turned went nuclear with the advent of widely available LLMs. We are hard at work on features using a wide range of models, and while spam isn’t our priority right now, it’s something we will investigate.
---
### Author:  [@mattdm](https://meta.discourse.org/u/mattdm)
#### Post date: [March 2, 2023, 4:34pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/37 "2023-03-02T16:34:03Z")
Okay, so: I turned it on. _How do I know it’s working?_
Other than turning the thresholds down really low to catch everything, I mean.
Is there a diagnostic mode or log where I can see what a given post has scored?
---
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 2, 2023, 4:41pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/38 "2023-03-02T16:41:53Z")
The easiest way is to provoke it by posting something insulting. Make sure your user’s group is not bypassed in plugin settings.
The better way is to query [Data Explorer](https://meta.discourse.org/t/32566?silent=true). Please refer to one of my queries in this post:
> [@Discourse Disorder](https://meta.discourse.org/t/disorder-automatic-toxicity-detection-for-your-community/246587/25):
>
> I have also modified your query to display scoring in a more convenient way using [Data Explorer](https://meta.discourse.org/t/32566). Credits go to ChatGPT and [PostgreSQL clues by Leonardo](https://meta.discourse.org/t/discourse-data-explorer/32566/263): SELECT json\_extract\_path\_text(pcf.value::json, 'classification', 'toxicity') AS toxicity, json\_extract\_path\_text(pcf.value::json, 'classification', 'severe\_toxicity') AS severe\_toxicity, json\_extract\_path\_text(pcf.value::json, 'classification', 'obscene') AS obscene, json\_extract\_path\_text(pcf.value::json, 'classification', 'identity\_a…
---
### Author:  [@mattdm](https://meta.discourse.org/u/mattdm)
#### Post date: [March 2, 2023, 4:46pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/39 "2023-03-02T16:46:43Z")
Thanks. That’s returning 0s across the board for all posts so far… is that to be expected?
---
### Author:  [@kinetiksoft](https://meta.discourse.org/u/kinetiksoft)
#### Post date: [March 2, 2023, 4:48pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/40 "2023-03-02T16:48:06Z")
The majority of our posts have 0s across all the criteria too. This is normal for a forum with a healthy communication.
---
### Author:  [@mattdm](https://meta.discourse.org/u/mattdm)
#### Post date: [March 2, 2023, 4:49pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/41 "2023-03-02T16:49:41Z")
Cool — I wasn’t sure how trigger-happy the model is. 🙂
---
### Author:  [@danielabc](https://meta.discourse.org/u/danielabc)
#### Post date: [April 15, 2023, 5:08am UTC](https://meta.discourse.org/t/discourse-disorder/246587/42 "2023-04-15T05:08:49Z")
I installed the plugin, but it is not working, do I have to do some extra configuration?
---
### Author:  [@mjr4684](https://meta.discourse.org/u/mjr4684)
#### Post date: [April 20, 2023, 5:44pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/43 "2023-04-20T17:44:57Z")
I’m seeing a large number of the following errors from the plugin:
Job exception: uninitialized constant Jobs::ClassifyChatMessage::ChatMessage
The issue appears to occur when one of my plugins creates a chat message using the following command:
Chat::MessageCreator.create(chat\_channel: matching\_channel, user: message\_user, content: raw).chat\_message
Thanks
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [April 20, 2023, 6:06pm UTC](https://meta.discourse.org/t/discourse-disorder/246587/45 "2023-04-20T18:06:22Z")
Ohhh this should have broken with the new Chat reorganization. We are on the verge of launching a new plugin that will incorporate the functionality of this one in the next days, so stay tuned.
[Previous page](https://meta.discourse.org/t/discourse-disorder/246587.md?page=1)
[Next page](https://meta.discourse.org/t/discourse-disorder/246587.md?page=3)