Community Sentiment and Toxicity queries

[SaraDev (Sara Devlaeminck)]

For those wondering how to view the classification results database for the Community Sentiment and Toxicity modules, this can be done using the Data Explorer plugin, and the classification_results table.

This is useful for seeing how the AI plugin is functioning on your site and classifying posts.

AI Sentiment

SELECT target_id as post_id,
model_used,
classification->'negative' as negative,
classification->'neutral' as neutral,
classification->'positive' as positive
from classification_results
WHERE model_used = 'sentiment'
order by id desc

AI Emotion:

SELECT target_id as post_id,
model_used,
classification->'neutral' as neutral,
classification->'sadness' as sadness,
classification->'surprise' as surprise,
classification->'fear' as fear,
classification->'anger' as anger,
classification->'joy' as joy,
classification->'disgust' as disgust
from classification_results
WHERE model_used = 'emotion'
order by id desc

AI Toxicity:

SELECT target_id as post_id,
classification->'toxicity' as toxicity,
classification->'severe_toxicity' as severe_toxicity,
classification->'obscene' as obscene,
classification->'identity_attack' as identity_attack,
classification->'insult' as insult,
classification->'threat' as threat,
classification->'sexual_explicit' as sexual_explicit
From classification_results
WHERE classification_type = 'toxicity'
order by id desc

[Samantha_O]

Is there a way to add a filter to this query to get the posts from a particular thread? Right now, it seems it is getting posts across the entire Community, which is nice, but also cumbersome to find a particular post/thread you are looking for, not to mention you will run into the row limit for this.

[Lilly]

Hi @Samantha_O

Does this work for specific topics? you will have to specify the topic ID in the params.

-- [params]
-- int :topic_id = 

SELECT cr.target_id as post_id,
cr.model_used,
cr.classification->'negative' as negative,
cr.classification->'neutral' as neutral,
cr.classification->'positive' as positive
FROM classification_results cr
JOIN posts p ON p.id = cr.target_id
WHERE cr.model_used = 'sentiment'
AND p.topic_id = :topic_id
ORDER BY cr.id DESC

[JammyDodger]

-- [params]
-- topic_id :topic_id

SELECT 
    cr.target_id as post_id,
    cr.model_used,
    cr.classification->'negative' as negative,
    cr.classification->'neutral' as neutral,
    cr.classification->'positive' as positive
FROM classification_results cr
  JOIN posts p ON p.id = cr.target_id
WHERE cr.model_used = 'sentiment'
  AND p.topic_id = :topic_id
ORDER BY p.id

Ahh you pipped me to it.

But here’s one for a specific post as well just to not feel like a wasted reply:

-- [params]
-- topic_id :topic_id
-- int :post_number

SELECT 
    cr.target_id as post_id,
    cr.model_used,
    cr.classification->'negative' as negative,
    cr.classification->'neutral' as neutral,
    cr.classification->'positive' as positive
FROM classification_results cr
  JOIN posts p ON p.id = cr.target_id
WHERE cr.model_used = 'sentiment'
  AND p.topic_id = :topic_id
  AND p.post_number = :post_number

The topic_id and post_number can both be found in the URL, so it’s quite user-friendly.

Topic_id:

https://meta.discourse.org/t/community-sentiment-and-toxicity-queries/275741/4

Post_number:

https://meta.discourse.org/t/community-sentiment-and-toxicity-queries/275741/4

[Lilly]

I think this should target category id:

-- [params]
-- int :category_id = 

SELECT cr.target_id as post_id,
cr.model_used,
cr.classification->'negative' as negative,
cr.classification->'neutral' as neutral,
cr.classification->'positive' as positive
FROM classification_results cr
JOIN posts p ON p.id = cr.target_id
JOIN topics t ON t.id = p.topic_id
WHERE cr.model_used = 'sentiment'
AND t.category_id = :category_id
ORDER BY cr.id DESC

[Samantha_O]

Thank you both! When I run these, it doesn’t find older topics/topic ids. I wonder - does the sentiment analysis only happen on new posts?

edited for clarity

[Lilly]

@JammyDodger would know better, but yes I think the sentiment analysis works on new posts. I think in order to get it to use the old posts would likely require some rails commands to process them through the sentiment model, and into the classification_results table (where the new posts’ analyses results are).

[JammyDodger]

It works on new posts once it’s turned on. I’m not sure if there is a backfill option for it?

[Falco]

Only new posts from now. We can consider backfilling that information after we start reporting on the data and we validate it.

[Samantha_O]

Would one of you (or another brilliant query-ist) be able to help me fine-tune this a bit? I would like to have params for dates and those dates would correspond to the post date. The end goal here would be to be able to look at changes over time while waiting to see if the dashboard can do that in the future

[SaraDev (Sara Devlaeminck)]

Hi Samantha,

If you wanted to refine those queries to include start and end date parameters, you could use a query like this:

-- [params]
-- date :start_date = 2023-01-01
-- date :end_date = 2024-01-01

SELECT 
p.created_at,
cr.target_id as post_id,
cr.model_used,
cr.classification->'negative' as negative,
cr.classification->'neutral' as neutral,
cr.classification->'positive' as positive
FROM classification_results cr
JOIN posts p ON p.id = cr.target_id
WHERE cr.model_used = 'sentiment'
AND p.created_at BETWEEN :start_date AND :end_date
ORDER BY cr.id DESC

Example Results:

created_at	post	model_used	negative	neutral	positive
2023-11-08T21:21:23.913Z	post_id	sentiment	58	38	2

The important part added here is joining the posts table with the classification_results table so that we can add:

AND p.created_at BETWEEN :start_date AND :end_date

To the WHERE statement to filter the posts by their created_at date.

Since the emotion and toxicity classification queries are similar, you could modify those queries with this addition to include start and end date parameters as well.

[Lilly]

Dashboard Report - Overall Sentiment

This is an SQL version of the Dashboard Report for Overall Sentiment. This report requires the Discourse AI plugin and Sentiment Analysis to be enabled. This dashboard report compares the number of posts classified either positive or negative with the “Sentiment” AI, over a specified period. By leveraging sentiment analysis, the report categorizes posts as either positive or negative based on a defined threshold, which can be adjusted. This categorization is then used to calcula…

[SaraDev (Sara Devlaeminck)]

Adding this one here as well:

Dashboard Report - Post Emotion

This is an SQL version of the Dashboard Report for Post Emotion. This report requires the Discourse AI plugin and Sentiment Analysis to be enabled. This dashboard report shows the number of posts classified with one of following emotions, group by poster trust level, within a specified date range: Sadness Surprise Fear Anger Joy Disgust -- [params] -- date :start_date = 2024-01-16 -- date :end_date = 2024-02-16 -- int :threshold = 30 SELECT u.trust_level AS trust_level, …

[sonpage (Page D)]

I tried using this query (our sentiment setting is set up) but it’s not returning any results. Anything I should try to troubleshoot?

[Falco]

I’ve updated the guides at Dashboard Report - Overall Sentiment and Dashboard Report - Post Emotion, can you try again with the updated queries?