Classification in the Discourse Sentiment Dashboard

[Lauraruskovic]

Hi everyone,

I’m exploring the sentiment analysis feature available in the Discourse admin dashboard (Admin > Dashboard > Sentiment) and I have questions about how it works:

1. How is sentiment assigned to posts? Is it based on a machine learning model, keyword matching, or a combination of both?
2. Are there specific keywords or phrases that trigger a particular sentiment? If so, is there a way to view, edit or export the full list?

Thanks in advance.
Laura

[NateDhaliwal]

Sentiment is part of the ai plugin. You can read more about it here:

Discourse AI - Sentiment

This topic covers the configuration of the Sentiment feature of the Discourse AI plugin. Required user level: Moderator Sentiment keeps tabs on your community by analyzing posts and providing sentiment and emotional scores to give you an overall sense of your community for any period of time. These insights can be helpful in determining the type of users posting within your community and interacting with one another. Features Overall sentiment: compa…

[Lauraruskovic]

Thank you. Is there a way to adjust this?

We’ve noticed that sentiment is only assigned to a small number of posts. Since we find this feature very helpful, we were wondering whether there is a way to expand or configure the sentiment classification so that it applies to more posts.

Alternatively, is it possible to have sentiment determined by users (for example, members of our community) rather than by AI? If that’s not currently supported, would there be a way for us to manually assign or manage sentiment classifications ourselves?

[NateDhaliwal]

Backfill for older postsare controlled by this setting:

@Discourse

Once enabled, Sentiment will automatically classify all new posts and backfill posts from the last 60 days via a scheduled job that runs every 5 minutes. To backfill posts older than 60 days, increase the ai_sentiment_backfill_post_max_age_days site setting.

As for customizing this… the agent used for Sentiment can be found in the AI > Agents tab, under the name ‘Sentiment classifier’.

You can also edit Sentiment-related settings at /admin/plugins/discourse-ai/settings?filter=sentiment.

[Lauraruskovic]

Thank you for pointing me in the right direction! I was able to find the Sentiment classifier agent. However, I’m unable to edit its prompt, even though I have admin access. The fields appear to be locked or read-only.

Is there a specific permission or configuration that needs to be enabled to allow editing of this built-in agent? Or would the recommended approach be to create a custom agent to replace or supplement the default Sentiment classifier?

Any guidance on this would be greatly appreciated!

[NateDhaliwal]

Hmm… if I am not wrong, you can’t edit them (yet?). You could self-host a custom LLM if you’re very eager to tweak it though.

[Lauraruskovic]

Yes, I’d like to customize it. What do you mean by LLM? Do you mean creating my own agent?

[NateDhaliwal]

I’m not as familiar with this, but you can check this out:

Self-Hosting Sentiment and Emotion for DiscourseAI

The Discourse AI plugin has support for requesting emotion/sentiment classification of new posts, which is stored in the database and can be used in reports and admin dashboards. Discourse AI supports two types of classification, each requiring its own model: Sentiment — classifies posts as positive, negative, or neutral (using cardiffnlp/twitter-roberta-base-sentiment-latest) Emotion — classifies posts across 28 emotion labels like joy, anger, surprise, etc. (using SamLowe/roberta-base-go_em…

Though I am not too sure to what extent you can customize the prompts. It’s fixed a for a reason, as they are quite fine-tuned for better accuracy, if I am not mistaken.