Discourse metrics are already pretty good, however, to effectively understand the health of your community or maybe certain topics, it would be great to have some kind of built-in metric for this or maybe a plugin that would monitor certain boards or discussions within a Discourse instance.
Sentiment calculation is easy to do with an excel spreadsheet formula or maybe an open source API.
We do have a plugin the uses Google’s Perspective API, which looks at individual posts/users
Topic-level sentiment seems like an interesting angle though, or even on category/tag level. That could potentially help someone better allocate moderation resources on a very large forum (if for example, the feature category had bad overall sentiment, assign more moderators there)?
You’ve pinpointed better what I wanted to say. I’m looking at the same angle, a tool for moderators or staff, to monitor at a glance the sentiment level from a topic and take preemptive measures by driving the discussion in such a way that the conversation sentiment goes to positive.
I will also look at the Google Perspective API Plugin.
Additionally, it could be useful if a topic sentiment analysis plugin could have the option of taking an automatic action if sentiment started dropping… for example, closing the topic for 24 hours to let everyone cool off (I’d just caution that the weight of sentiment should be spread across participants so an individual couldn’t game the system to force a topic to auto-close).
I’m more interested in this the more I think about it. One of the weaknesses we found in analyzing sentiment of individual posts was that the Perspective API had no idea when someone was joking or being sarcastic… if the sentiment of replies is being measured at the same time, that would potentially be a stronger signal (because if someone is actually being abusive, the sentiment in the replies will more likely be bad too!)
I’m curious if the benefits outweigh the potential costs too, in another topic this week we saw that the translate API racked up a $1000 bill for one site through language detection alone.
If your site is so large that you rely on a topic indicator for sentiment, the potential costs in generating that little traffic light could be considerable.
Google does structure the pricing quite differently to Translate. Sentiment analysis in the NLAPI is $1 for 1000 units of up to 1000 characters, so each reply will be a minimum of one unit. As each reply would need to be scored separately, every response is one unit minimum. At a very rough guesstimate it’s maybe 1/8th the cost of the translate API, but that’s still a fairly big number to substitute staff who interact with the posts bring written.