Hello everyone,
I’m new to Discourse and on a bit of a mission to find out more about the Sentiment Analysis feature, as I’m looking at recommending it to the client that I’m working with.
I just wanted to see if anyone was currently using the plug-in and if they had any feedback on it!
Any advice or information would be much appreciated!
Can you describe what problem you are trying to solve with some concrete examples?
Hi @sam
There’s no specific problem as such, so I’m not sure how to answer this, but I’ll try!
I am a Community Manager for a client who uses Discourse. I have no prior experience of Discourse and have joined this community to try and learn more about it, as I am interested in how plug-ins could enhance the user experience.
My client has a specific interest in sentiment tracking, and I would like to know more about other people’s experiences with the Sentiment Analysis tracking feature which is part of the Discourse AI plug-in.
Apologies if I’m not explaining myself well, I’m neurodivergent and explaining concepts I’m unfamiliar with can be a little challenging!
hi @sukinova, welcome to the community!
It’s a pretty new feature and I don’t have a specific opinion to share myself right now. In general though you can find related topics tagged with ai-sentiment.
You could also watch first posts on this tag and be notified when someone posts a new topic about it, like this:
Can you try digging in a bit. Try to get some concrete examples.
Of the many AI features we ship this is probably one of the least used. In theory it feels like it could be helpful but in practice we are not seeing too may practical uses for it. You would need a lot of volume for it to potentially be useful.
Hey @sukinova, welcome to Discourse!
Currently, the sentiment analysis in Discourse is very simple and straightforward.
Every post on Discourse is classified on sentiment
We offer a sentiment over time visualization on the dashboard
Here is how it looks:
This year we plan on expanding that, to allow community managers to drill down and see sentiment per category, tag, topic, etc. in our reports page.
If you have any requirements not covered by those I’d be eager to hear!
I love the concept but I’m skeptical of the current state. For the Fedora forum it reports a slight skew overall to negative, which I do not think is reality. And… I’m definitely sure we have emotions other than overwhelming sadness and surprise.
I haven’t tried out the Discourse feature yet, but I do have experience with sentiment analysis generally. For instance, from a post I wrote about comments on Interpersonal Skills—Stack Exchange:
There are some fairly negative words that are upvoted: awful, arrogant, unsafe, creepy and trash. Now those almost certainly describe the content of the post they are attached to. Plenty of people have creepy ideas when it comes to human relationships. Unsurprisingly, this means that higher scored comments have lower sentiment scores on Interpersonal Skills:
score n sentiment 0 39532 -0.085 1 13925 -0.081 2 7077 -0.133 3 4171 -0.152 4+ 11278 -0.182
I also noted that moderators were more likely to delete comments with negative sentiment scores. This isn’t because moderators had access to the analysis but just because humans notice patterns intuitively. Sentiment analysis in this case was helpful for understanding some of the dynamics of the community and its moderation, but I didn’t see much reason to build it into the software.[1]
Similarly, I built an engagement model that included comment sentiment to predict whether a user would ask a second question on Stack Overflow. In that model comment sentiment was not significant. Of course that’s one model and not even for a Discourse community. But it is a data point that suggests people aren’t as repelled by negativity (as defined by the sentiment analysis system I picked) as you might assume.
There are a lot of reasons for optimism about this sort of artificial intelligence, but I’d caution against letting enthusiasm overtake us. Misapplied sentiment analysis can potentially lead decision makers into fundamental mistakes. Remember Chesterton’s fence!
Not to say doing so is invalid! Rather the advantages seem to be less direct than one would imagine. ↩︎
