The discussion evaluates the utility of Discourse’s sentiment analysis feature. nickdb initially questioned its value, finding the data ambiguous and lacking insight. In response, Falco noted that new capabilities allow using LLMs or Jev-style decision models for more accurate classification, though at a higher cost.
satonotdead shared their experience with an earlier “emotions” dashboard featuring emojis, which they found more intuitive and useful than the current technical display. They suggested that merging these approaches could improve adoption, as the current interface is too “dry” for general interpretation. However, nickdb tested the agent-based LLM approach and decided against it due to excessive token consumption. Finally, sam clarified that while LLMs are overkill for this task, decision models offer a much cheaper alternative.
Evening all, or possibly morning/afternoon.
I turned on sentiment analysis quite some time back, but to me it just seems to not add anything of value. The data is just lacking or at best ambiguous.
Wondering how others are using it or interpreting its value?
We recently made it possible to use an LLM instead to do the classification, and @sam is adding the capacity to use Jev style decision models to do sentiment classification.
It will be more costly, but with greater accuracy.
Great, I must have missed this, would you be so kind as to send a pointer? I don’t mind burning a few extra tokens if it can provide something more valuable.
I think you are looking this: Classification in the Discourse Sentiment Dashboard - #11 by fzngagan
I wonder if it’s because my self-hosted model isn’t configured correctly or because my memory is failing, but I remember seeing emojis (emotions) in the admin dashboard that later disappeared from my instance.
That analysis was more precise and also useful, because everyone can interpret it. I think the traditional sentiment technical display, is an unconscious barrier to its adoption by humans.
I mean, the functionality may indeed add value, but then we fail at their interpretation because it’s too “dry”. Merging both analyses, I believe, would give it much more room to play with.
And perhaps adding some closer implementations, without the need to go to the admin dashboard or using custom queries.
I have a lot left to investigate and learn on this. My initial question remains open: is there today in Discourse an emotions dashboard, or did I dream it?
I did a brief agent-based test. It seriously eats tokens. Will just call it quits for now, so many better uses of tokens that will provide meaningful benefits.
Decision models make it very cheap, agree using llms for this is way overkill.