Discourse AI - Sentiment

:bookmark: This topic covers the configuration of the Sentiment feature of the Discourse AI plugin.

:person_raising_hand: Required user level: Administrator

Sentiment keeps tabs on your community by analyzing posts and providing sentiment and emotional scores to give you an overall sense of your community for any period of time. These insights can be helpful in determining the type of users posting within your community and interacting with one another.

Features

  • Overall sentiment: compares the number of posts classified as either positive or negative
  • Bar graph showcasing toggleable numerical value for positive, negative, and overall scores
  • Emotion: number of topics and posts classified by multiple emotions, grouped by time frame
    • Today
    • Yesterday
    • Last 7 days
    • Last 30 days
  • Reports for any period of time that can be accessed via settings
    • Yearly
    • Quarterly
    • Monthly
    • Weekly
    • Custom range
  • Applicable only for admin users

Enabling Sentiment

Configuration

Sentiment is enabled by default for hosted customers. For manual steps see below

  1. Go to Admin settings-> Plugins → search or find discourse-ai and make sure its enabled
  2. Enable ai_sentiment_enabled for Sentiment Analysis
  3. Head over to /admin/dashboard/sentiment to see their respective reports

:information_source: Once enabled, Sentiment will classify all posts going forward and from the last 60 days. To classify all of your site’s historical posts, a backfill task must be run from the console.

:discourse2: Hosted by us?

Contact us at team@discourse.org and we’ll do it for you.

:mechanic: Self-hosted?

./launcher enter app
rake ai:sentiment:backfill

Technical FAQ

How is topic/post data processed? How are scores assigned?

  • Sentiment has a “per post” fidelity. For each post we are able to tell sentiment and then cut that data in many shapes (per tag / category / time etc… ). It compares the number of posts classified as either positive or negative. These are calculated when positive or negative scores > the set threshold score.

Are there any plans to add support for other languages?

  • In the future Yes! both by adding multilingual simple Machine Learning (ML) models and by using multilingual Large Language Models (LLMs) to classify the data, instead of dedicated models.

What models are used to power Sentiment?

Caveats

  • Posts classified as neutral (neither positive or negative) are not shown
  • Private messages (PMs) are excluded from calculations

Last edited by @Saif 2025-02-13T20:41:14Z

Last checked by @hugh 2024-08-06T05:27:46Z

Check documentPerform check on document:
10 Likes

A post was merged into an existing topic: Problems with Sentiment Backfill

A post was merged into an existing topic: Problems with Sentiment Backfill

The OP has been updated with a new video showcasing the updated features of Sentiment including a ton more emotions and understanding which topics/posts are associated with each emotion

I configured the sentiment model config with a model_name, endpoint, and api_key copied from the LLM settings, where it passes the test, but I get the error below in /logs.

(But maybe I don’t understand correctly, because why doesn’t sentiment use one of the configured LLMs?)

Using claude-3-5-sonnet.


{"type":"error","error":{"type":"invalid_request_error","message":"anthropic-version: header is required"}} (Net::HTTPBadResponse)
/var/www/discourse/plugins/discourse-ai/lib/inference/hugging_face_text_embeddings.rb:71:in `classify'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:142:in `request_with'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:78:in `block (4 levels) in bulk_classify!'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/promises.rb:1593:in `evaluate_to'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/promises.rb:1776:in `block in on_resolvable'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:359:in `run_task'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:350:in `block (3 levels) in create_worker'
<internal:kernel>:187:in `loop'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:341:in `block (2 levels) in create_worker'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:340:in `catch'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:340:in `block in create_worker'
Status: 400

{"type":"error","error":{"type":"invalid_request_error","message":"anthropic-version: header is required"}} (Net::HTTPBadResponse)
/var/www/discourse/plugins/discourse-ai/lib/inference/hugging_face_text_embeddings.rb:71:in `classify'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:142:in `request_wit...

The sentiment module doesn’t use general LLMs, but models specifically fine tuned to sentiment classification. If you want to run those models on your own that is documented at Self-Hosting Sentiment and Emotion for DiscourseAI

3 Likes

@Falco I just noticed the sentiment has stopped running as of Jan 2025. My guess that there’s this new setting ai_sentiment_model which as the above link explains if for running your own dedicated sentiment model/image. I noticed that after updating discourse now ai_sentiment_model_configs is all blank (should it be blank?).

When I try to run a backfill rake it gives me an error:

rake ai:sentiment:backfill
rake aborted!
ActiveRecord::StatementInvalid: PG::SyntaxError: ERROR:  syntax error at or near ")" (ActiveRecord::StatementInvalid)
LINE 1: ...e_upload_id", "posts"."outbound_message_id" FROM () as posts...
                                                             ^
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:69:in `exec_params'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:69:in `exec_params'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:894:in `block (2 levels) in exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1004:in `block in with_raw_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:976:in `with_raw_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:893:in `block in exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1119:in `log'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:892:in `exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:872:in `execute_and_clear'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:66:in `internal_exec_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:647:in `select'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:73:in `select_all'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:251:in `select_all'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/querying.rb:70:in `_query_by_sql'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1431:in `block (2 levels) in exec_main_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:415:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_handling.rb:296:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1430:in `block in exec_main_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:143:in `disable_query_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/query_cache.rb:30:in `uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:78:in `block in uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1355:in `_scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:541:in `scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:78:in `uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1450:in `skip_query_cache_if_necessary'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1414:in `exec_main_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1392:in `block in exec_queries'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:143:in `disable_query_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/query_cache.rb:30:in `uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:120:in `public_send'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:120:in `block in method_missing'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1355:in `_scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:541:in `scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:120:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1450:in `skip_query_cache_if_necessary'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1386:in `exec_queries'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1167:in `load'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:336:in `records'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:380:in `block in batch_on_unloaded_relation'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:378:in `batch_on_unloaded_relation'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:269:in `in_batches'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:157:in `find_in_batches'
/var/www/discourse/plugins/discourse-ai/lib/tasks/modules/sentiment/backfill.rake:7:in `block in <main>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'

Caused by:
PG::SyntaxError: ERROR:  syntax error at or near ")" (PG::SyntaxError)
LINE 1: ...e_upload_id", "posts"."outbound_message_id" FROM () as posts...
                                                             ^
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:69:in `exec_params'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:69:in `exec_params'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:894:in `block (2 levels) in exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1004:in `block in with_raw_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:976:in `with_raw_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:893:in `block in exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1119:in `log'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:892:in `exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:872:in `execute_and_clear'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:66:in `internal_exec_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:647:in `select'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:73:in `select_all'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:251:in `select_all'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/querying.rb:70:in `_query_by_sql'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1431:in `block (2 levels) in exec_main_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:415:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_handling.rb:296:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1430:in `block in exec_main_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:143:in `disable_query_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/query_cache.rb:30:in `uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:78:in `block in uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1355:in `_scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:541:in `scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:78:in `uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1450:in `skip_query_cache_if_necessary'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1414:in `exec_main_query'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1392:in `block in exec_queries'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:143:in `disable_query_cache'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/query_cache.rb:30:in `uncached'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:120:in `public_send'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:120:in `block in method_missing'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1355:in `_scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:541:in `scoping'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:120:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1450:in `skip_query_cache_if_necessary'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1386:in `exec_queries'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:1167:in `load'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation.rb:336:in `records'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:380:in `block in batch_on_unloaded_relation'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:378:in `batch_on_unloaded_relation'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:269:in `in_batches'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/batches.rb:157:in `find_in_batches'
/var/www/discourse/plugins/discourse-ai/lib/tasks/modules/sentiment/backfill.rake:7:in `block in <main>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => ai:sentiment:backfill
(See full trace by running task with --trace)

My question is, this was working fine until Nov/Dec 2024. How was it working earlier without a dedicated sentiment model/server? Is there a way to get it working again using a generic public service or built in service without having to run a dedicated server for it? I really like discourse is a self contained setup which makes it easy for simple deployment. Having to do custom deployments increases the complexity and maintenance costs which is troublesome for small deployments. Is there way to get back to the pre 2025 setup where sentiment worked out of the box?

The plugin defaulted to a server located in DigitalOcean which I put together to make testing easier.

I have since changed the plugin defaults to a clean slate and people who want AI classified need to run servers following the documentation here on Meta.

Indeed, but we were paying that cost for testing purposes. It’s not sustainable to offer that for every self hoster.

Worth mentioning, that we do offer this classification service on GPU accelerated servers as part of our hosting service.

2 Likes