Discourse AI Sentiment Analysis Issues: Hugging Face Model Format & Azure Endpoint Failure
Hi Discourse Community and Developers,
I’m encountering significant issues when attempting to configure and use the Sentiment Analysis feature within the Discourse AI plugin on my forum. It appears there are two distinct problems preventing it from working correctly.
Issue 1: Hugging Face Model Response Format Mismatch
I’ve configured the cardiffnlp/twitter-roberta-base-sentiment model from Hugging Face for sentiment analysis. While my API key is valid and I can successfully call the API from my Discourse instance using curl, the Discourse AI plugin seems to be parsing the response incorrectly due to a change in the Hugging Face model’s output format.
My curl command (confirming valid API key and new format):
Bashcurl -X POST https://api-inference.huggingface.co/models/cardiffnlp/twitter-roberta-base-sentiment \ -H "Authorization: Bearer hf_xxxxxxxxxxx" \ -H "Content-Type: application/json" \ -d "{\"inputs\": \"I love Discourse!\"}"
Output from curl (showing the new nested array format):
[[{"label":"LABEL_2","score":0.9891520738601685},{"label":"LABEL_1","score":0.009014752693474293},{"label":"LABEL_0","score":0.0018332178005948663}]]
Problem: The twitter-roberta-base-sentiment model used to return a single array of label-score hashes: [{"label": "LABEL_2", "score": 0.98}, ...]. However, it now returns a nested array: [[{"label": "LABEL_2", "score": 0.98}, ...]].
The Discourse AI plugin’s hardcoded parsing logic (specifically, classification["label"][/\d+/].to_i as indicated by the backtrace) does not account for this outer array layer. This leads to a TypeError when it tries to access a Symbol as an Integer.
Error Message (from Job Exception):
no implicit conversion of Symbol into Integer (TypeError)
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:163:in block in transform_result’
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:163:in each’
/var/www/discourse/plugin…`
Full Backtrace for Hugging Face Issue:
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/promises.rb:1268:in `raise'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/promises.rb:1268:in `wait_until_resolved!'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/promises.rb:998:in `value!'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:93:in `bulk_classify!'
/var/www/discourse/plugins/discourse-ai/app/jobs/scheduled/sentiment_backfill.rb:27:in `execute'
/var/www/discourse/app/jobs/base.rb:316:in `block (2 levels) in perform'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management.rb:21:in `with_connection'
/var/www/discourse/app/jobs/base.rb:303:in `block in perform'
/var/www/discourse/app/jobs/base.rb:299:in `each'
/var/www/discourse/app/jobs/base.rb:299:in `perform'
/var/www/discourse/app/jobs/base.rb:379:in `perform'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/mini_scheduler-0.18.0/lib/mini_scheduler/manager.rb:137:in `process_queue'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/mini_scheduler-0.18.0/lib/mini_scheduler/manager.rb:77:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/mini_scheduler-0.18.0/lib/mini_scheduler/manager.rb:63:in `block (2 levels) in ensure_worker_threads'
<internal:kernel>:187:in `loop'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:341:in `block (2 levels) in create_worker'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:340:in `catch'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:340:in `block in create_worker'
no implicit conversion of Symbol into Integer (TypeError)
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:163:in `block in transform_result'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:163:in `each'
/var/www/discourse/plugin...
Issue 2: Microsoft Azure Model Configuration Leads to Hugging Face Error
When I attempted to switch to the Microsoft Text Analytics model within the Discourse AI settings, I encountered a 404 Resource not found error, and surprisingly, the backtrace still points to hugging_face_text_embeddings.rb.
Error Message (from Job Exception):
Job exception: 416 errors
{"error":{"code":"404","message": "Resource not found"}} (Net::HTTPBadResponse)
Relevant Backtrace Snippet (pointing to Hugging Face despite Microsoft model selected):
/var/www/discourse/plugins/discourse-ai/lib/inference/hugging_face_text_embeddings.rb:76:in `do_request!'
/var/www/discourse/plugins/discourse-ai/lib/inference/hugging_face_text_embeddings.rb:51:in `classify_by_sentiment!'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/post_classification.rb:156:in `request_with'
Observation: This indicates that even when I select and configure the Microsoft model’s endpoint and API key, the Discourse AI plugin appears to be hardcoded or incorrectly routing the sentiment analysis requests through Hugging Face-specific logic or endpoints. This prevents the Microsoft model from being used at all.
Configuration Screenshots:
I’ve attached screenshot of my Discourse AI settings to show the configuration:
- Detailed configuration for AI sentiment models (showing both Hugging Face and Microsoft models) - I tested with just Hugging Face or just Microsoft model configs with the same result
These issues make the sentiment analysis feature effectively unusable. It seems the plugin requires an update to handle the new Hugging Face response format and to correctly route requests when different sentiment providers are configured.
Any assistance or guidance on these issues would be greatly appreciated.
Thank you!
This topic covers the configuration of the Sentiment feature of the Discourse AI plugin.
Required user level: Administrator
Once enabled, Sentiment will classify all posts going forward and from the last 60 days. To classify all of your site’s historical posts, a backfill task must be run from the console.
Hosted by us?
Self-hosted?
