Encountering Error in Sentiment Backfill Rake Task

Support
,

AI-generated summary

Users are experiencing errors when running the sentiment backfill rake task. RBoy initially encountered an error after 6-10 minutes of processing, with the task aborting due to a Net::HTTPBadResponse exception. The error persists even after restarting the task. RBoy also noticed a log message related to scoped order being ignored. Bathinda reported a similar issue, with sentiment analysis failing and backfill tasks not working. RBoy mentioned that no errors are visible in the Discourse logs, only the exception on the console.

1

I’m trying to run the sentiment backfill rake, it starts working for a while (say about 6-10 minutes) and then it throws an error.

:/var/www/discourse# rake ai:sentiment:backfill --trace
** Invoke ai:sentiment:backfill (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute ai:sentiment:backfill
............................................................rake aborted!
Net::HTTPBadResponse: Net::HTTPBadResponse (Net::HTTPBadResponse)
/var/www/discourse/plugins/discourse-ai/lib/inference/discourse_classifier.rb:14:in `perform!'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/sentiment_classification.rb:42:in `request_with'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/sentiment_classification.rb:34:in `block in request'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/sentiment_classification.rb:33:in `each'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/sentiment_classification.rb:33:in `reduce'
/var/www/discourse/plugins/discourse-ai/lib/sentiment/sentiment_classification.rb:33:in `request'
/var/www/discourse/plugins/discourse-ai/lib/classificator.rb:13:in `classify!'
/var/www/discourse/plugins/discourse-ai/lib/tasks/modules/sentiment/backfill.rake:21:in `block (2 levels) in <main>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:71:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:71:in `block in find_each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:138:in `block in find_in_batches'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:245:in `block in in_batches'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:229:in `loop'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:229:in `in_batches'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:137:in `find_in_batches'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:70:in `find_each'
/var/www/discourse/plugins/discourse-ai/lib/tasks/modules/sentiment/backfill.rake:17:in `block in <main>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/cli.rb:451:in `exec'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/cli.rb:34:in `dispatch'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/cli.rb:28:in `start'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/exe/bundle:28:in `block in <top (required)>'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.5.3/exe/bundle:20:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => ai:sentiment:backfill

If I try to run it again, it aborts immediately with the same error, i.e. no incremental processing. Looks like it reached some point in the process where now it’s encountering this error.

Do you need any additional logs?

2

@Falco I tried this again today after a reboot and it gives the same error instantly, as soon as the rake started. Any suggestions?

EDIT: I noticed this in the discourse logs if it helps:

Message (4 copies reported)

Scoped order is ignored, it's forced to be batch order.

Backtrace

/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:292:in `act_on_ignored_order'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:215:in `in_batches'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:137:in `find_in_batches'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.1/lib/active_record/relation/batches.rb:70:in `find_each'
/var/www/discourse/plugins/discourse-ai/lib/tasks/modules/sentiment/backfill.rake:17:in `block in <main>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
3

For me too, Sentiment Analysis isn’t working. And Back-Fill is failing in the same way, as yours.
I’ve posted my screenshots of failing backfill here:

image
image902×708 82.3 KB

4

I’m not seeing any errors in discourse logs when I try to run it (anymore), just the exception on the console.