AI Triaging failing on Google Interaction Endpoints

Product Support

AI-generated summary

A user reported that AI triaging failed after migrating Google Vertex LLM configurations to Interaction Endpoints, resulting in an invalid_request error stating that model turns with thought summaries must start with a thought block. The issue occurred despite initial success and affected various agents using Flash 3.7 and 3.8 models.

The user later identified the root cause as the content window configured in the LLM settings being too small. Increasing the context window resolved the issue temporarily. They suggested that Discourse could improve handling by detecting insufficient context windows rather than truncating messages into invalid formats.

1

So I updated all the google vertex LLM to Interaction Endpoints and recently I started noticing this error in the logs as AI was no longer responding to posts automatically. What’s going on here? It was working initially but stopped working a few weeks ago.

Using Flash 3.7 and 3.8 for the various agents. AI Bot , Forum Helper, Inferred Concepts using 3.8, others using 3.7 with thinking this config:

image
image590×1002 31.2 KB

image
image610×757 24.5 KB

Message

Error responding to: /t/metadata-lookups-failing-for-movie-stargate-the-ark-of-truth/5969/2 in LlmAgentTriage.handle : DiscourseAi::Completions::Endpoints::Base::CompletionFailed : {"error":{"message":"Model turns with thought summaries must start with a thought block in thinking models.","code":"invalid_request"}}

Backtrace

/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:499:in 'block (3 levels) in DiscourseAi::Completions::Endpoints::Base#perform_completion_request_with_retries'
net-http-0.9.1/lib/net/http.rb:2461:in 'block in Net::HTTP#transport_request'
net-http-0.9.1/lib/net/http/response.rb:321:in 'Net::HTTPResponse#reading_body'
net-http-0.9.1/lib/net/http.rb:2458:in 'Net::HTTP#transport_request'
net-http-0.9.1/lib/net/http.rb:2410:in 'Net::HTTP#request'
rack-mini-profiler-5.0.0/lib/patches/net_patches.rb:19:in 'block in Net::HTTP#request_with_mini_profiler'
rack-mini-profiler-5.0.0/lib/mini_profiler/profiling_methods.rb:51:in 'Rack::MiniProfiler::ProfilingMethods#step'
rack-mini-profiler-5.0.0/lib/patches/net_patches.rb:18:in 'Net::HTTP#request_with_mini_profiler'
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:487:in 'block (2 levels) in DiscourseAi::Completions::Endpoints::Base#perform_completion_request_with_retries'
net-http-0.9.1/lib/net/http.rb:1630:in 'Net::HTTP#start'
net-http-0.9.1/lib/net/http.rb:1064:in 'Net::HTTP.start'
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:468:in 'block in DiscourseAi::Completions::Endpoints::Base#perform_completion_request_with_retries'
<internal:kernel>:168:in 'Kernel#loop'
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:444:in 'DiscourseAi::Completions::Endpoints::Base#perform_completion_request_with_retries'
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:196:in 'DiscourseAi::Completions::Endpoints::Base#perform_completion!'
/var/www/discourse/plugins/discourse-ai/lib/completions/llm.rb:258:in 'DiscourseAi::Completions::Llm#generate'
/var/www/discourse/plugins/discourse-ai/lib/agents/bot.rb:292:in 'DiscourseAi::Agents::Bot#reply'
/var/www/discourse/plugins/discourse-ai/lib/ai_bot/playground.rb:669:in 'DiscourseAi::AiBot::Playground#reply_to'
/var/www/discourse/plugins/discourse-ai/lib/ai_bot/playground.rb:224:in 'DiscourseAi::AiBot::Playground.reply_to_post'
/var/www/discourse/plugins/discourse-ai/lib/automation/llm_agent_triage.rb:13:in 'DiscourseAi::Automation::LlmAgentTriage.handle'
/var/www/discourse/plugins/discourse-ai/discourse_automation/llm_agent_triage.rb:44:in 'block (2 levels) in <main>'
/var/www/discourse/plugins/automation/app/models/discourse_automation/automation.rb:163:in 'block in DiscourseAutomation::Automation#trigger!'
/var/www/discourse/plugins/automation/app/models/discourse_automation/stat.rb:12:in 'DiscourseAutomation::Stat.log'
/var/www/discourse/plugins/automation/app/models/discourse_automation/automation.rb:161:in 'DiscourseAutomation::Automation#trigger!'
/var/www/discourse/plugins/automation/app/jobs/regular/discourse_automation/trigger.rb:30:in 'Jobs::DiscourseAutomation::Trigger#execute'
/var/www/discourse/app/jobs/base.rb:313:in 'block (2 levels) in Jobs::Base#perform'
rails_multisite-9.0.0/lib/rails_multisite/connection_management/null_instance.rb:54:in 'RailsMultisite::ConnectionManagement::NullInstance#with_connection'
rails_multisite-9.0.0/lib/rails_multisite/connection_management.rb:23:in 'RailsMultisite::ConnectionManagement.with_connection'
/var/www/discourse/app/jobs/base.rb:300:in 'block in Jobs::Base#perform'
/var/www/discourse/app/jobs/base.rb:297:in 'Array#each'
/var/www/discourse/app/jobs/base.rb:297:in 'Jobs::Base#perform'
sidekiq-7.3.10/lib/sidekiq/processor.rb:220:in 'Sidekiq::Processor#execute_job'
sidekiq-7.3.10/lib/sidekiq/processor.rb:185:in 'block (4 levels) in Sidekiq::Processor#process'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:180:in 'Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:183:in 'block in Sidekiq::Middleware::Chain#traverse'
/var/www/discourse/lib/sidekiq/suppress_user_email_errors.rb:6:in 'Sidekiq::SuppressUserEmailErrors#call'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:182:in 'Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:183:in 'block in Sidekiq::Middleware::Chain#traverse'
/var/www/discourse/lib/sidekiq/discourse_event.rb:6:in 'Sidekiq::DiscourseEvent#call'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:182:in 'Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:183:in 'block in Sidekiq::Middleware::Chain#traverse'
/var/www/discourse/lib/sidekiq/pausable.rb:136:in 'Sidekiq::Pausable#call'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:182:in 'Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:183:in 'block in Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/job/interrupt_handler.rb:9:in 'Sidekiq::Job::InterruptHandler#call'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:182:in 'Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:183:in 'block in Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/metrics/tracking.rb:26:in 'Sidekiq::Metrics::ExecutionTracker#track'
sidekiq-7.3.10/lib/sidekiq/metrics/tracking.rb:134:in 'Sidekiq::Metrics::Middleware#call'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:182:in 'Sidekiq::Middleware::Chain#traverse'
sidekiq-7.3.10/lib/sidekiq/middleware/chain.rb:173:in 'Sidekiq::Middleware::Chain#invoke'
sidekiq-7.3.10/lib/sidekiq/processor.rb:184:in 'block (3 levels) in Sidekiq::Processor#process'
sidekiq-7.3.10/lib/sidekiq/processor.rb:145:in 'block (6 levels) in Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/job_retry.rb:118:in 'Sidekiq::JobRetry#local'
sidekiq-7.3.10/lib/sidekiq/processor.rb:144:in 'block (5 levels) in Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/config.rb:39:in 'block in <class:Config>'
sidekiq-7.3.10/lib/sidekiq/processor.rb:139:in 'block (4 levels) in Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/processor.rb:281:in 'Sidekiq::Processor#stats'
sidekiq-7.3.10/lib/sidekiq/processor.rb:134:in 'block (3 levels) in Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/job_logger.rb:15:in 'Sidekiq::JobLogger#call'
sidekiq-7.3.10/lib/sidekiq/processor.rb:133:in 'block (2 levels) in Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/job_retry.rb:85:in 'Sidekiq::JobRetry#global'
sidekiq-7.3.10/lib/sidekiq/processor.rb:132:in 'block in Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/job_logger.rb:40:in 'Sidekiq::JobLogger#prepare'
sidekiq-7.3.10/lib/sidekiq/processor.rb:131:in 'Sidekiq::Processor#dispatch'
sidekiq-7.3.10/lib/sidekiq/processor.rb:183:in 'block (2 levels) in Sidekiq::Processor#process'
sidekiq-7.3.10/lib/sidekiq/processor.rb:182:in 'Thread.handle_interrupt'
sidekiq-7.3.10/lib/sidekiq/processor.rb:182:in 'block in Sidekiq::Processor#process'
sidekiq-7.3.10/lib/sidekiq/processor.rb:181:in 'Thread.handle_interrupt'
sidekiq-7.3.10/lib/sidekiq/processor.rb:181:in 'Sidekiq::Processor#process'
sidekiq-7.3.10/lib/sidekiq/processor.rb:86:in 'Sidekiq::Processor#process_one'
sidekiq-7.3.10/lib/sidekiq/processor.rb:76:in 'Sidekiq::Processor#run'
sidekiq-7.3.10/lib/sidekiq/component.rb:10:in 'Sidekiq::Component#watchdog'
sidekiq-7.3.10/lib/sidekiq/component.rb:19:in 'block in Sidekiq::Component#safe_thread'
2

If this helps anyone, it appears the content window configured in the LLM configuration was too small. Increasing it seems to have fixed it (for now atleast). Will keep an eye on it.

Perhaps discourse could find another way to handle it if the context window isn’t large enough rather than cutting off the message which makes it an invalid message.