Discourse AI: Need to Remove JSON Response to get Antrophic Sonnet 5-working in e.g. semantic search

Support
1

I may be completely off here and this is common knowledge to everyone but me… If so, I am sorry. But I have hade some strange errors in my installation, such as:

DiscourseAi::Completions::Endpoints::Anthropic: status: 400 - body: 

{
“type”:“error”,
“error”:{“type”:“invalid_request_error”,
“message”:“This model does not support assistant message prefill. The conversation must end with a user message.”},“request_id”:“req_011CcqMN6eswEfRHPmQmPLC4”
}

Which was strange since most other AI-related stuff worked. This happened primarly with my summarizer and with search as in:

https://..../forum/discourse-ai/embeddings/semantic-search?q=search_term

Which gave a 500 ERROR.

When I checked logs I got:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "This model does not support assistant message prefill. The conversation must end with a user message."
  },
  "request_id": "req_011CcqQvXPAHSfvoUeu92gf9"
}

I traced this back to Content Create-persona

image
image764×1632 59.4 KB

And the error seems to be this part:

image
image764×1632 59.4 KB

because when I removed it (which wasn’t possible on this specific persona) by doing a clone and setting this setting to my clone

image
image2034×194 16.7 KB

Semantic search started to work. I needed to do a similar thing in e.g. my summarizer.

Is this something you guys experienced as well?

2

Can you show how Sonnet 5 is configured on your instance?

Do you have Disable native structured output (use assistant message prefilling instead, required for Claude 3.x and older) enabled by any chance?

3

The forum in question is running 2026.1.5 ESR, and this option was added after 2026.1, so I guess a lot of Claude specific logic is missing there.

4

Oh, thanks for sharing!

Trying to use a new model with an old Discourse is not a winning strategy :smiley:.

The providers keep changing API formats, parameters, etc. We do the best to keep the compatibility with thousands of models, but that does require constant updates to keep up, specially if you want to keep up with models launched half a year (centuries in AI years) than the Discourse version.