Discourse AI: Need to Remove JSON Response to get Antrophic Sonnet 5-working in e.g. semantic search

Support
1

I may be completely off here and this is common knowledge to everyone but me… If so, I am sorry. But I have hade some strange errors in my installation, such as:

DiscourseAi::Completions::Endpoints::Anthropic: status: 400 - body: 

{
“type”:“error”,
“error”:{“type”:“invalid_request_error”,
“message”:“This model does not support assistant message prefill. The conversation must end with a user message.”},“request_id”:“req_011CcqMN6eswEfRHPmQmPLC4”
}

Which was strange since most other AI-related stuff worked. This happened primarly with my summarizer and with search as in:

https://..../forum/discourse-ai/embeddings/semantic-search?q=search_term

Which gave a 500 ERROR.

When I checked logs I got:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "This model does not support assistant message prefill. The conversation must end with a user message."
  },
  "request_id": "req_011CcqQvXPAHSfvoUeu92gf9"
}

I traced this back to Content Create-persona

image
image764×1632 59.4 KB

And the error seems to be this part:

image
image764×1632 59.4 KB

because when I removed it (which wasn’t possible on this specific persona) by doing a clone and setting this setting to my clone

image
image2034×194 16.7 KB

Semantic search started to work. I needed to do a similar thing in e.g. my summarizer.

Is this something you guys experienced as well?

2

Can you show how Sonnet 5 is configured on your instance?

Do you have Disable native structured output (use assistant message prefilling instead, required for Claude 3.x and older) enabled by any chance?