Error using Claude 3.7 Sonnet with Discourse AI plugin

[emansilla]

Hello,

I have been trying to use Claude 3.7 Sonnet with the forum helper persona and with custom personas and I get the error below. I have the latest version of Discourse and the latest version of the Discourse AI plugin. The steps to reproduce are simple, just ask a question to the bot and it will perform a tool use such as search, category or read and afterwards I get the error. The LLM is configured with reasoning enabled and 1024 tokens used for reasoning. Can you please confirm if this is a bug or if something is missing in my configuration? Thank you very much!

{“type”:“error”,“error”:{“type”:“invalid_request_error”,“message”:“messages.1.content.0.type: Expected thinkingorredacted_thinking, but foundtool_use. Whenthinkingis enabled, a finalassistantmessage must start with a thinking block (preceeding the lastmost set oftool_useandtool_resultblocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disablethinking . Please consult our documentation at [Building with extended thinking - Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking)”}}

[Falco]

Have you updated Discourse in the last 24h ? If not, you need to update to get support for this new model.

[emansilla]

Yes, I updated yesterday afternoon. I saw the update about full support for Claude Sonnet 3.7.

[Falco]

I just tested AI Bot just now with this config and it is working fine

[emansilla]

Thank you for your help. I have used the same settings and the error continues.

Below are my settings:

I have the latest version of everything:

Below are the debug input and output (slightly redacted for privacy):

INPUT (page 1):

Tokens de solicitud: 1482
Tokens de respuesta: 202
model: claude-3-7-sonnet-20250219
max_tokens: 38192
thinking:
type: enabled
budget_tokens: 8192
messages:
0:
role: user
content: XXXXX: Revisa si este anuncio cumple todas las políticas de XXXXX. 

XXXXX

system: You are a helpful Discourse assistant.
You _understand_ and **generate** Discourse Markdown.
You live in a Discourse Forum Message.

You live in the forum with the URL: https://XXXXX
The title of your site: XXXXX
The description is: XXXXX
The participants in this conversation are: XXXXX, XXXXX
The date now is: 2025-02-26 14:08:13 UTC, much has changed since you were trained.

You were trained on OLD data, lean on search to get up to date information about this forum
When searching try to SIMPLIFY search terms
Discourse search joins all terms with AND. Reduce and simplify terms to find more results.
stream: true
tools:
0:
name: categories
description: Will list the categories on the current discourse instance, prefer to format with # in front of the category name
input_schema:
type: object
properties:
required:
1:
name: search
description: Will search topics in the current discourse instance, when rendering always prefer to link to the topics you find
input_schema:
type: object
properties:
search_query:
type: string
description: Specific keywords to search for, space separated (correct bad spelling, remove connector words)
user:
type: string
description: Filter search results to this username (only include if user explicitly asks to filter by user)
order:
type: string
description: search result order
enum:
0: latest
1: latest_topic
2: oldest
3: views
4: likes
limit:
type: integer
description: limit number of results returned (generally prefer to just keep to default)
max_posts:
type: integer
description: maximum number of posts on the topics (topics where lots of people posted)
tags:
type: string
description: list of tags to search for. Use + to join with OR, use , to join with AND
category:
type: string
description: category name to filter to
before:
type: string
description: only topics created before a specific date YYYY-MM-DD
after:
type: string
description: only topics created after a specific date YYYY-MM-DD
status:
type: string
description: search for topics in a particular state
enum:
0: open
1: closed
2: archived
3: noreplies
4: single_user
required:
2:
name: read
description: Will read a topic or a post on this Discourse instance
input_schema:
type: object
properties:
topic_id:
type: integer
description: the id of the topic to read
post_numbers:
type: array
description: the post numbers to read (optional)
items:
type: integer
required:
0: topic_id
3:
name: tags
description: Will list the 100 most popular tags on the current discourse instance
input_schema:
type: object
properties:
required:

INPUT (page 2):

Tokens de solicitud: 778
Tokens de respuesta: 450
model: claude-3-7-sonnet-20250219
max_tokens: 38192
thinking:
type: enabled
budget_tokens: 8192
messages:
0:
role: user
content: Given the following conversation:

{{{
User said:
Revisa si este anuncio cumple todas las políticas de XXXX. 

XXXX

Model said:
Para revisar si este anuncio cumple con todas las políticas de XXXX, necesito consultar las políticas vigentes en el sistema. Voy a buscar información sobre las políticas de publicación de anuncios laborales.

<details>
<summary>Buscar</summary>
<p>Se han encontrado 53 <a href='/search?q=politicas+publicacion+anuncios+ofertas+laborales'>resultados</a> para 'politicas publicacion anuncios ofertas laborales'</p>
</details>
<span></span>



Lo sentimos, parece que nuestro sistema ha encontrado un problema inesperado al intentar responder.

[details='Detalles del error']
{"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `tool_use`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking"}}
[/details]


}}}

Reply only with a title that is 7 words or less.
system: You are titlebot. Given a conversation, you will suggest a title.

- You will never respond with anything but the suggested title.
- You will always match the conversation language in your title suggestion.
- Title will capture the essence of the conversation.

INPUT (page 3):

Tokens de solicitud: 925
Tokens de respuesta: 154
model: gpt-4o-mini
messages:
0:
role: system
content: You are an advanced summarization bot that generates concise, coherent summaries of provided text.

- Only include the summary, without any additional commentary.
- You understand and generate Discourse forum Markdown; including links, _italics_, **bold**.
- Maintain the original language of the text being summarized.
- Aim for summaries to be 400 words or less.
- Each post is formatted as "<POST_NUMBER>) <USERNAME> <MESSAGE>"
- Cite specific noteworthy posts using the format [DESCRIPTION](/t/-/1488/POST_NUMBER)
- Example: links to the 3rd and 6th posts by sam: sam ([#3](/t/-/1488/3), [#6](/t/-/1488/6))
- Example: link to the 6th post by jane: [agreed with](/t/-/1488/6)
- Example: link to the 13th post by joe: [joe](/t/-/1488/13)
- When formatting usernames either use @USERNMAE OR [USERNAME](/t/-/1488/POST_NUMBER)
1:
role: user
content: Here are the posts inside <input></input> XML tags:

<input>1) user1 said: I love Mondays 2) user2 said: I hate Mondays</input>

Generate a concise, coherent summary of the text above maintaining the original language.
2:
role: assistant
content: Two users are sharing their feelings toward Mondays. [user1](/t/-/1488/1) hates them, while [user2](/t/-/1488/2) loves them.
3:
role: user
content: The discussion title is: Revisión de oferta laboral XXXXX.

Here are the posts, inside <input></input> XML tags:

<input>
(1 XXXXX said: XXXXX (2 XXXXXsaid: Para revisar si este anuncio cumple con todas las políticas de XXXXX, necesito consultar las políticas vigentes en el sistema. Voy a buscar información sobre las políticas de publicación de anuncios laborales.

<details>
<summary>Buscar</summary>
<p>Se han encontrado 53 <a href='/search?q=politicas+publicacion+anuncios+ofertas+laborales'>resultados</a> para 'politicas publicacion anuncios ofertas laborales'</p>
</details>
<span></span>



Lo sentimos, parece que nuestro sistema ha encontrado un problema inesperado al intentar responder.

[details='De talles del error']
{"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `tool_use`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking"}}
[/details]
</input>

Generate a concise, coherent summary of the text above maintaining the original language.

OUTPUT:

Tokens de solicitud: 1482
Tokens de respuesta: 202
event: message_start
data: {"type":"message_start","message":{"id":"msg_017uvsRAvCrkin8deo7KfSJt","type":"message","role":"assistant","model":"claude-3-7-sonnet-20250219","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":1482,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":3}} }

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"","signature":""} }

event: ping
data: {"type": "ping"}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"I nee"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"d to review this job posting to ensure"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" it complies with"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" XXXXX policies,"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" excluding the requirement about transparency of the company"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" name. To do this effectively"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":", I should first search for information"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" about XXXXX policies for"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" job postings."} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"\n\nLet me search for information"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" about the policies or guidelines for job postings"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" on the XXXXX platform"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"."} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"EuYBCkQYAiJAV6gVRVTK/ffuBAUV445vA+2ceuLiwxoP8zp1cUlPnNePLQ/PPDz25BQ9d0GyvgP2mxXVoibY8Y5fF0Y5ZXaR9BIM6B73OywtDTgvsnqpGgyLjAssqp2Lfyq3ucoiMKGlt32Bdl4/9lUWS3bXwPB2GCUeMUkgbKxJGzUF2CN+oVd/2RSSz5hzJynbKEiUbypQd1OjBx+f5vzMznA6drsgbFxUT3yzYzqBxpIWr8x7O8t8v63gcZw/mo3lIJrMSrAbVWL4bxXKe9cEq8LcG3DxffOSmsowGfeoKnAzkvsJus0="} }

event: content_block_stop
data: {"type":"content_block_stop","index":0 }

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Para revisar si este anuncio cum"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"ple con todas las polí"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"ticas de XXXXX "} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"XXXXX , necesito consul"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"tar las políticas vig"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"entes en el sistema."} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" Voy a buscar"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" información sobre las políticas de public"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"ación de anuncios laborales."} }

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: content_block_start
data: {"type":"content_block_start","index":2,"content_block":{"type":"tool_use","id":"toolu_01JwNffkhK6d1LPtcbNHRtry","name":"search","input":{}} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":""} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"{\"search_q"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"uer"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"y\": \"po"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"lit"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"ic"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"as publicac"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"ion anun"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"cios "} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"ofertas labo"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"rales\"}"}}

event: content_block_stop
data: {"type":"content_block_stop","index":2 }

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":202} }

event: message_stop
data: {"type":"message_stop" }

[Falco]

Oh I see, the problem is mixing thinking and native tool use on the official Anthropic API.

[emansilla]

True, when I disable native tool support in the LLM settings, it works.

[sam (Sam Saffron)]

Thanks, will have a look, need to figure out how to conserve the thinking blocks, we are not returning them at the moment, we will need quite a few changes

Reducted thinking blocks are extra confusing

[sam (Sam Saffron)]

Thinking is fixed per:

FEATURE: implement thinking token support (#1155)

It is a very complex change, the protocol from anthropic is not easy to follow, you need to carry thinking blocks around.

Let me know how you go.

[emansilla]

Thank you for fixing it. I have done a couple of tests, and it is working now. I have noticed that with Claude 3.7 Sonnet the token usage is much higher, reaching many times my Tier 3 limits of 80k tokens per minute. I am hitting the limit even with these restrictive settings:

{“type”:“error”,“error”:{“type”:“rate_limit_error”,“message”:“This request would exceed the rate limit for your organization (86982fb5-12e6-4c93-be7a-4e51d63d1bf3) of 80,000 input tokens per minute. For details, refer to: Rate limits - Anthropic. You can see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase.”}}

[sam (Sam Saffron)]

I think Anthropic are having scaling issues with 3.7 at the moment, there was an outage yesterday and generally the API can be quite uneven perf wise.

Recommend giving them a few weeks so they nut out the issues.

Also, thinking about adding token caching, at least to chat will see if I can swing that.