Claude 3.7 Sonnet을 Discourse AI 플러그인과 함께 사용할 때 오류 발생

[emansilla]

안녕하세요,

Claude 3.7 Sonnet을 포럼 헬퍼 페르소나 및 사용자 정의 페르소나와 함께 사용해 보려고 했는데 아래 오류가 발생합니다. 최신 버전의 Discourse와 최신 버전의 Discourse AI 플러그인을 설치해 두었습니다. 재현 방법은 간단합니다. 봇에게 질문을 하면 검색, 카테고리, 읽기 등의 도구 사용(tool use)을 수행한 후에 오류가 발생합니다. LLM은 추론(reasoning)이 활성화된 상태로 설정되어 있으며, 추론에 1024개의 토큰이 사용되도록 설정되어 있습니다. 이것이 버그인지, 아니면 제 설정에 누락된 부분이 있는지 확인해 주실 수 있을까요? 감사합니다!

{"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0.type: Expected thinkingorredacted_thinking, but foundtool_use. Whenthinkingis enabled, a finalassistantmessage must start with a thinking block (preceeding the lastmost set oftool_useandtool_resultblocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disablethinking . Please consult our documentation at [Building with extended thinking - Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking)"}}

[Falco]

지난 24시간 동안 Discourse를 업데이트하셨나요? 그렇지 않다면 이 새로운 모델을 지원하려면 업데이트가 필요합니다.

[emansilla]

네, 어제 오후에 업데이트를 했습니다. Claude Sonnet 3.7에 대한 완전한 지원에 관한 업데이트 내용을 확인했습니다.

[Falco]

방금 이 설정으로 AI 봇을 테스트했는데 정상적으로 작동합니다.

[emansilla]

도와주셔서 감사합니다. 동일한 설정을 사용했는데도 오류가 계속 발생합니다.

제 설정은 다음과 같습니다:

모든 것이 최신 버전입니다:

아래는 디버그 입력 및 출력입니다(개인정보 보호를 위해 일부가 편집되었습니다):

입력 (1페이지):

Tokens de solicitud: 1482
Tokens de respuesta: 202
model: claude-3-7-sonnet-20250219
max_tokens: 38192
thinking:
type: enabled
budget_tokens: 8192
messages:
0:
role: user
content: XXXXX: Revisa si este anuncio cumple todas las políticas de XXXXX. 

XXXXX

system: You are a helpful Discourse assistant.
You _understand_ and **generate** Discourse Markdown.
You live in a Discourse Forum Message.

You live in the forum with the URL: https://XXXXX
The title of your site: XXXXX
The description is: XXXXX
The participants in this conversation are: XXXXX, XXXXX
The date now is: 2025-02-26 14:08:13 UTC, much has changed since you were trained.

You were trained on OLD data, lean on search to get up to date information about this forum
When searching try to SIMPLIFY search terms
Discourse search joins all terms with AND. Reduce and simplify terms to find more results.
stream: true
tools:
0:
name: categories
description: Will list the categories on the current discourse instance, prefer to format with # in front of the category name
input_schema:
type: object
properties:
required:
1:
name: search
description: Will search topics in the current discourse instance, when rendering always prefer to link to the topics you find
input_schema:
type: object
properties:
search_query:
type: string
description: Specific keywords to search for, space separated (correct bad spelling, remove connector words)
user:
type: string
description: Filter search results to this username (only include if user explicitly asks to filter by user)
order:
type: string
description: search result order
enum:
0: latest
1: latest_topic
2: oldest
3: views
4: likes
limit:
type: integer
description: limit number of results returned (generally prefer to just keep to default)
max_posts:
type: integer
description: maximum number of posts on the topics (topics where lots of people posted)
tags:
type: string
description: list of tags to search for. Use + to join with OR, use , to join with AND
category:
type: string
description: category name to filter to
before:
type: string
description: only topics created before a specific date YYYY-MM-DD
after:
type: string
description: only topics created after a specific date YYYY-MM-DD
status:
type: string
description: search for topics in a particular state
enum:
0: open
1: closed
2: archived
3: noreplies
4: single_user
required:
2:
name: read
description: Will read a topic or a post on this Discourse instance
input_schema:
type: object
properties:
topic_id:
type: integer
description: the id of the topic to read
post_numbers:
type: array
description: the post numbers to read (optional)
items:
type: integer
required:
0: topic_id
3:
name: tags
description: Will list the 100 most popular tags on the current discourse instance
input_schema:
type: object
properties:
required:

입력 (2페이지):

Tokens de solicitud: 778
Tokens de respuesta: 450
model: claude-3-7-sonnet-20250219
max_tokens: 38192
thinking:
type: enabled
budget_tokens: 8192
messages:
0:
role: user
content: Given the following conversation:

{{{
User said:
Revisa si este anuncio cumple todas las políticas de XXXX. 

XXXX

Model said:
Para revisar si este anuncio cumple con todas las políticas de XXXX, necesito consultar las políticas vigentes en el sistema. Voy a buscar información sobre las políticas de publicación de anuncios laborales.

<details>
<summary>Buscar</summary>
<p>Se han encontrado 53 <a href='/search?q=politicas+publicacion+anuncios+ofertas+laborales'>resultados</a> para 'politicas publicacion anuncios ofertas laborales'</p>
</details>
<span></span>



Lo sentimos, parece que nuestro sistema ha encontrado un problema inesperado al intentar responder.

[details='Detalles del error']
{"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `tool_use`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking"}}
[/details]


}}}

Reply only with a title that is 7 words or less.
system: You are titlebot. Given a conversation, you will suggest a title.

- You will never respond with anything but the suggested title.
- You will always match the conversation language in your title suggestion.
- Title will capture the essence of the conversation.

입력 (3페이지):

Tokens de solicitud: 925
Tokens de respuesta: 154
model: gpt-4o-mini
messages:
0:
role: system
content: You are an advanced summarization bot that generates concise, coherent summaries of provided text.

- Only include the summary, without any additional commentary.
- You understand and generate Discourse forum Markdown; including links, _italics_, **bold**.
- Maintain the original language of the text being summarized.
- Aim for summaries to be 400 words or less.
- Each post is formatted as "<POST_NUMBER>) <USERNAME> <MESSAGE>"
- Cite specific noteworthy posts using the format [DESCRIPTION](/t/-/1488/POST_NUMBER)
- Example: links to the 3rd and 6th posts by sam: sam ([#3](/t/-/1488/3), [#6](/t/-/1488/6))
- Example: link to the 6th post by jane: [agreed with](/t/-/1488/6)
- Example: link to the 13th post by joe: [joe](/t/-/1488/13)
- When formatting usernames either use @USERNMAE OR [USERNAME](/t/-/1488/POST_NUMBER)
1:
role: user
content: Here are the posts inside <input></input> XML tags:

<input>1) user1 said: I love Mondays 2) user2 said: I hate Mondays</input>

Generate a concise, coherent summary of the text above maintaining the original language.
2:
role: assistant
content: Two users are sharing their feelings toward Mondays. [user1](/t/-/1488/1) hates them, while [user2](/t/-/1488/2) loves them.
3:
role: user
content: The discussion title is: Revisión de oferta laboral XXXXX.

Here are the posts, inside <input></input> XML tags:

<input>
(1 XXXXX said: XXXXX (2 XXXXXsaid: Para revisar si este anuncio cumple con todas las políticas de XXXXX, necesito consultar las políticas vigentes en el sistema. Voy a buscar información sobre las políticas de publicación de anuncios laborales.

<details>
<summary>Buscar</summary>
<p>Se han encontrado 53 <a href='/search?q=politicas+publicacion+anuncios+ofertas+laborales'>resultados</a> para 'politicas publicacion anuncios ofertas laborales'</p>
</details>
<span></span>



Lo sentimos, parece que nuestro sistema ha encontrado un problema inesperado al intentar responder.

[details='De talles del error']
{"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `tool_use`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking"}}
[/details]
</input>

Generate a concise, coherent summary of the text above maintaining the original language.

출력:

Tokens de solicitud: 1482
Tokens de respuesta: 202
event: message_start
data: {"type":"message_start","message":{"id":"msg_017uvsRAvCrkin8deo7KfSJt","type":"message","role":"assistant","model":"claude-3-7-sonnet-20250219","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":1482,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":3}} }

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"","signature":""} }

event: ping
data: {"type": "ping"}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"I nee"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"d to review this job posting to ensure"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" it complies with"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" XXXXX policies,"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" excluding the requirement about transparency of the company"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" name. To do this effectively"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":", I should first search for information"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" about XXXXX policies for"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" job postings."} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"\n\nLet me search for information"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" about the policies or guidelines for job postings"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":" on the XXXXX platform"} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"."} }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"EuYBCkQYAiJAV6gVRVTK/ffuBAUV445vA+2ceuLiwxoP8zp1cUlPnNePLQ/PPDz25BQ9d0GyvgP2mxXVoibY8Y5fF0Y5ZXaR9BIM6B73OywtDTgvsnqpGgyLjAssqp2Lfyq3ucoiMKGlt32Bdl4/9lUWS3bXwPB2GCUeMUkgbKxJGzUF2CN+oVd/2RSSz5hzJynbKEiUbypQd1OjBx+f5vzMznA6drsgbFxUT3yzYzqBxpIWr8x7O8t8v63gcZw/mo3lIJrMSrAbVWL4bxXKe9cEq8LcG3DxffOSmsowGfeoKnAzkvsJus0="} }

event: content_block_stop
data: {"type":"content_block_stop","index":0 }

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Para revisar si este anuncio cum"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"ple con todas las polí"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"ticas de XXXXX "} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"XXXXX , necesito consul"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"tar las políticas vig"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"entes en el sistema."} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" Voy a buscar"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" información sobre las políticas de public"} }

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"ación de anuncios laborales."} }

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: content_block_start
data: {"type":"content_block_start","index":2,"content_block":{"type":"tool_use","id":"toolu_01JwNffkhK6d1LPtcbNHRtry","name":"search","input":{}} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":""} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"{\"search_q"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"uer"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"y\": \"po"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"lit"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"ic"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"as publicac"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"ion anun"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"cios "} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"ofertas labo"} }

event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"rales\"}"}}

event: content_block_stop
data: {"type":"content_block_stop","index":2 }

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":202} }

event: message_stop
data: {"type":"message_stop" }

[Falco]

아, 그렇군요. 문제는 공식 Anthropic API에서 사고 과정(thinking)과 네이티브 도구 사용(native tool use)을 혼합하는 것입니다.

[emansilla]

맞아요, LLM 설정에서 네이티브 도구 지원을 비활성화하면 작동합니다.

[sam (Sam Saffron)]

감사합니다. 확인해 보겠습니다. 현재 thinking 블록을 반환하지 않고 있어서, 이를 보존하는 방법을 알아내야 합니다. 꽤 많은 변경 사항이 필요할 것 같습니다.

축약된 thinking 블록은 더 혼란스럽습니다.

[sam (Sam Saffron)]

Thinking은 다음 커밋에서 수정되었습니다:

FEATURE: implement thinking token support (#1155)

매우 복잡한 변경 사항입니다. Anthropic의 프로토콜을 따르는 것이 쉽지 않으며, thinking 블록을 계속 관리해야 합니다.

진행 상황을 알려주세요.

[emansilla]

수정해 주셔서 감사합니다. 몇 가지 테스트를 해 보았는데, 이제 잘 작동합니다. Claude 3.7 Sonnet의 경우 토큰 사용량이 훨씬 더 높다는 것을 알게 되었습니다. 1분당 80k 토큰인 제 Tier 3 한도의 여러 배에 달하고 있습니다. 아래처럼 제한적인 설정을 사용해도 한도에 도달하고 있습니다:

{“type”:“error”,“error”:{“type”:“rate_limit_error”,“message”:“This request would exceed the rate limit for your organization (86982fb5-12e6-4c93-be7a-4e51d63d1bf3) of 80,000 input tokens per minute. For details, refer to: Rate limits - Anthropic. You can see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase.”}}

[sam (Sam Saffron)]

Anthropic이 현재 3.7에서 확장성 문제를 겪고 있는 것 같습니다. 어제 장애가 발생했고, 일반적으로 API의 성능이 다소 불안정합니다.

문제를 해결할 수 있도록 몇 주 정도 기다려 주는 것이 좋습니다.

또한, 적어도 채팅에 토큰 캐싱을 추가하는 것을 고려하고 있습니다. 가능한지 확인해 보겠습니다.