대부분의 요약이 실패하는 이유는 LLM으로 전송된 텍스트의 일부가 뒤집혀서(!)이기 때문입니다. 게시물이 단순히 다음과 같은 경우:
This post is to test whether sections of text get reversed in the payload sent to the AI endpoint.
실제로 LLM에 전송되는 텍스트는 다음과 같습니다:
This post is to test whether sections of text get .tniopdne IA eht ot tnes daolyap eht ni desrever
이 간단한 테스트에서 문자열의 정확히 중간 지점에서 접히거나 반전되는 것을 볼 수 있습니다.
이것이 말이 안 되는 것처럼 들릴 수 있다는 점을 잘 알고 있으므로, 스크린샷과 ai_api_audit_logs의 해당 raw_request_payload를 공유합니다:
ai_api_audit_logs.raw_request_payload
{
"model": "claude-sonnet-4-0",
"max_tokens": 64000,
"messages": [
{
"role": "user",
"content": "Here are the posts inside <input></input> XML tags:\n\n<input>1) user1 said: I love Mondays 2) user2 said: I hate Mondays</input>\n\nGenerate a concise, coherent summary of the text above maintaining the original language."
},
{
"role": "assistant",
"content": "{\"summary\":\"Two users are sharing their feelings toward Mondays. [user1](/t/-/64255/1) hates them, while [user2](/t/-/64255/2) loves them.\"}"
},
{
"role": "user",
"content": "The discussion title is: Testing Text Reversal.\n\nCategory: testing.\n\n\nHere are the posts, inside <input></input> XML tags:\n\n<input>\n (1 admin said: This post is to test whether sections of text get .tniopdne IA eht ot tnes daolyap eht ni desrever \n</input>\n\nGenerate a concise, coherent summary of the text above maintaining the original language."
},
{
"role": "assistant",
"content": "{"
}
],
"system": "You are an advanced summarization bot that generates concise, coherent summaries of provided text.\nYou are also capable of enhancing an existing summaries by incorporating additional posts if asked to.\n\n- Only include the summary, without any additional commentary.\n- You understand and generate Discourse forum Markdown; including links, _italics_, **bold**.\n- Maintain the original language of the text being summarized.\n- Aim for summaries to be 400 words or less.\n- Each post is formatted as \"<POST_NUMBER>) <USERNAME> <MESSAGE>\"\n- Cite specific noteworthy posts using the format [DESCRIPTION](/t/-/64255/POST_NUMBER)\n- Example: link to the 6th post by jane: [agreed with](/t/-/64255/6)\n- Example: link to the 13th post by joe: [joe](/t/-/64255/13)\n- When formatting usernames use [USERNAME](/t/-/64255/POST_NUMBER)\n\nFormat your response as a JSON object with a single key named \"summary\", which has the summary as the value.\nYour output should be in the following format:\n\n{\"summary\": \"xx\"}\n\nWhere \"xx\" is replaced by the summary.\nreply with valid JSON only",
"stream": true
}
흥미롭게도, Sonnet은 뒤집힌 텍스트가 규칙을 회피하거나 숨기려는 시도라고 판단하여 많은 주제의 요약을 거부합니다.
현재 3.6.0.beta3-latest 버전으로 업데이트되어 있습니다.
