# JSON 流解析中断时翻译被静默截断（未引发错误）

**URL:** <https://meta.discourse.org/t/translation-silently-truncated-when-json-stream-parsing-breaks-no-error-raised/407251>\
**Category:** Bug\
**Tags:** ai, dynaloc\
**Created:** [2026年七月10日 09:31 UTC](https://meta.discourse.org/t/translation-silently-truncated-when-json-stream-parsing-breaks-no-error-raised/407251 "2026-07-10T09:31:25Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)\
**Post date:** [2026年七月14日 23:39 UTC](https://meta.discourse.org/t/translation-silently-truncated-when-json-stream-parsing-breaks-no-error-raised/407251/4 "2026-07-14T23:39:19Z")

</div>

开始着手修复

> <https://github.com/discourse/discourse/pull/41716>
>
> Reported at https://meta.discourse.org/t/translation-silently-truncated-when-jso…n-stream-parsing-breaks-no-error-raised/407251
> 
> \## The bug
> 
> Some providers stream structured output whose string values were unescaped by an outer JSON parse, so real newlines appear inside string values. When that happened, \`JsonStreamingTracker\` had two failure modes:
> 
> \- It marked the stream broken and \`StructuredOutput\` fell back to \`BestEffortJsonParser\`, whose extraction regex (\`\[^"\]+\`) cut the value at the first escaped quote and left \`\\n\` sequences as literal text. A 2,000-char translation could come back as ~50 chars, cut right before the first quoted word — exactly what the report shows.
> \- Its escape-and-resume hack (\`String#dump\` + buffer-growth offset) miscomputed the resume index whenever the chunk contained non-ASCII, quotes, or backslashes, silently duplicating or corrupting content \*\*without ever marking the stream broken\*\*.
> 
> Testing a realistic corrupted payload across chunk sizes 1–60: 31 produced the truncated fallback, 27 produced silent corruption, 1 raised, and only 1 came out correct. Either way the result was persisted as a successful translation with nothing in the logs.
> 
> \## The fix
> 
> Replace the hand-rolled parsing with two gems and keep only glue:
> 
> \- \*\*json\_completer\*\* (pure Ruby): \`JsonStreamingTracker\` now feeds the cumulative buffer — with control characters re-escaped — to an incremental, truncation-tolerant parser and notifies consumers of changed keys. The corrupted payloads above stream correctly at every chunk size, so the broken-stream path is only reached for responses that aren't JSON at all.
> \- \*\*smarter\_json\*\*: \`BestEffortJsonParser\` becomes a three-attempt chain (strict-with-completion → control-chars re-escaped → lenient) covering the quirk shapes the old regexes handled: single quotes, unquoted keys, markdown fences, prose-wrapped JSON.
> 
> This deletes the vendored 668-line SAX parser, the resume hack, and all manual regex extraction (net −694 lines), and adds a log warning whenever a response falls back to best-effort parsing.
> 
> \## Behavior changes
> 
> \- Scalars now stream progressively: mid-stream \`read\_buffered\_property(:number)\` returns the digits buffered so far instead of \`nil\`. Consumers act on final values, so this only affects mid-stream reads.
> \- Arrays of objects stream partial objects mid-stream instead of returning \`nil\` until finish.
> \- A trailing comma in an array reads as a \`nil\` placeholder slot until the next element arrives.
> \- Partial tool calls surface a few more progressive updates (the openai endpoint spec count moved 128 → 134); values still only ever grow.
> 
> \## Tests
> 
> \- Regression specs for the report: unescaped control characters with escaped quotes/emoji streamed across chunk boundaries, fenced + unescaped responses, truncated JSON, numeric casting.
> \- 916 examples green across \`completions/\`, \`translation/\`, \`modules/ai\_helper/\`, and \`utils/\`.

---

_[View the full topic](https://meta.discourse.org/t/translation-silently-truncated-when-json-stream-parsing-breaks-no-error-raised/407251)._