Title: In SSE, streaming completion parsing fails when there is no space after data:, resulting in “model returned an empty response”
Problem Overview
DiscourseAi::Completions::JsonStreamDecoder only recognizes the SSE line format data: {...} (with a single space after the colon). For providers that output the compact format data:{...}, all chunks are silently discarded, causing the same endpoint to work correctly for non-streaming requests but report “model returned an empty response” for streaming requests.
According to the SSE specification, the single leading space after the colon is optional and is stripped if present. data:{...} is fully compliant. OpenAI officially sends data: {...}, so this issue was never exposed, but many OpenAI-compatible gateways send the compact format.
Environment
- discourse-ai plugin, branch
release/2026.7
- Provider:
vllm (also affects open_ai and all endpoints using the default LINE_REGEX)
- OpenAI-compatible Chat Completions endpoint
Symptoms
When clicking “Test” in /admin/plugins/discourse-ai/ai-llms/:id/edit, the following error occurs:
Streaming request failed: Model returned an empty response.
The non-streaming test passed. LlmValidator#run_test tests non-streaming first, then streaming, so last_failed_mode remains set to :streaming.
Root Cause
plugins/discourse-ai/lib/completions/json_stream_decoder.rb:9
LINE_REGEX = /data: ({.*})\s*$/
# ^ The space here is mandatory
In the #<< method, lines that do not match are skipped via next if !matches. With the compact format, no lines match, so no chunks are parsed, and the accumulated streaming result is naturally empty.
Non-streaming requests are unaffected because they use decode, which directly parses the entire body with JSON.parse, completely bypassing this decoder.
Reproduction
Feed a raw compact-format stream directly into the decoder:
raw = <<~SSE
data:{"id":"chatcmpl-1","choices":[{"index":0,"delta":{"content":"2"}}]}
data:{"id":"chatcmpl-1","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data:[DONE]
SSE
DiscourseAi::Completions::JsonStreamDecoder.new.<<(raw).size
# => 0 (expected is 2)
Testing with a 3583-byte compact SSE response from a certain gateway: the current regex parses 0 chunks, while making the space optional parses 13 chunks.
Fix Proposal
--- a/plugins/discourse-ai/lib/completions/json_stream_decoder.rb
+++ b/plugins/discourse-ai/lib/completions/json_stream_decoder.rb
@@ -6,7 +6,7 @@ module DiscourseAi
class JsonStreamDecoder
attr_reader :buffer
- LINE_REGEX = /data: ({.*})\s*$/
+ LINE_REGEX = /data:\s*({.*})\s*$/
The impact is minimal: only cohere.rb and ollama.rb pass a custom line_regex, and neither uses the default value. All other endpoints consume standard SSE, and accepting an optional space brings the implementation closer to the specification.
Accompanying test case:
it "should handle streams that omit the space after data:" do
stream = <<~TEXT.split("|")
data:{"hel|lo": "world"}
data:{"foo": "bar"}
data:{"ba|z": "qux"|}
data:[DONE]
TEXT
results = []
stream.each { |chunk| results << (decoder << chunk) }
expect(results.flatten.compact).to eq([{ hello: "world" }, { foo: "bar" }, { baz: "qux" }])
end
Locally verified: All decoder specs pass; end-to-end LlmValidator#run_test against the problematic provider passes both non-streaming and streaming rounds.
Temporary Workaround (No Code Changes)
Check Disable streaming completions in the LLM configuration. Endpoints::Base#perform will then use replay_non_streaming_as_streaming!, completely bypassing the decoder. The trade-off is losing true token-by-token streaming output—long responses will not appear until fully generated.
Another Side Issue Discovered
If your gateway exposes the Responses API at a path that does not contain the substring /v1/responses, OpenAiResponses.can_contact? will not match, and Discourse will fall back to Chat Completions, resulting in the following error:
Unknown parameter: 'messages'. Expects 'input' instead of 'messages'.
The check in open_ai_responses.rb:9-11 performs a substring match on the URL, so paths like /api/router/responses are missed. Pointing the LLM URL to the gateway’s chat/completions path resolves this.
FIX: Accept SSE `data:` lines without a space in `JsonStreamDecoder` by singi2016cn · Pull Request #46447 · discourse/discourse