Non-streaming request failed

Discourse v2026.7.5 — Commits · discourse/discourse · GitHub — Ember v6.10.1

How do I configure streaming correctly?

Non-streaming request failed: {"error":{"code":400,"message":"Unknown parameter: 'messages'.Expects 'input' instead of 'messages'.","type":"InvalidParameter"}}


לייק 1

As seen in your screenshots, the problem is that the vLLM endpoint expects input, while Discourse is making messages.

Solution:

  1. Keep Provider as vLLM.
  2. Change the API URL to the OpenAI-compatible endpoint of your provider for Chat Completions (/v1 is the base URL in most cases).
  3. Make sure the endpoint supports /v1/chat/completions.
  4. Leave streaming on if available.
  5. Click Save and Run Test.
    Important note: Turning off streaming will not solve this problem. The API request format should be correct.

If your gateway supports /responses only, you need a corresponding endpoint or an adapter.

The issue now is that I have configured it on my end, but Discourse is not sending streaming requests. My service supports streaming requests. This seems to be a Discourse issue.

As your vLLM server is stream-enabled, test Discourse’s AI integration to see if:

  • It correctly sends stream: true.

  • The validation error for the non-streaming fails the streaming test too.

  • It uses the correct stream response parser for the SSE format.

    This might need to be addressed in Discourse’s LLM requests codebase.

לייק 1

Title: In SSE, streaming completion parsing fails when there is no space after data:, resulting in “model returned an empty response”

Problem Overview

DiscourseAi::Completions::JsonStreamDecoder only recognizes the SSE line format data: {...} (with a single space after the colon). For providers that output the compact format data:{...}, all chunks are silently discarded, causing the same endpoint to work correctly for non-streaming requests but report “model returned an empty response” for streaming requests.

According to the SSE specification, the single leading space after the colon is optional and is stripped if present. data:{...} is fully compliant. OpenAI officially sends data: {...}, so this issue was never exposed, but many OpenAI-compatible gateways send the compact format.

Environment

  • discourse-ai plugin, branch release/2026.7
  • Provider: vllm (also affects open_ai and all endpoints using the default LINE_REGEX)
  • OpenAI-compatible Chat Completions endpoint

Symptoms

When clicking “Test” in /admin/plugins/discourse-ai/ai-llms/:id/edit, the following error occurs:

Streaming request failed: Model returned an empty response.

The non-streaming test passed. LlmValidator#run_test tests non-streaming first, then streaming, so last_failed_mode remains set to :streaming.

Root Cause

plugins/discourse-ai/lib/completions/json_stream_decoder.rb:9

LINE_REGEX = /data: ({.*})\s*$/
#             ^ The space here is mandatory

In the #<< method, lines that do not match are skipped via next if !matches. With the compact format, no lines match, so no chunks are parsed, and the accumulated streaming result is naturally empty.

Non-streaming requests are unaffected because they use decode, which directly parses the entire body with JSON.parse, completely bypassing this decoder.

Reproduction

Feed a raw compact-format stream directly into the decoder:

raw = <<~SSE
  data:{"id":"chatcmpl-1","choices":[{"index":0,"delta":{"content":"2"}}]}

  data:{"id":"chatcmpl-1","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

  data:[DONE]
SSE

DiscourseAi::Completions::JsonStreamDecoder.new.<<(raw).size
# => 0   (expected is 2)

Testing with a 3583-byte compact SSE response from a certain gateway: the current regex parses 0 chunks, while making the space optional parses 13 chunks.

Fix Proposal

--- a/plugins/discourse-ai/lib/completions/json_stream_decoder.rb
+++ b/plugins/discourse-ai/lib/completions/json_stream_decoder.rb
@@ -6,7 +6,7 @@ module DiscourseAi
     class JsonStreamDecoder
       attr_reader :buffer

-      LINE_REGEX = /data: ({.*})\s*$/
+      LINE_REGEX = /data:\s*({.*})\s*$/

The impact is minimal: only cohere.rb and ollama.rb pass a custom line_regex, and neither uses the default value. All other endpoints consume standard SSE, and accepting an optional space brings the implementation closer to the specification.

Accompanying test case:

it "should handle streams that omit the space after data:" do
  stream = <<~TEXT.split("|")
    data:{"hel|lo": "world"}

    data:{"foo": "bar"}

    data:{"ba|z": "qux"|}

    data:[DONE]
  TEXT

  results = []
  stream.each { |chunk| results << (decoder << chunk) }

  expect(results.flatten.compact).to eq([{ hello: "world" }, { foo: "bar" }, { baz: "qux" }])
end

Locally verified: All decoder specs pass; end-to-end LlmValidator#run_test against the problematic provider passes both non-streaming and streaming rounds.

Temporary Workaround (No Code Changes)

Check Disable streaming completions in the LLM configuration. Endpoints::Base#perform will then use replay_non_streaming_as_streaming!, completely bypassing the decoder. The trade-off is losing true token-by-token streaming output—long responses will not appear until fully generated.

Another Side Issue Discovered

If your gateway exposes the Responses API at a path that does not contain the substring /v1/responses, OpenAiResponses.can_contact? will not match, and Discourse will fall back to Chat Completions, resulting in the following error:

Unknown parameter: 'messages'. Expects 'input' instead of 'messages'.

The check in open_ai_responses.rb:9-11 performs a substring match on the URL, so paths like /api/router/responses are missed. Pointing the LLM URL to the gateway’s chat/completions path resolves this.

FIX: Accept SSE `data:` lines without a space in `JsonStreamDecoder` by singi2016cn · Pull Request #46447 · discourse/discourse

The root cause is confirmed: Discourse’s JsonStreamDecoder requires a space after data: in SSE responses.

Permanent fix: Update json_stream_decoder.rb:
</> Ruby

LINE_REGEX = /data:\s*({.})\s$/

This allows both data: {…} and data:{…} formats.

Temporary workaround: Enable Disable streaming completions in the LLM configuration. This bypasses the decoder but disables real-time token streaming. #PM

I was need to use a custom injection script via LiteLLM in order to use Qwen models in Discourse. If it is possible to you, I can share it.

@singi2016cn Here you go, it is scoped to their translate models. Note it was vibe-coded and I hope you understand it and can improve and adapt it to your actual use-case:

import json
import logging
from litellm.integrations.custom_logger import CustomLogger

QWEN_MT_MODEL_NAMES = {"qwen-translate"}

def extract_translation_request(messages):
    """
    discourse-ai sends multi-turn few-shot examples followed by the real
    request as the LAST user message, JSON-encoded as:
    {"content": "<text to translate>", "target_locale": "<code>"}
    Returns (text_to_translate, target_locale) or (None, None) if not found.
    """
    user_messages = [m for m in messages if m.get("role") == "user"]
    if not user_messages:
        return None, None

    last = user_messages[-1]
    content = last.get("content", "")

    try:
        parsed = json.loads(content)
        text = parsed.get("content")
        locale = parsed.get("target_locale")
        if text and locale:
            return text, locale
    except (json.JSONDecodeError, TypeError):
        pass

    return None, None

class QwenMTHook(CustomLogger):
    async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
        model = data.get("model", "")

        if model not in QWEN_MT_MODEL_NAMES:
            return data

        messages = data.get("messages", [])
        text, target_locale = extract_translation_request(messages)

        if text is None:
            logging.warning(
                "[QwenMT] could not extract translation request, passing through unmodified"
            )
            return data

        # Qwen-MT: exactly one user message, no system/few-shot examples
        data["messages"] = [{"role": "user", "content": text}]

        # Strip response_format — Qwen-MT can't honor it, and DashScope
        # rejects json_object mode unless "json" appears in the prompt
        data.pop("response_format", None)

        extra_body = data.get("extra_body") or {}
        extra_body["translation_options"] = {
            "source_lang": "auto",
            "target_lang": target_locale,
        }
        data["extra_body"] = extra_body

        # Flag for post-call hook so we know to wrap the plain response
        metadata = data.get("metadata") or {}
        metadata["qwen_mt_wrap_output"] = True
        data["metadata"] = metadata

        logging.warning(
            "[QwenMT] translating to=%s text=%.80s",
            target_locale,
            text,
        )
        return data

    async def async_post_call_success_hook(self, data, user_api_key_dict, response):
        metadata = data.get("metadata") or {}
        if not metadata.get("qwen_mt_wrap_output"):
            return response

        try:
            translated_text = response.choices[0].message.content
            wrapped = json.dumps({"output": translated_text})
            response.choices[0].message.content = wrapped
            logging.warning("[QwenMT] wrapped output=%.80s", wrapped)
        except (AttributeError, IndexError) as e:
            logging.warning("[QwenMT] failed to wrap response: %s", e)

        return response

proxy_handler_instance = QwenMTHook()
לייק 1