Logos proxy drops SSE event delimiters and returns empty streams to OpenAI clients

[bassner]

Summary

Logos corrupts Server-Sent Events (SSE) framing while proxying streaming model responses. The proxy reads the upstream response with httpx.Response.aiter_lines(), discards empty lines, and emits each remaining line with only one trailing newline.

SSE events are delimited by a blank line (\n\n). Removing empty lines therefore merges the upstream events into an unterminated event. OpenAI-compatible clients can receive HTTP 200 while yielding zero parsed chunks, even though the upstream model generated a response and Logos recorded its token usage.

Non-streaming requests are unaffected.

User-visible impact

This breaks streamed Iris chat responses when Pyris accesses models through Logos:

1. Artemis requests a streamed chat execution from Pyris.
2. Pyris sends a request with stream: true to Logos.
3. Logos successfully invokes the upstream provider and records generated output tokens.
4. Logos removes the SSE event delimiters while proxying the response.
5. The OpenAI SDK in Pyris sees an empty iterator: no content deltas, tool-call fragments, finish reason, or usage chunk.
6. Pyris completes the agent step with an empty output and Artemis displays an empty assistant response.

The problem is not model-specific. It affects proxied streaming responses on both API surfaces:

• /v1/chat/completions
• /v1/responses

It also affects tool-calling streams, because tool-call deltas use the same SSE framing.

Evidence

The behavior is reproducible with the OpenAI Python SDK:

stream = client.chat.completions.create(
    model="<allowed-model>",
    messages=[{"role": "user", "content": "Reply with OK."}],
    stream=True,
    stream_options={"include_usage": True},
)

chunks = list(stream)
assert chunks  # currently fails: chunks == []

The equivalent Responses API request also yields no events:

stream = client.responses.create(
    model="<allowed-model>",
    input="Reply with OK.",
    stream=True,
    store=False,
)

assert list(stream)  # currently fails

At the same time, Logos's upstream execution and request accounting show a successful streaming request with non-zero output tokens. This localizes the loss to the downstream proxying/framing path rather than authentication, model execution, or upstream streaming.

Root cause

The current implementation in logos/logos-orchestrator/src/logos/pipeline/executor.py uses line iteration and skips blank lines:

async for line in resp.aiter_lines():
    if line:
        yield (line + "\n").encode()

For a valid upstream stream such as:

data: {"choices":[{"delta":{"content":"Hello"}}]}

data: {"choices":[{"delta":{"content":" world"}}]}

data: [DONE]

aiter_lines() removes line endings, and the if line condition removes the empty delimiter lines. The downstream body consequently becomes equivalent to:

data: {...}
data: {...}
data: [DONE]

Without blank event separators, standards-compliant SSE parsers do not dispatch the individual events. The OpenAI SDK consequently completes the HTTP response without yielding any model chunks.

Proposed fix

Forward the upstream byte stream without reconstructing its line framing:

async for chunk in resp.aiter_bytes():
    if chunk:
        yield chunk

This preserves:

• blank SSE event delimiters;
• event: lines used by the Responses API;
• arbitrary network chunk boundaries;
• UTF-8 data across chunks;
• the upstream [DONE] marker.

Commit 917bf658 already contains this specific change, but it is part of a larger Whisper-focused commit. The SSE fix and its regression coverage should be landed independently so it can be reviewed and deployed without unrelated behavior changes.

Required regression tests

Add focused tests for the proxy executor and, ideally, an SDK-level integration test:

1. Provide an upstream byte stream containing multiple data: events separated by \n\n.
2. Split the input across arbitrary byte boundaries, including inside an event and around delimiter bytes.
3. Assert that the proxy output is byte-for-byte identical to the upstream stream.
4. Assert that blank lines and data: [DONE]\n\n are preserved.
5. Cover Responses API streams containing event: plus data: lines.
6. Parse the proxied result with the OpenAI SDK or an SSE parser and assert that at least one content/tool event and the terminal event are observed.
7. Retain coverage for upstream HTTP errors and mid-stream failures.

Expected behavior

A streaming request through Logos must expose the same sequence of parseable SSE events as the upstream provider. Clients must receive content deltas, tool-call deltas, terminal events/finish reasons, and usage data where supplied by the provider.

Acceptance criteria

• Logos preserves upstream SSE bytes and blank event delimiters.
• OpenAI Chat Completions clients receive non-empty streamed chunks through Logos.
• OpenAI Responses clients receive non-empty streamed events through Logos.
• Tool-calling streams remain parseable.
• Non-streaming requests remain unchanged.
• Upstream and mid-stream error handling remains covered.
• An end-to-end streamed Iris chat through Pyris and Logos produces a visible assistant response in Artemis.