Skip to content

Add :sse_middleware option to allow custom SSE middleware for streaming#119

Merged
nvelasco merged 5 commits into
masterfrom
nvelasco/add-sse-middleware-option
Jul 1, 2026
Merged

Add :sse_middleware option to allow custom SSE middleware for streaming#119
nvelasco merged 5 commits into
masterfrom
nvelasco/add-sse-middleware-option

Conversation

@nvelasco

Copy link
Copy Markdown
Contributor

Problem

Tesla.Middleware.SSE assumes each HTTP chunk is a complete, parseable SSE message. Some LLM providers occasionally send partial chunks (e.g. a bare {) as the first HTTP frame of a streaming response, causing it to crash with:

  %FunctionClauseError{module: Tesla.Middleware.SSE, function: :decode_body, arity: 1}

Change

Introduces a :sse_middleware option on LlmComposer.Settings that allows consumers to inject a custom SSE middleware instead of the default Tesla.Middleware.SSE. The value is forwarded through ProvidersRunner to HttpClient where the middleware stack is assembled.

Default behaviour is preserved — if :sse_middleware is not set, Tesla.Middleware.SSE is used as before.

Usage

  settings = %LlmComposer.Settings{
    stream_response: true,
    sse_middleware: [{MyApp.CustomSSEMiddleware, only: :data}],
    providers: [{LlmComposer.Providers.Google, [model: "gemini-flash"]}]
  }

@nvelasco nvelasco self-assigned this Jun 30, 2026
@nvelasco
nvelasco force-pushed the nvelasco/add-sse-middleware-option branch from 6d28060 to 09b1f47 Compare June 30, 2026 13:33
@nvelasco
nvelasco requested review from dmoralesl and sonic182 June 30, 2026 15:55
Comment thread CHANGELOG.md
@nvelasco

nvelasco commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

I'm going to recreate the PR
Don't know why but the CI doesn't end

@nvelasco nvelasco closed this Jul 1, 2026
@nvelasco nvelasco reopened this Jul 1, 2026
@sonic182

sonic182 commented Jul 1, 2026

Copy link
Copy Markdown
Member

if something doesn't work, just restart (close and reopen pr) 😄

@nvelasco
nvelasco merged commit dcb79c2 into master Jul 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants