diff --git a/CHANGELOG.md b/CHANGELOG.md index 137375b..14e17af 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.20.1] - 2026-07-01 + +### Added + +- Added `:sse_middleware` option for streaming responses, allowing consumers to inject a custom SSE middleware instead of the default `Tesla.Middleware.SSE`. + ## [0.20.0] - 2026-06-29 ### Added @@ -297,10 +303,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), - **Include Bedrock Support**: Included Bedrock support as provider only with `completion()`s support. ## [0.3.5] - 2024-12-18 -- **Fix default api_key setting value**: The default value was en empty string. Now it is nil to be evaluated as false when getting the key from the map later. +- **Fix default api_key setting value**: The default value was en empty string. Now it is nil to be evaluated as false when getting the key from the map later. ## [0.3.4] - 2024-12-17 -- **OpenAI API Keys**: Use an API key passed as a parameter when calling chat_completion — overriding the global API key defined in the config. The param is sent inside the settings. +- **OpenAI API Keys**: Use an API key passed as a parameter when calling chat_completion — overriding the global API key defined in the config. The param is sent inside the settings. ## [0.3.3] - 2024-12-04 - **Timeouts**: Configurable OpenAI's timeout. Default set to 50 seconds. @@ -327,6 +333,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), - Initial release with support for basic message handling, interaction with OpenAI and Ollama models, and a foundational structure for model settings and function execution. --- +[0.20.1]: https://github.com/doofinder/llm_composer/compare/0.20.0...0.20.1 [0.20.0]: https://github.com/doofinder/llm_composer/compare/0.19.6...0.20.0 [0.19.6]: https://github.com/doofinder/llm_composer/compare/0.19.5...0.19.6 [0.19.5]: https://github.com/doofinder/llm_composer/compare/0.19.4...0.19.5 diff --git a/lib/llm_composer/http_client.ex b/lib/llm_composer/http_client.ex index 505d3fb..963e751 100644 --- a/lib/llm_composer/http_client.ex +++ b/lib/llm_composer/http_client.ex @@ -47,7 +47,7 @@ defmodule LlmComposer.HttpClient do cond do stream -> - resp ++ [{Tesla.Middleware.SSE, only: :data}] + resp ++ Keyword.get(opts, :sse_middleware, [{Tesla.Middleware.SSE, only: :data}]) retries_disabled?(opts) -> resp ++ diff --git a/lib/llm_composer/providers_runner.ex b/lib/llm_composer/providers_runner.ex index 72dbfc1..f748315 100644 --- a/lib/llm_composer/providers_runner.ex +++ b/lib/llm_composer/providers_runner.ex @@ -115,8 +115,14 @@ defmodule LlmComposer.ProvidersRunner do |> Keyword.put_new(:stream_response, settings.stream_response) |> Keyword.put_new(:track_costs, settings.track_costs) |> Keyword.put_new(:api_key, settings.api_key) + |> maybe_put_sse_middleware(settings) end + defp maybe_put_sse_middleware(opts, %{sse_middleware: nil}), do: opts + + defp maybe_put_sse_middleware(opts, %{sse_middleware: middleware}), + do: Keyword.put_new(opts, :sse_middleware, middleware) + defp get_provider_router do :llm_composer |> Application.get_env(:provider_router, []) diff --git a/lib/llm_composer/settings.ex b/lib/llm_composer/settings.ex index 906c744..c98538b 100644 --- a/lib/llm_composer/settings.ex +++ b/lib/llm_composer/settings.ex @@ -9,6 +9,7 @@ defmodule LlmComposer.Settings do provider: nil, provider_opts: nil, providers: nil, + sse_middleware: nil, stream_response: false, system_prompt: nil, track_costs: false, @@ -21,6 +22,7 @@ defmodule LlmComposer.Settings do - `:provider` — provider module (e.g. `LlmComposer.Providers.OpenAI`). Used with `:provider_opts`. - `:provider_opts` — keyword list of options forwarded to the single provider. - `:providers` — list of `{module, opts}` tuples for multi-provider routing via `ProvidersRunner`. + - `:sse_middleware` — custom Tesla SSE middleware list to use instead of the default `Tesla.Middleware.SSE` when `stream_response: true`. Useful when the provider sends incomplete SSE chunks that would crash the default middleware. - `:stream_response` — when `true`, the provider returns a streaming enumerable. - `:system_prompt` — system prompt string sent to the model. - `:track_costs` — when `true`, token usage and cost are computed and attached to the response. @@ -31,6 +33,7 @@ defmodule LlmComposer.Settings do provider: module() | nil, provider_opts: keyword() | nil, providers: [{module(), keyword()}] | nil, + sse_middleware: [{module(), keyword()}] | nil, stream_response: boolean(), system_prompt: String.t() | nil, track_costs: boolean(), diff --git a/mix.exs b/mix.exs index 78827a8..d799c26 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule LlmComposer.MixProject do def project do [ app: :llm_composer, - version: "0.20.0", + version: "0.20.1", elixir: "~> 1.18", start_permanent: Mix.env() == :prod, deps: deps(),