feat(realtime): stream the LLM / TTS / transcription pipeline stages#10176

Open

localai-bot wants to merge 15 commits into

feat/realtime-streaming

Commits on Jun 5, 2026

feat(realtime): pipeline streaming + disable_thinking config
mudler
committed
feat(realtime): sentence segmenter for streamed LLM->TTS pipelining
mudler
committed
feat(realtime): streaming TTS/transcription methods on Model interface
mudler
committed
feat(realtime): emitSpeech with flag-gated streaming TTS
mudler
committed
feat(realtime): route response audio through emitSpeech (streaming TTS)
mudler
committed
feat(realtime): streaming transcription text deltas
mudler
committed
feat(realtime): pipeline disable_thinking maps to enable_thinking off
mudler
committed
feat(realtime): speechStreamer for token-streamed LLM->TTS
mudler
committed
feat(realtime): wire streamLLMResponse for token-streamed replies
mudler
committed
docs(realtime): document pipeline streaming + disable_thinking
mudler
committed
fix(realtime): register pipeline streaming/thinking config fields
mudler
committed
fix(realtime): always strip reasoning from spoken output
mudler
committed
fix(realtime): clean TTS temp path before read (gosec G304)
mudler
committed
refactor(realtime): buffer whole message for TTS, drop sentence segmenter
mudler
committed
feat(realtime): stream tool-call turns via tokenizer-template autoparser
mudler
committed