Talk: add gateway TTS fallback when ElevenLabs is not configured#1
Talk: add gateway TTS fallback when ElevenLabs is not configured#1boydyngo wants to merge 1 commit into
Conversation
When no ElevenLabs API key is set, Talk mode now calls the gateway's talk.speak method (which supports Edge TTS, Kokoro, and other speech providers) instead of falling directly to the macOS system voice. This gives users access to natural-sounding neural voices (e.g. Microsoft Edge TTS) without requiring an ElevenLabs subscription. The macOS system voice remains as a last-resort fallback if the gateway call fails. Flow: ElevenLabs key → ElevenLabs | No key → gateway talk.speak | fail → system voice Co-Authored-By: boydyngo <boydyngo@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a gateway-backed TTS fallback for Talk mode so users without an ElevenLabs API key can still get neural voices via the gateway’s talk.speak, with macOS system voice as the final fallback.
Changes:
- Route the no-ElevenLabs path to a new
playGatewayTTS()method callingtalk.speak. - Add gateway TTS response decoding + audio playback via
TalkAudioPlayer. - Switch Talk config loading from
talk.configtoconfig.get.
|
|
||
| let result = await TalkAudioPlayer.shared.play(data: audioData) | ||
| self.ttsLogger.info( | ||
| "talk gateway TTS done finished=\(result.finished, privacy: .public)") |
There was a problem hiding this comment.
playGatewayTTS ignores the playback outcome: if TalkAudioPlayer fails to decode/play the returned audio, it returns finished=false but this method still returns successfully, so the caller will not fall back to system voice. Consider treating finished == false as an error (throw) so the existing catch path can trigger the last-resort fallback, and also ensure this playback path is interruptible (currently stopSpeaking only stops the streaming players and the system synthesizer, not TalkAudioPlayer).
| "talk gateway TTS done finished=\(result.finished, privacy: .public)") | |
| "talk gateway TTS done finished=\(result.finished, privacy: .public)") | |
| guard result.finished else { | |
| guard self.isCurrent(input.generation) else { return } | |
| throw NSError( | |
| domain: "TalkGatewayTTS", code: 2, | |
| userInfo: [NSLocalizedDescriptionKey: "gateway audio playback did not finish"]) | |
| } |
| if let apiKey = input.apiKey, !apiKey.isEmpty, let voiceId = input.voiceId { | ||
| try await self.playElevenLabs(input: input, apiKey: apiKey, voiceId: voiceId) | ||
| } else { | ||
| try await self.playSystemVoice(input: input) | ||
| try await self.playGatewayTTS(input: input) | ||
| } |
There was a problem hiding this comment.
This fallback now routes through gateway TTS, but some of the surrounding logging still says it is “falling back to system voice” when the ElevenLabs API key/voiceId is missing. Updating those warnings to mention gateway TTS (with system voice as the final fallback) will make diagnostics match runtime behavior.
| let snap: ConfigSnapshot = try await GatewayConnection.shared.requestDecoded( | ||
| method: .talkConfig, | ||
| params: ["includeSecrets": AnyCodable(true)], | ||
| method: .configGet, | ||
| params: nil, | ||
| timeoutMs: 8000) |
There was a problem hiding this comment.
Switching from talk.config (with includeSecrets) to config.get may change authorization/shape/secret-redaction semantics. In particular, talk.config is already used elsewhere to retrieve talk.apiKey when includeSecrets is true (e.g. apps/ios/Sources/Voice/TalkModeManager.swift:1674), but config.get has no includeSecrets parameter in the protocol model. If config.get returns redacted secrets or is restricted in some modes, this can break ElevenLabs configuration loading. Consider keeping talk.config for Talk mode config (and only fetching additional UI keys via config.get if needed).
Summary
talk.speakmethod instead of falling directly to the macOS system voiceMotivation
The current Talk mode only has two TTS paths: ElevenLabs (paid) or macOS system voice (robotic). Users who don't have an ElevenLabs API key are stuck with AVSpeechSynthesizer which sounds mechanical. The gateway already supports multiple free speech providers via talk.speak — this change wires Talk mode to use them.
Changes
apps/macos/Sources/OpenClaw/TalkModeRuntime.swift — 1 file, +44/-3 lines
TTS flow after this change
ElevenLabs API key set? -> ElevenLabs (unchanged)
No key? -> Gateway talk.speak (Edge TTS / Kokoro / configured provider)
Gateway fails? -> macOS system voice (last resort)
Test plan
Ready for upstream submission to openclaw/openclaw.
Generated with Claude Code