diff --git a/src/langsmith/trace-gemini-live.mdx b/src/langsmith/trace-gemini-live.mdx index 485a34e059..d79795f53c 100644 --- a/src/langsmith/trace-gemini-live.mdx +++ b/src/langsmith/trace-gemini-live.mdx @@ -4,70 +4,61 @@ sidebarTitle: Gemini Live description: Trace Gemini Live voice agents built with the Google Agent Development Kit (ADK) in LangSmith. --- -The [Gemini Live API](https://ai.google.dev/gemini-api/docs/live-api) enables low-latency, bidirectional voice interactions with Gemini models over a persistent WebSocket connection. This guide shows how to trace a Gemini Live voice agent built with the [Google Agent Development Kit (ADK)](https://google.github.io/adk-docs/streaming/) to LangSmith. - -Gemini Live is a speech-to-speech model: it processes audio natively and exchanges a continuous stream of events with your application over a persistent WebSocket connection, rather than making discrete request/response calls. The following sections show those events and how to turn them into a LangSmith trace. For our high-level principles on getting the most out of your voice agent traces, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). - -## The ADK Live event model - -As the conversation runs, ADK streams a series of events to your application. Each event reports something that happened in the conversation: a chunk of audio, a transcript fragment, a tool call, a turn boundary, or an interruption. Every event has the same shape, and most of its fields are optional, so you determine what an event represents from **which fields are populated**: - -| Populated field | Meaning | -| --- | --- | -| `content.parts[*].inline_data` | A chunk of agent audio (PCM16 bytes). The agent's voice arrives as a flood of these. | -| `input_transcription` | A fragment of the *user's* speech transcript. A final event repeats the full utterance with `finished=True`. | -| `output_transcription` | A fragment of the *agent's* speech transcript. | -| `content.parts[*].function_call` | The model requested a tool (name and arguments). | -| `content.parts[*].function_response` | ADK executed the tool and is returning the result to the model. | -| `turn_complete` | The server finished its half of the exchange. | -| `interrupted` | The server detected user barge-in over the agent. Flush your speaker buffer. | - -## How events map to LangSmith runs - -To get the most out of your traces, capture each meaningful event and the data it contains in a single conversation trace, with one span per event: - -```text -conversation ← root run (combined audio recording; ls_modality="audio") -│ metadata: thread_id, model, event_count, duration_s -│ -├─ input_transcription ← a fragment of the user's speech transcript -├─ output_transcription ← a fragment of the agent's speech transcript -├─ function_call: get_weather ← the model requested the tool -├─ function_response: get_weather ← ADK ran the tool; result heading back -├─ turn_complete ← turn boundary -└─ interrupted ← barge-in -``` +Trace your [Gemini Live](https://ai.google.dev/gemini-api/docs/live-api) voice agents, built with the [Google Agent Development Kit (ADK)](https://google.github.io/adk-docs/streaming/), to LangSmith with the LangSmith ADK integration. For high-level conventions, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). -## Installation + +The ADK Live integration requires `langsmith[google-adk]`. To trace ADK's non-streaming paths (text agents, tools, multi-agent workflows), see [Trace Google ADK applications](/langsmith/trace-with-google-adk). + -```bash -pip install "google-adk>=2.0" google-genai "langsmith>=0.4" -``` +Gemini Live is a speech-to-speech model that ADK streams to your app as `run_live` events. The integration captures each conversation as a single LangSmith trace, with a span for every meaningful event (transcripts, tool calls, turn boundaries, and interruptions). You register one plugin and do not create any spans yourself. The flood of audio chunks is played but not traced. -Install `sounddevice` and `numpy` as well if you want to capture local audio and attach the conversation recording. + +`Runner.run_live` is not covered by ADK's standard OpenTelemetry instrumentation, so this plugin is what produces a trace for a Live voice conversation. + -## Set up your environment +## Install -The following steps demonstrate how to trace using the LangSmith SDK. You can also trace using OpenTelemetry directly. See [Trace with OpenTelemetry](/langsmith/trace-with-opentelemetry). + -```bash -export LANGSMITH_API_KEY=... -export LANGSMITH_TRACING=true -export LANGSMITH_PROJECT=my-voice-app -export GOOGLE_API_KEY=... +```bash pip +pip install "langsmith[google-adk]" google-genai ``` -## Quickstart +```bash uv +uv add "langsmith[google-adk]" google-genai +``` - -This guide focuses on the tracing layer. It assumes you already have a working ADK Live app: the `LlmAgent`, `Runner`, and `LiveRequestQueue` that produce the `run_live` event stream, plus your microphone and speaker I/O. For a complete, runnable implementation of all of that, see the [voice demo repository](https://github.com/langchain-ai/voice-demo/tree/main/src/voice_demo/adk). - + + +Voice apps also need an audio library such as `sounddevice` for microphone and speaker I/O. + +## Set environment variables -### Step 1: Build the RunConfig +```bash .env +LANGSMITH_API_KEY= +LANGSMITH_TRACING=true +LANGSMITH_PROJECT=gemini-live-voice +GOOGLE_API_KEY= +``` + +## Set up tracing + +Import `LangSmithLivePlugin` and register it on your `Runner`. It runs alongside your own `run_live` loop, so your loop keeps doing only audio playback and UI: ```python from google.adk.agents.run_config import RunConfig, StreamingMode +from google.adk.runners import Runner from google.genai import types as genai_types +from langsmith.integrations.google_adk import LangSmithLivePlugin + +plugin = LangSmithLivePlugin(project_name="gemini-live-voice") + +runner = Runner( + app_name="voice-app", + agent=root_agent, + session_service=session_service, + plugins=[plugin], +) run_config = RunConfig( response_modalities=["AUDIO"], @@ -75,125 +66,32 @@ run_config = RunConfig( input_audio_transcription=genai_types.AudioTranscriptionConfig(), output_audio_transcription=genai_types.AudioTranscriptionConfig(), ) -``` - - -**Transcription is opt-in.** You get no transcripts unless you enable input and output transcription in the `RunConfig`. The `finished=True` transcription event carries the complete utterance, so there is no need to accumulate fragments client-side. - - -### Step 2: Open the conversation root run -Open one run for the whole conversation and mark it as a voice trace with `ls_modality="audio"`, following the [single-trace convention](/langsmith/trace-voice-fundamentals#trace-each-conversation-as-a-single-trace). Keep this run open for the lifetime of the session and finalize it when the session ends. - -```python -from langsmith import RunTree - -session = RunTree( - name="conversation", - run_type="chain", - extra={"metadata": {"thread_id": thread_id, "model": MODEL, "ls_modality": "audio"}}, -) -session.post() -``` - -### Step 3: Trace each event - -Define a small helper that opens a child run for one event, records its scrubbed payload, and closes it when the block exits. The `scrub` pass replaces raw audio bytes with a placeholder so the spans stay small: - -```python -from contextlib import contextmanager - - -def scrub(obj): - """Replace raw audio bytes with a placeholder so spans stay small.""" - if isinstance(obj, bytes): - return f"<{len(obj)} bytes>" - if isinstance(obj, dict): - return {k: scrub(v) for k, v in obj.items()} - if isinstance(obj, list): - return [scrub(v) for v in obj] - return obj - - -@contextmanager -def event_span(parent, event, *, name, inbound): - """Trace one event as a child run under the conversation root. - - User-to-model events land in `inputs`; model-to-user events land in - `outputs`, so the trace reads in the natural direction of flow. - """ - payload = scrub(event.raw.model_dump()) - child = parent.create_child( - name=name, - run_type="chain", - inputs=payload if inbound else {}, - ) - child.post() - try: - yield child - finally: - child.end(outputs={} if inbound else payload) - child.patch() -``` - -Then loop over the events from your app's `run_live` stream, skipping the audio-only chunks and spanning the rest. `runner`, `adk_session`, and `queue` come from your ADK Live app (see the [demo agent](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/adk/agent.py)); `LiveEvent` is the wrapper defined in the note below: - -```python -async for raw_event in runner.run_live( - user_id=USER_ID, +async for event in runner.run_live( + user_id=user_id, session_id=adk_session.id, live_request_queue=queue, run_config=run_config, ): - event = LiveEvent(raw_event) - if event.is_audio_only: - continue # tracing audio will make your traces very noisy - - with event_span(session, event, name=event.label, inbound=event.is_inbound): - ... # handle the event: capture the transcript, run a tool, and so on + ... # play audio, handle barge-in, update the UI ``` - -Skip audio-only events, the chunks of agent speech. They arrive in the thousands over a short conversation and would bury the trace, so play them to the speaker but do not span them. - +By default the plugin generates a [thread](/langsmith/threads) id per conversation; pass a `thread_id_provider` to supply your own. -A `LiveEvent` wrapper with helper functions is defined in the [demo repository](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/adk/events.py). Adapt the implementation to your own code. +Transcription is opt-in. Set both `input_audio_transcription` and `output_audio_transcription` on the `RunConfig`, or the trace has no transcripts. -## Attach audio +## Record the conversation audio - -Audio rates differ by direction: ADK Live expects 16 kHz PCM16 input and produces 24 kHz output. If your microphone capture is not 16 kHz, resample it on the send path. - - -To listen to a conversation alongside its transcript, attach a single combined recording of the whole conversation to the root run. Record both sides into one stereo WAV (the user's mic on the left channel, the agent's audio on the right) so interruptions show up as overlap between the channels. Write the user's mic frames as you send them to ADK, and tap the speaker for the agent's audio so audio flushed on barge-in never reaches the recording and the file reflects what the user actually heard. - -For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). For the cross-provider rationale, see [Record a single combined audio file](/langsmith/trace-voice-fundamentals#record-a-single-combined-audio-file). - -Finalize the root run when the session ends. Wrap the event loop in a `try`/`finally` so the run always closes, even on error: +Feed the plugin your microphone and playback audio, and it attaches a single stereo recording (user left, agent right) to the trace: ```python -try: - ... # the run_live event loop from Step 3 -except Exception as exc: - session.error = f"{type(exc).__name__}: {exc}" # surface failures on the root run -finally: - session.end() - session.patch() +plugin.record_user_audio(mic_chunk) # user mic PCM16 +plugin.record_agent_audio(played_chunk) # agent PCM16 as played ``` -The demo repository wraps the full recording flow for each framework, including mic resampling, speaker-tap capture, and stereo WAV reconstruction. For Gemini Live, see the [ADK agent](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/adk/agent.py) and the shared [recording helpers](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/sdk_tracing.py). - -## Troubleshooting - -- **No transcription configs means empty-looking traces.** This is the most common failure mode. Both `input_audio_transcription` and `output_audio_transcription` must be set on the `RunConfig`. -- **Don't accumulate transcript fragments.** Use the `finished=True` event's full text; fragments are only for live UI display. -- **Don't span audio-only events.** A few minutes of conversation produces thousands of them. -- **Fields co-occur.** Classify by priority, not by assuming one field per event. -- **Tools run inside ADK.** Do not synthesize your own tool runs. Doing so double-counts what `function_call` and `function_response` already record. -- **Resample the mic** if your capture isn't 16 kHz (ADK input is 16 kHz, output is 24 kHz). -- **Mute ADK's startup noise** for a console UI: `logging.getLogger("google_adk").setLevel(logging.ERROR)` suppresses the experimental-feature warning for `run_live` and the MCP-not-installed line. +Record the user's microphone capture before resampling it for ADK, and tap the speaker for the agent's audio, so the recording reflects what was actually heard. Feed both channels at the same sample rate (the plugin's `sample_rate`, 24 kHz by default). For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). ## Next steps @@ -205,3 +103,4 @@ The demo repository wraps the full recording flow for each framework, including Attach the conversation audio recording to your trace. + diff --git a/src/langsmith/trace-openai-realtime.mdx b/src/langsmith/trace-openai-realtime.mdx index 39e542942d..0f070ac381 100644 --- a/src/langsmith/trace-openai-realtime.mdx +++ b/src/langsmith/trace-openai-realtime.mdx @@ -4,179 +4,151 @@ sidebarTitle: OpenAI Realtime description: Trace OpenAI Realtime voice agents in LangSmith using the LangSmith SDK. --- -The [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime) powers low-latency speech-to-speech voice agents. This guide shows how to trace a Realtime app to LangSmith. +Trace your [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime) voice agents to LangSmith. For high-level conventions, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). -OpenAI Realtime is a speech-to-speech model: it processes audio natively and exchanges a continuous stream of typed JSON events with your application over a persistent WebSocket connection, rather than making discrete request/response calls. The following sections show those events and how to turn them into a LangSmith trace. For our high-level philosophy on getting the most out of your voice agent traces, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). +OpenAI Realtime is a speech-to-speech model that streams typed events over a WebSocket. Either way you build it, the integration captures each conversation as a single LangSmith trace, with a span for every meaningful event (transcripts, model responses, and tool calls) grouped by turn. You enable it with one call and do not create any spans yourself. The flood of audio chunks is played but not traced. -For a complete implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo). +## Choose an approach -## The event model +There are two ways to build an OpenAI Realtime agent, and each has its own integration: -Every event has a discriminated `type` string that indicates what it represents: audio, a tool call, and so on. - -The client **sends** events to configure the session and stream audio. You do not trace these as spans; they are requests, and their effects come back as server events: +| Approach | Use when | Trace with | +| --- | --- | --- | +| **Raw connection** | You drive the WebSocket loop and run tools yourself, and want full control with minimal dependencies (`openai` only). | `wrap_realtime` | +| **OpenAI Agents SDK** | You want the Agents SDK to own the turn and tool-call loop, with support for handoffs and multi-agent setups. | `wrap_realtime_session` | -| Client event | What it does | -| --- | --- | -| `session.update` | Configure the session: instructions, voice, audio formats, transcription model, turn detection, tools. | -| `input_audio_buffer.append` | Stream a base64 PCM16 mic chunk. Sent continuously. | -| `conversation.item.create` | Add an item—used to return a `function_call_output` after running a tool. | -| `response.create` | Ask the model to generate a response (needed explicitly when turn detection uses `create_response: false`). | +Both produce the same kind of trace. Pick the one that matches how you built your agent. -The server **sends back** events: +## Set environment variables -| Server event | What it carries | Traced? | -| --- | --- | --- | -| `session.created` / `session.updated` | Handshake / config acknowledgement. | Yes | -| `input_audio_buffer.speech_started` | Server VAD heard the user start—the **barge-in signal**; flush the speaker buffer. | Yes | -| `input_audio_buffer.speech_stopped` | Server VAD heard the user stop. | Yes | -| `input_audio_buffer.committed` | The audio buffer became a conversation item. | Yes | -| `conversation.item.created` | An item was added server-side. | Yes | -| `conversation.item.input_audio_transcription.completed` | The full user transcript for the turn. | Yes | -| `response.created` | The model started generating. | Yes | -| `response.output_audio.delta` | One chunk of agent speech (base64 PCM16); hundreds per response. | No—played, never spanned | -| `response.output_audio_transcript.delta` | Streaming fragment of the agent's transcript. | No | -| `response.output_audio_transcript.done` | The agent's full transcript for the response. | Yes | -| `response.function_call_arguments.delta` / `.done` | Streaming / final tool-call arguments. | `.done` only | -| `response.output_item.*`, `response.content_part.*` | Structural progress of the response. | Yes | -| `response.done` | The complete response object: all output items (including every `function_call`), plus token usage. | Yes | -| `error` | Server-reported error. | Yes | -| `rate_limits.updated` | Quota bookkeeping. | Yes | - - - -## How events map to LangSmith runs - -We recommend tracing the whole conversation as a single trace, with one span per traced event in arrival order: - -```text -realtime_session ← root run (chain) -│ metadata: thread_id, model, event_count, duration_s, ls_modality=audio -│ attachments: conversation.wav (stereo: L=user, R=agent) -│ -├─ input_audio_buffer.speech_started -├─ input_audio_buffer.speech_stopped -├─ conversation.item.input_audio_transcription.completed -├─ response.created -├─ response.function_call_arguments.done -├─ response.done -│ └─ lookup_weather × N ← tool runs, nested under the event that announced them -├─ response.done ← the spoken follow-up after tools -└─ error ← only if the server sent one +```bash .env +LANGSMITH_API_KEY= +LANGSMITH_TRACING=true +LANGSMITH_PROJECT=openai-realtime-voice +OPENAI_API_KEY= ``` - -**The noise rule:** we recommend skipping every event type ending in `.delta`, because the matching `.done` event repeats the complete payload. Tracing both records everything twice. `response.output_audio.delta` in particular is the agent's voice: hundreds of chunks per response that would bury the trace. Play it to the speaker, but never make it a span. - +## Raw connection + +Use this when you open the WebSocket yourself with `client.realtime.connect()` and drive the event loop. + +### Install -## Installation + -```bash -pip install "langsmith>=0.4" "openai>=1.50" +```bash pip +pip install "langsmith[openai]" ``` -The demo also uses `sounddevice` and `numpy` for the mic/speaker and to build the WAV attachment. +```bash uv +uv add "langsmith[openai]" +``` + + + +### Set up tracing + +`wrap_realtime` returns a transparent proxy of your connection. Your existing `async for event in connection` loop, `session.update`, and tool handling stay the same: + +```python +from langsmith.integrations.openai import wrap_realtime +from openai import AsyncOpenAI -## Set up your environment +client = AsyncOpenAI() -The following steps demonstrate how to trace using the LangSmith SDK. You can also trace using OpenTelemetry directly. See [Trace with OpenTelemetry](/langsmith/trace-with-opentelemetry). +async with client.realtime.connect(model="gpt-realtime") as raw, wrap_realtime( + raw, + thread_id=thread_id, + project_name="openai-realtime-voice", +) as connection: + await connection.session.update(session={...}) # your existing config -```bash -export LANGSMITH_API_KEY=... -export LANGSMITH_TRACING=true -export LANGSMITH_PROJECT=my-voice-app -export OPENAI_API_KEY=... + async for event in connection: + ... # your existing handling: play audio, run tools, update UI ``` -## Quickstart +A stable `thread_id` you generate per conversation (for example, a UUID) groups the trace into a LangSmith [thread](/langsmith/threads). Any [`@traceable`](/langsmith/annotate-code) tools you run while handling an event nest under that event automatically. -This guide focuses on the tracing layer. It assumes you already have a working Realtime app: the WebSocket `connection`, the `session.update` configuration, and your microphone and speaker I/O. For a complete, runnable implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo/tree/main/src/voice_demo/openai). Enable `input_audio_transcription` (and the agent transcript) in your `session.update`, or the transcription events that make the trace readable never arrive. +Enable `input_audio_transcription` (and the agent transcript) in your `session.update`, or the transcription events that make the trace readable never arrive. -### Step 1: Open the conversation root at connect time +### Record the conversation audio -Use one `RunTree` per conversation: +Feed the proxy your microphone and playback audio, and it attaches a single stereo recording (user left, agent right) to the trace. Pass `is_agent_speaking` so barge-ins are flagged: ```python -from langsmith import RunTree - -root = RunTree( - name="realtime_session", - run_type="chain", - inputs={}, - project_name="my-voice-app", - extra={"metadata": {"thread_id": thread_id, "model": MODEL, "ls_modality": "audio"}}, -) -root.post() +async with client.realtime.connect(model="gpt-realtime") as raw, wrap_realtime( + raw, + thread_id=thread_id, + is_agent_speaking=lambda: speaker.buffered_bytes() > 0, +) as connection: + async for event in connection: + if event.type == "input_audio_buffer.append": + connection.record_user_audio(mic_chunk) # user mic PCM16 + elif event.type == "response.output_audio.delta": + connection.record_agent_audio(played_chunk) # agent PCM16 as played ``` -A stable `thread_id` you generate per conversation (for example, a UUID) groups the trace into a LangSmith [thread](/langsmith/threads); `ls_modality="audio"` marks it as a voice conversation. +## OpenAI Agents SDK -### Step 2: Span each received event, skipping the noise +Use this when you build the agent with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/realtime/guide/) (`RealtimeAgent` / `RealtimeRunner`), which owns the turn and tool-call loop. -Define a small helper that opens a child run for one event, records the scrubbed payload as the run's input, and closes it when the block exits: + +The Agents SDK's built-in realtime tracing uploads to OpenAI's own dashboard. Call `agents.set_tracing_disabled(True)` to avoid a second, separate upload path. + -```python -from contextlib import contextmanager - -@contextmanager -def event_span(parent, event, *, name): - """Trace one event as a child run, with its payload as the run's input.""" - payload = event.model_dump(mode="json") - child = parent.create_child(name=name, run_type="chain", inputs={"event": payload}) - child.post() - try: - yield child - finally: - child.end() - child.patch() -``` +### Install -Then loop over the events from your open Realtime `connection`, skipping the `.delta` noise and tracing the rest: + -```python -async for event in connection: - if event.type.endswith(".delta"): - continue # the matching .done event repeats the full payload +```bash pip +pip install "langsmith[openai-agents]" +``` - with event_span(root, event, name=event.type) as event_run: - ... # your handling for this event type +```bash uv +uv add "langsmith[openai-agents]" ``` -### Step 3: Run tools nested under the announcing event + + +### Set up tracing + +`wrap_realtime_session` wraps the `RealtimeSession` and enters it for you. Iterate it as you would the original; the SDK runs tools and manages turns: ```python -import json - -from langsmith.run_helpers import tracing_context - -if event.type == "response.done": - calls = [i for i in (event.response.output or []) if i.type == "function_call"] - for call in calls: - with tracing_context(parent=event_run): - result = await execute_tool(call.name, call.arguments) # traced child - await connection.conversation.item.create(item={ - "type": "function_call_output", - "call_id": call.call_id, - "output": json.dumps(result), - }) - if calls: - await connection.response.create() # ask for the spoken follow-up -``` +from agents import set_tracing_disabled +from agents.realtime import RealtimeAgent, RealtimeRunner +from langsmith.integrations.openai_agents_sdk import wrap_realtime_session +set_tracing_disabled(True) -## Attach the conversation audio +runner = RealtimeRunner( + starting_agent=RealtimeAgent(name="assistant", instructions="...", tools=[...]), +) +session = await runner.run() + +async with wrap_realtime_session( + session, + thread_id=thread_id, + project_name="openai-realtime-voice", +) as conn: + async for event in conn: + ... # your handling: play audio, update UI +``` -To listen to a conversation alongside its transcript, attach a single combined recording of the whole conversation to the root run. Record both the user and the agent in one file (for example, a stereo WAV with the user mic on one channel and the agent on the other), captured from what was played to the client so the recording reflects what was actually heard, including speech cut off by a barge-in. The Realtime API streams agent audio as `response.output_audio.delta` events: decode and write those bytes to your output device, and tap that same output to build the recording. +The conversation transcript is reconstructed from the session's `history` snapshots, so messages appear even though the SDK streams them as partials. -For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). For the cross-provider rationale, see [Record a single combined audio file](/langsmith/trace-voice-fundamentals#record-a-single-combined-audio-file). +### Record the conversation audio -When the conversation ends, finalize the root run: +Feed the proxy your microphone and playback audio: ```python -root.end() -root.patch() +async for event in conn: + if event.type == "audio": + conn.record_agent_audio(event.audio.data) # agent PCM16 as played + # feed user mic PCM16 as you send it to the session: + # conn.record_user_audio(mic_chunk) ``` ## Next steps @@ -189,3 +161,4 @@ root.patch() Attach the conversation audio recording to your trace. + diff --git a/src/langsmith/trace-with-google-adk.mdx b/src/langsmith/trace-with-google-adk.mdx index babd433730..287f9862c8 100644 --- a/src/langsmith/trace-with-google-adk.mdx +++ b/src/langsmith/trace-with-google-adk.mdx @@ -10,6 +10,10 @@ import ExampleMultiAgent from '/snippets/langsmith/integrations/google-adk/examp This guide shows you how to trace [Google Agent Development Kit (ADK)](https://github.com/google/adk-python) agents in LangSmith. You'll configure automatic tracing for your ADK applications to capture agent invocations, tool calls, and LLM interactions. + +This guide covers ADK's standard execution paths (text agents, tools, and multi-agent workflows). To trace a **Gemini Live** voice agent, which uses ADK's `Runner.run_live` streaming loop, see [Trace Gemini Live applications](/langsmith/trace-gemini-live). + + ## Installation Install the required packages using your preferred package manager: diff --git a/src/langsmith/trace-with-livekit.mdx b/src/langsmith/trace-with-livekit.mdx index 3ec6ca4e40..d004084449 100644 --- a/src/langsmith/trace-with-livekit.mdx +++ b/src/langsmith/trace-with-livekit.mdx @@ -3,247 +3,101 @@ title: Trace LiveKit applications sidebarTitle: LiveKit --- -LangSmith can capture traces generated by [LiveKit Agents](https://docs.livekit.io/agents/) using OpenTelemetry instrumentation. This guide shows you how to automatically capture traces from your LiveKit voice AI agents and send them to LangSmith for monitoring and analysis. +Trace your [LiveKit Agents](https://docs.livekit.io/agents/) voice agents to LangSmith with the LangSmith LiveKit integration. For high-level conventions, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). -For our high-level guiding principles on tracing voice agents, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). + +The LiveKit integration requires `langsmith[livekit]`. + -For a complete implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo). +The integration captures each conversation as a single LangSmith trace, with a span for every pipeline stage (STT, LLM, TTS) grouped by turn, plus LiveKit's per-stage latency and token metrics. You enable it with one call and do not create any spans yourself. -## Installation +## Install -Install the required packages: +Install the integration along with the LiveKit plugins your agent uses: ```bash pip -pip install langsmith livekit livekit-agents livekit-plugins-openai livekit-plugins-silero livekit-plugins-turn-detector opentelemetry-exporter-otlp python-dotenv +pip install "langsmith[livekit]" "livekit-agents[openai,silero,turn-detector]" ``` ```bash uv -uv add langsmith livekit livekit-agents livekit-plugins-openai livekit-plugins-silero livekit-plugins-turn-detector opentelemetry-exporter-otlp python-dotenv +uv add "langsmith[livekit]" "livekit-agents[openai,silero,turn-detector]" ``` -## Quickstart tutorial +## Set environment variables -Follow this step-by-step tutorial to create a voice AI agent with LiveKit and LangSmith tracing. You'll build a complete working example by copying and pasting code snippets. - -### Step 1: Set up your environment - -Create a `.env` file in your project directory: +The integration reads your LangSmith credentials from the environment and exports to LangSmith for you: ```bash .env -OTEL_EXPORTER_OTLP_ENDPOINT=https://api.smith.langchain.com/otel -OTEL_EXPORTER_OTLP_HEADERS=x-api-key=, Langsmith-Project=livekit-voice +LANGSMITH_API_KEY= +LANGSMITH_TRACING=true +LANGSMITH_PROJECT=livekit-voice LIVEKIT_URL= LIVEKIT_API_KEY= LIVEKIT_API_SECRET= OPENAI_API_KEY= ``` -### Step 2: Download the span processor - -LiveKit emits OpenTelemetry spans, but most of the useful data rides in LiveKit-specific attributes that LangSmith doesn't recognize by default. A custom span processor translates those attributes so your traces render properly in LangSmith. - -Add the [custom span processor file](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/livekit/processor.py) and save it as `langsmith_processor.py` in your project directory. - - - -The span processor enriches LiveKit Agents' OpenTelemetry spans with LangSmith-compatible attributes so your traces display properly in LangSmith. - -**Key functions:** - -- Converts LiveKit span types (stt, llm, tts, agent, session, job) to LangSmith format. -- Adds `gen_ai.prompt.*` and `gen_ai.completion.*` attributes for message visualization. -- Surfaces LiveKit's metrics (time-to-first-token, time-to-first-byte, end-to-end latency, and other `lk.*` analysis data) as run metadata. -- Tracks and aggregates conversation messages across the conversation. -- Renders the whole-conversation transcript and attaches the call recording to the root run. +## Set up tracing -It only reshapes spans it recognizes as LiveKit's; any other span (for example, a nested LangChain or LangGraph run) passes through untouched. The processor activates when you import it in your code. - - - -### Step 3: Create your voice agent file - -Create a new file called `agent.py` and add the following code. We'll build it section by section so you can copy and paste each part. - -#### Part 1: Import dependencies and set up tracing +Import `configure_livekit` and call it once before creating your `AgentServer`. It builds the tracer provider, registers the LangSmith span processor, and wires it into LiveKit: ```python -import sys -import os -from pathlib import Path -from dotenv import load_dotenv - -# Load environment variables -load_dotenv() - -# Import LiveKit components +from langsmith.integrations.livekit import configure_livekit from livekit import agents -from livekit.agents import AgentServer, AgentSession, Agent -from livekit.agents.telemetry import set_tracer_provider -from livekit.plugins import openai, silero -from livekit.plugins.turn_detector.multilingual import MultilingualModel -from opentelemetry.sdk.trace import TracerProvider - -# Import span processor to enable LangSmith tracing -from langsmith_processor import LangSmithSpanProcessor - -# Set up LangSmith tracing -def setup_langsmith(): - """Setup OpenTelemetry tracing to export spans to LangSmith.""" - endpoint = os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT") - headers = os.getenv("OTEL_EXPORTER_OTLP_HEADERS") - - if not endpoint or not headers: - print("⚠️ Warning: OTEL environment variables not set. Tracing disabled.") - return - - # Create tracer provider with custom span processor - trace_provider = TracerProvider() - trace_provider.add_span_processor(LangSmithSpanProcessor()) - - # Set as LiveKit's tracer provider - set_tracer_provider(trace_provider) - print("✅ LangSmith tracing enabled") - -# Enable tracing before creating agents -setup_langsmith() -``` - -#### Part 2: Define your agent +from livekit.agents import Agent, AgentServer, AgentSession -```python -class Assistant(Agent): - def __init__(self) -> None: - super().__init__( - instructions="""You are a helpful voice AI assistant. - You eagerly assist users with their questions. - Keep responses concise and conversational.""", - ) -``` +# Enable tracing before creating agents. +configure_livekit() -#### Part 3: Set up the agent server - -```python server = AgentServer() @server.rtc_session() async def my_agent(ctx: agents.JobContext): - # Create agent session with STT, LLM, TTS, and VAD session = AgentSession( stt="deepgram/nova-2:en", llm="openai/gpt-5.4-mini", - tts=openai.TTS(model="tts-1", voice="alloy"), - vad=silero.VAD.load(), - turn_detection=MultilingualModel(), - ) - - # Start the session - await session.start( - room=ctx.room, - agent=Assistant(), + tts="openai/tts-1:alloy", ) - -if __name__ == "__main__": - # Run in console mode for local testing - sys.argv = [sys.argv[0], "console"] - agents.cli.run_app(server) + await session.start(room=ctx.room, agent=Agent(instructions="You are a helpful assistant.")) ``` -### Step 4: Run your agent - -Run your voice agent in console mode for local testing: - -```bash -python agent.py console -``` +This works for both the STT/LLM/TTS cascade and speech-to-speech models: build the `AgentSession` with a realtime model (for example, `lk_openai.realtime.RealtimeModel(...)`) and the tracing setup is unchanged. -Your agent will start and connect to LiveKit. Speak through your microphone, and all traces will automatically appear in LangSmith. +## Record the conversation audio -View the complete [agent.py code](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/livekit/agent.py). - -## Advanced usage - -### Trace speech-to-speech models - -The previous example uses a cascade pipeline (separate STT, LLM, and TTS services). LiveKit also supports speech-to-speech models, where a single realtime model handles audio in and out. To trace one, build the session with a realtime model instead of the STT/LLM/TTS stack. The tracing setup is identical and the span processor is unchanged: +LiveKit records the call audio; the integration attaches it to the trace. Pass an `audio_path_provider` that returns the recording path, and enable recording on the session: ```python -from livekit.plugins import openai as lk_openai - -session = AgentSession(llm=lk_openai.realtime.RealtimeModel(voice="marin")) -``` - -### Attach the conversation audio - -To listen to a conversation alongside its transcript, record the call and attach the audio file to the root run. Record what was played to the client so the recording reflects what was actually heard, including any speech cut off by an interruption. For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). - -### Custom metadata and tags +from pathlib import Path -You can add custom metadata to your traces using span attributes: +_audio_path: Path | None = None +configure_livekit(audio_path_provider=lambda: _audio_path) -```python -from opentelemetry import trace - -class Assistant(Agent): - def __init__(self) -> None: - super().__init__( - instructions="You are a helpful assistant.", - ) - - # Get current span and add custom attributes - tracer = trace.get_tracer(__name__) - span = trace.get_current_span() - if span: - span.set_attribute("langsmith.metadata.agent_type", "voice_assistant") - span.set_attribute("langsmith.metadata.version", "1.0") - span.set_attribute("langsmith.span.tags", "livekit,voice-ai,production") +@server.rtc_session() +async def my_agent(ctx: agents.JobContext): + global _audio_path + _audio_path = ctx.session_directory / "audio.ogg" + await session.start( + room=ctx.room, + agent=Agent(instructions="You are a helpful assistant."), + record={"audio": True}, + ) ``` -## Troubleshooting - -### Spans not appearing in LangSmith - -If traces aren't showing up in LangSmith: - -1. **Verify environment variables**: Ensure `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` are set correctly in your `.env` file. -2. **Check setup order**: Make sure `setup_langsmith()` is called **before** creating `AgentServer`. -3. **Check API key**: Confirm your LangSmith API key has write permissions. -4. **Look for confirmation**: You should see "✅ LangSmith tracing enabled" in the console when starting. - -### Messages not showing correctly - -If conversation messages aren't displaying properly: - -1. **Check span processor**: Verify `langsmith_processor.py` is in your project directory and imported correctly. -2. **Verify imports**: Ensure `LangSmithSpanProcessor` is imported in your agent.py. -3. **Enable debug logging**: Set `LANGSMITH_PROCESSOR_DEBUG=true` in your environment to see detailed logs. - -### Connection issues - -If your agent can't connect to LiveKit: - -1. **Verify LiveKit URL**: Check `LIVEKIT_URL` is set correctly in your `.env` file. -2. **Check credentials**: Ensure `LIVEKIT_API_KEY` and `LIVEKIT_API_SECRET` are correct. -3. **Test connection**: Try connecting to your LiveKit server with the LiveKit CLI first. -4. **Console mode**: For local testing, always use: `python agent.py console`. - -### Import errors - -If you're getting import errors: - -1. **Install dependencies**: Run the complete pip install command from Step 1. -2. **Check Python version**: Ensure you're using Python 3.9 or higher. -3. **Verify langsmith_processor**: Make sure `langsmith_processor.py` is downloaded and in the same directory as `agent.py`. -4. **Check LiveKit plugins**: Ensure you have the correct LiveKit plugins installed for your STT/LLM/TTS providers. - -### Agent not responding +The recording reflects what was played to the client, so a barge-in shows up truncated. In console mode, also pass `--record` on the command line. For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). -If your agent connects but doesn't respond: +## Next steps -1. **Check API keys**: Verify your OpenAI API key (or other provider keys) are correct. -2. **Test services**: Ensure your STT, LLM, and TTS services are accessible. -3. **Check instructions**: Make sure your Agent has proper instructions. -4. **Review logs**: Look for errors in the console output. + + + Core conventions for tracing voice agents. + + + Attach the conversation audio recording to your trace. + + + diff --git a/src/langsmith/trace-with-pipecat.mdx b/src/langsmith/trace-with-pipecat.mdx index 8e016ffa32..417d4eae3b 100644 --- a/src/langsmith/trace-with-pipecat.mdx +++ b/src/langsmith/trace-with-pipecat.mdx @@ -3,333 +3,104 @@ title: Trace Pipecat applications sidebarTitle: Pipecat --- -LangSmith can capture traces generated by [Pipecat](https://pipecat.ai/) using OpenTelemetry instrumentation. This guide shows you how to automatically capture traces from your Pipecat voice AI pipelines and send them to LangSmith for monitoring and analysis. +Trace your [Pipecat](https://pipecat.ai/) voice agents to LangSmith with the LangSmith Pipecat integration. For high-level conventions, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). -For our high-level guiding principles on tracing voice agents, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals). + +The Pipecat integration requires `langsmith[pipecat]`. + -For a complete implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo). +The integration captures each conversation as a single LangSmith trace, with a span for every pipeline stage (STT, LLM, TTS) grouped by turn. You enable it with one call and do not create any spans yourself. -## Installation +## Install -Install the required packages: +Install the integration along with the Pipecat service extras your pipeline uses: ```bash pip -pip install langsmith "pipecat-ai[whisper,openai,local]" opentelemetry-exporter-otlp python-dotenv +pip install "langsmith[pipecat]" "pipecat-ai[whisper,openai,local]" ``` ```bash uv -uv add langsmith "pipecat-ai[whisper,openai,local]" opentelemetry-exporter-otlp python-dotenv +uv add "langsmith[pipecat]" "pipecat-ai[whisper,openai,local]" ``` - -If you plan to use the advanced audio recording features, also install: `pip install scipy numpy` - - -## Quickstart tutorial - -Follow this step-by-step tutorial to create a voice AI agent with Pipecat and LangSmith tracing. You'll build a complete working example by copying and pasting code snippets. - -### Step 1: Set up your environment +## Set environment variables -Create a `.env` file in your project directory: +The integration reads your LangSmith credentials from the environment and exports to LangSmith for you: ```bash .env -OTEL_EXPORTER_OTLP_ENDPOINT=https://api.smith.langchain.com/otel -OTEL_EXPORTER_OTLP_HEADERS=x-api-key=, Langsmith-Project=pipecat-voice +LANGSMITH_API_KEY= +LANGSMITH_TRACING=true +LANGSMITH_PROJECT=pipecat-voice OPENAI_API_KEY= ``` -### Step 2: Download the span processor - -Pipecat emits OpenTelemetry spans, but its attribute names aren't ones LangSmith recognizes by default. A custom span processor translates those attributes so your traces render properly in LangSmith. - -Add the [custom span processor file](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/pipecat/processor.py) and save it as `langsmith_processor.py` in your project directory. - - - -The span processor enriches Pipecat's OpenTelemetry spans with LangSmith-compatible attributes so your traces display properly in LangSmith. - -**Key functions:** +## Set up tracing -- Converts Pipecat span types (stt, llm, tts, turn, conversation) to LangSmith format. -- Adds `gen_ai.prompt.*` and `gen_ai.completion.*` attributes for message visualization. -- Renders the whole-conversation transcript onto the root run. -- Handles audio file attachments (for advanced usage). - -It only reshapes spans it recognizes as Pipecat's; any other span (for example, a nested LangChain or LangGraph run) passes through untouched. The processor activates when you import it in your code. - - - -### Step 3: Create your voice agent file - -Create a new file called `agent.py` and add the following code. We'll build it section by section so you can copy and paste each part. - -#### Part 1: Import dependencies +Import `configure_pipecat` and call it once before building your pipeline. Enable tracing on the `PipelineTask`: ```python -import asyncio -import uuid -from dotenv import load_dotenv - -# Load environment variables -load_dotenv() - -# Import Pipecat components -from pipecat.audio.vad.silero import SileroVADAnalyzer -from pipecat.pipeline.pipeline import Pipeline -from pipecat.pipeline.runner import PipelineRunner +from langsmith.integrations.pipecat import configure_pipecat from pipecat.pipeline.task import PipelineParams, PipelineTask -from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext -from pipecat.services.whisper.stt import WhisperSTTService -from pipecat.services.openai import OpenAILLMService, OpenAITTSService -from pipecat.transports.local.audio import LocalAudioTransport, LocalAudioTransportParams - -# Import the span processor setup to enable LangSmith tracing -from langsmith_processor import setup_langsmith_tracing -``` - -#### Part 2: Define the main function - -```python -async def main(): - # Generate unique thread ID for LangSmith - thread_id = str(uuid.uuid4()) - print(f"Starting conversation: {thread_id}") - - # Configure OpenTelemetry export to LangSmith and register the span processor. - # This reads OTEL_EXPORTER_OTLP_ENDPOINT / OTEL_EXPORTER_OTLP_HEADERS from your - # environment and returns the processor so you can register a recording later. - span_processor = setup_langsmith_tracing() - - # Configure audio input/output with voice activity detection - transport = LocalAudioTransport( - LocalAudioTransportParams( - audio_in_enabled=True, - audio_out_enabled=True, - vad_analyzer=SileroVADAnalyzer(), - ) - ) - - # Initialize AI services - stt = WhisperSTTService() - llm = OpenAILLMService(model="gpt-5.4-mini") - tts = OpenAITTSService(voice="alloy") - - # Set up conversation context with system prompt - context = OpenAILLMContext( - messages=[ - { - "role": "system", - "content": "You are a helpful voice assistant. Keep responses concise and conversational." - } - ] - ) - context_aggregator = llm.create_context_aggregator(context) - - # Build the processing pipeline - pipeline = Pipeline([ - transport.input(), # Capture microphone input - stt, # Convert speech to text - context_aggregator.user(), # Add user message to context - llm, # Generate AI response - tts, # Convert response to speech - transport.output(), # Play through speakers - context_aggregator.assistant(), # Add assistant response to context - ]) - - # Create task with tracing enabled - task = PipelineTask( - pipeline, - params=PipelineParams(enable_metrics=True), - enable_tracing=True, - enable_turn_tracking=True, - conversation_id=thread_id, - ) - - # Run the agent - runner = PipelineRunner() - await runner.run(task) -``` - -#### Part 3: Add the entry point - -```python -if __name__ == "__main__": - asyncio.run(main()) -``` - -### Step 4: Run your agent - -Run your voice agent: - -```bash -python agent.py -``` - -Speak to the agent through your microphone. All traces will automatically appear in LangSmith. -View the complete [agent.py code](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/pipecat/agent.py). +# Install the tracer and the LangSmith span processor. +configure_pipecat() -## Advanced usage - -### Trace a nested LangGraph agent - -Instead of a stock LLM service, you can use an in-process [LangChain or LangGraph](/oss/langgraph/overview) agent as the LLM stage of your pipeline. With a few adjustments, the agent's model and tool runs nest inside Pipecat's `llm` span so the whole conversation stays a single trace. - -Three things make this work: - -1. **Set `LANGSMITH_TRACING_MODE=otel`.** This makes the LangSmith SDK emit your LangChain/LangGraph runs as OpenTelemetry spans through the same provider, so they nest under Pipecat's `llm` span instead of forming separate top-level traces. -2. **Use a traced LLM service.** A bare `FrameProcessor` produces no `llm` span for the graph's runs to nest under. Subclass a traced service such as `OpenAILLMService` and run your graph from the traced context handler so its runs land inside the `llm` span. -3. **Set `llm_span_kind="chain"` on the span processor.** With the graph nested inside, Pipecat's `llm` span no longer performs the inference itself: the graph's own model nodes are the real LLM runs. Marking the wrapper a `chain` avoids an LLM run nested inside another LLM run. - -The resulting trace looks like this: - -```text -conversation ← root: whole transcript + audio recording -└── turn × N - ├── stt - ├── llm ← chain (orchestrates the graph) - │ ├── model ← ChatOpenAI (may emit tool calls) - │ ├── tools: lookup_weather ← tool run - │ └── model ← final answer (spoken) - └── tts +task = PipelineTask( + pipeline, + params=PipelineParams(enable_metrics=True), + enable_tracing=True, + enable_turn_tracking=True, + conversation_id=conversation_id, +) ``` -Only the final answer is spoken, but the tool-call exchange must be persisted back into Pipecat's `LLMContext` (as OpenAI-format messages) or the model loses its tool history on later turns. +Set `enable_tracing=True`, `enable_turn_tracking=True`, and `enable_metrics=True`. Turn tracking is required for tracing, and metrics drive the latency and token data on each span. -For a complete working implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo). - -### Custom metadata and tags - -You can add custom metadata to your traces using span attributes: - -```python -from opentelemetry import trace - -tracer = trace.get_tracer(__name__) - -async def run_voice_session(): - with tracer.start_as_current_span("voice_conversation") as span: - # Add custom metadata - span.set_attribute("langsmith.metadata.session_type", "voice_assistant") - span.set_attribute("langsmith.metadata.user_id", "user_123") - span.set_attribute("langsmith.span.tags", "pipecat,voice-ai,stt-llm-tts") - - # Your Pipecat pipeline code here - task = PipelineTask(pipeline, enable_tracing=True) - await task.queue_frames([TextFrame("Hello")]) -``` +If your LLM stage is an in-process [LangGraph or LangChain](/oss/langgraph/overview) agent, pass `configure_pipecat(llm_span_kind="chain")` so its model and tool runs nest correctly inside Pipecat's `llm` span. -### Recording and attaching audio to traces +## Record the conversation audio -Record the conversation and attach the audio to the root run so you can listen to it alongside the transcript. For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). - -Record **what was heard**, not what was generated. The naive approach, adding a recording `FrameProcessor` to the pipeline, taps the TTS frames upstream of `transport.output()`, so it over-captures: it includes audio the user never heard when a barge-in truncates the agent mid-sentence. Instead, tap at the device-write boundary, which the output transport reaches only *after* interruption truncation. - -The demo's [`RecordingLocalAudioTransport`](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/pipecat/recording_transport.py) does this: it records played agent audio in `write_audio_frame` and user audio off the input callback, then writes one stereo WAV (left channel user, right channel agent) using the shared [`build_stereo_session_wav`](https://github.com/langchain-ai/voice-demo/blob/main/src/voice_demo/sdk_tracing.py) helper. Use these as a reference implementation and adapt them to your project. - -Swap `LocalAudioTransport` for the recording transport, register the recording with the span processor so it attaches when the conversation span ends, and save it in a `finally` block: +Attach the conversation audio to the trace using Pipecat's [`AudioBufferProcessor`](https://docs.pipecat.ai/server/utilities/audio/audio-recording). Place it after `transport.output()` so it captures what was actually played (after any barge-in truncation), hand it to the integration, and start it once the session is running: ```python -from pathlib import Path -from datetime import datetime -from recording_transport import ConversationRecorder, RecordingLocalAudioTransport +from pipecat.processors.audio.audio_buffer_processor import AudioBufferProcessor -# Write the conversation recording to a per-run path -recordings_dir = Path.cwd() / "pipecat-recordings" -recordings_dir.mkdir(exist_ok=True) -timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") -recording_path = recordings_dir / f"conversation_{timestamp}.wav" +span_processor = configure_pipecat() -# Tap played agent audio and the user's mic at the device boundary -recorder = ConversationRecorder(recording_path) -transport = RecordingLocalAudioTransport( - LocalAudioTransportParams( - audio_in_enabled=True, - audio_out_enabled=True, - vad_analyzer=SileroVADAnalyzer(), - ), - recorder, -) +# Stereo: user on the left channel, agent on the right. +audiobuffer = AudioBufferProcessor(num_channels=2, buffer_size=32_000) +span_processor.attach_audio_buffer(audiobuffer, conversation_id=conversation_id) -# Attach the WAV to the conversation root span when the span ends -span_processor.register_recording( - thread_id, str(recording_path), audio_recorder=recorder -) - -# Build the pipeline with the recording transport pipeline = Pipeline([ - transport.input(), # mic in (taps user audio) + transport.input(), stt, context_aggregator.user(), llm, tts, - transport.output(), # speaker out (taps played agent audio) + transport.output(), + audiobuffer, # after output(): records what was heard context_aggregator.assistant(), ]) -# Run the pipeline, saving the recording on the way out -runner = PipelineRunner() -try: - await runner.run(task) -finally: - recorder.save_recording() +await audiobuffer.start_recording() ``` - -`register_recording` accepts any object with a `save_recording()` method as `audio_recorder`. The span processor calls it when the conversation span ends, so the file is on disk before it is read and attached. The `finally` block covers shutdown ordering and the case where tracing is disabled. - - -## Troubleshooting - -### Spans not appearing in LangSmith - -If traces aren't showing up in LangSmith: - -1. **Verify environment variables**: Ensure `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` are set correctly in your `.env` file. -2. **Check API key**: Confirm your LangSmith API key has write permissions. -3. **Verify import**: Make sure you're importing `setup_langsmith_tracing` from `langsmith_processor.py` and calling it before running the pipeline. -4. **Check .env loading**: Ensure `load_dotenv()` is called before importing Pipecat components. - -### Messages not showing correctly - -If conversation messages aren't displaying properly: - -1. **Check span processor**: Verify `langsmith_processor.py` is in your project directory and imported correctly. -2. **Verify thread ID**: Ensure you're setting a unique `conversation_id` in `PipelineTask`. -3. **Enable turn tracking**: Make sure `enable_turn_tracking=True` is set in `PipelineTask`. - -### Audio not working - -If your microphone or speakers aren't working: - -1. **Check permissions**: Ensure your terminal/IDE has microphone access. -2. **Test audio devices**: Verify your microphone and speakers work in other applications. -3. **VAD settings**: Try adjusting `SileroVADAnalyzer()` settings if speech isn't being detected. -4. **Check services**: Ensure OpenAI API key is valid and has access to Whisper and TTS. - -### Import errors - -If you're getting import errors: - -1. **Install dependencies**: Run `pip install langsmith "pipecat-ai[whisper,openai,local]" opentelemetry-exporter-otlp python-dotenv`. -2. **Check Python version**: Ensure you're using Python 3.9 or higher. -3. **Verify langsmith_processor**: Make sure `langsmith_processor.py` is downloaded and in the same directory as your `agent.py`. - -### Performance issues - -If responses are slow: - -1. **Use faster models**: Switch to `gpt-5.4-mini` for the LLM (already in the tutorial). -2. **Check network**: Ensure stable internet connection for API calls. -3. **Local STT**: Consider using local Whisper instead of API-based services. +The integration attaches the recording to the conversation root when it ends. For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). -### Advanced: Audio recording troubleshooting +## Next steps -For issues with the advanced audio recording features, see the [complete demo documentation](https://github.com/langchain-ai/voice-demo). + + + Core conventions for tracing voice agents. + + + Attach the conversation audio recording to your trace. + + +