Description
In sdk/trace/batch_span_processor.go, batchSpanProcessor.Shutdown spawns a goroutine to drain the queue and shut down the exporter. When the caller's ctx expires before the goroutine finishes, the select exits via <-ctx.Done() — but the goroutine continues running and eventually calls bsp.e.Shutdown(ctx) with that already-cancelled context.
go func() {
close(bsp.stopCh)
bsp.stopWait.Wait()
if bsp.e != nil {
exportErr = bsp.e.Shutdown(ctx) // ctx is already cancelled
}
close(wait)
}()
select {
case <-wait:
err = exportErr // only path that reads exportErr
case <-ctx.Done():
err = ctx.Err() // goroutine keeps running; exportErr is never read
}Sequence of events
- Caller passes a
ctx with a short deadline to bsp.Shutdown(ctx).
- Draining the queue takes longer than the deadline;
ctx becomes cancelled.
- The
select exits on <-ctx.Done(), returning ctx.Err() to the caller.
- The goroutine is still running. Eventually it calls
bsp.e.Shutdown(ctx) with the cancelled context.
- Most OTLP/gRPC exporters detect the cancelled context and return immediately without flushing.
exportErr is written by the goroutine, but the <-wait branch of the select is never executed, so the error is silently discarded.- Because
stopOnce has already fired, no subsequent Shutdown call is possible, leaving the exporter permanently un-closed and any remaining spans lost with no signal to the caller.
Expected behavior
When the caller's context expires, the background goroutine should still shut down the exporter using a fresh, independent context so the exporter has a real opportunity to flush. Any error from the exporter shutdown should be surfaced (e.g. via otel.Handle).
Suggested fix
Replace the cancelled context with a fresh timeout in the goroutine:
go func() {
close(bsp.stopCh)
bsp.stopWait.Wait()
if bsp.e != nil {
shutCtx, shutCancel := context.WithTimeout(context.Background(), defaultShutdownGrace)
defer shutCancel()
if err := bsp.e.Shutdown(shutCtx); err != nil {
otel.Handle(err)
}
}
close(wait)
}()where defaultShutdownGrace is a small constant (e.g. 5 s) that gives the exporter a final chance to flush independent of the caller's deadline.
Environment
- File:
sdk/trace/batch_span_processor.go (~line 162)
- Affects any OTLP/gRPC exporter that returns early on a cancelled context (which is the correct gRPC behaviour)
Description
In
sdk/trace/batch_span_processor.go,batchSpanProcessor.Shutdownspawns a goroutine to drain the queue and shut down the exporter. When the caller'sctxexpires before the goroutine finishes, theselectexits via<-ctx.Done()— but the goroutine continues running and eventually callsbsp.e.Shutdown(ctx)with that already-cancelled context.Sequence of events
ctxwith a short deadline tobsp.Shutdown(ctx).ctxbecomes cancelled.selectexits on<-ctx.Done(), returningctx.Err()to the caller.bsp.e.Shutdown(ctx)with the cancelled context.exportErris written by the goroutine, but the<-waitbranch of theselectis never executed, so the error is silently discarded.stopOncehas already fired, no subsequentShutdowncall is possible, leaving the exporter permanently un-closed and any remaining spans lost with no signal to the caller.Expected behavior
When the caller's context expires, the background goroutine should still shut down the exporter using a fresh, independent context so the exporter has a real opportunity to flush. Any error from the exporter shutdown should be surfaced (e.g. via
otel.Handle).Suggested fix
Replace the cancelled context with a fresh timeout in the goroutine:
where
defaultShutdownGraceis a small constant (e.g. 5 s) that gives the exporter a final chance to flush independent of the caller's deadline.Environment
sdk/trace/batch_span_processor.go(~line 162)