docs(design): pipeline pause semantics — recommend stop/start, defer warm-pause#2632
Merged
Merged
Conversation
…warm-pause Design doc for the v0.18 P4 'pause' prerequisite. Research + two-pass review against the tree established that a user-initiated stop already provides correct, crash-safe, at-least-once park-and-resume (not auto-restarted, drains in-flight to durable before teardown, resumes from last committed position), and stopped_reason=USER already labels operator intent. Recommendation: do NOT add a warm-idle pause primitive for v0.18. Model pause/resume as the existing stop/start RPCs (Alternative C). Warm-idle (A) and a first-class Paused state (B) are deferred with design sketches + the invariants they must satisfy. Cold resume is byte-for-byte crash recovery, which invariant 7 already requires every connector to survive, so warm-pause adds zero correctness — only resume latency. Consequence: UI-6 (operate) is unblocked with zero engine risk. Overturns the greenfield-UI doc's assumption that pause is prerequisite engine work — flagged for maintainer sign-off before acting on it. Tier 3 (docs). Advances v0.18 (P4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B3cLokwNJYLLBpdyt3cgGr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recommendation (needs maintainer sign-off before we act on it)
Do not add a warm-idle pause primitive to the engine for v0.18. Model pause/resume as the existing user stop/start RPCs. This overturns the greenfield-UI doc's assumption that
pauseis prerequisite engine work — so it's flagged for your explicit decision, not self-merged.Why
Research + an adversarial two-pass review (both verified every citation against the tree) established that a user-initiated stop already delivers what pause needs:
Initonly restartsSystemStopped) — stays parked until an explicit start,stopped_reason=USER(P3) already distinguishes an operator park from a system stop.The only things a dedicated warm-pause adds are (a) keeping plugin subprocesses warm for lower resume latency and (b) a distinct label. Cold resume is byte-for-byte the same operation as crash recovery, which invariant 7 already requires every connector to survive without loss — so warm-pause adds zero correctness, only latency, for a workflow (operator maintenance / relieve pressure / investigate) that is low-frequency and tolerates cold resume.
The review specifically stress-tested CDC / replication-slot / snapshot sources and found no correctness hole: a connector that would lose data across a park already loses it across a crash, independent of pause.
What this unblocks
UI-6 (operate) needs no pause engine work — it ships against the existing
StartPipeline/StopPipelineRPCs.Alternatives (in the doc)
StatusPausedmigration, backoff/force-stop interaction,kill -9-during-pause recovery — disproportionate data-path surface for a latency optimization with no demonstrated demand. Design sketch + required invariants captured for if/when demand (backed by benchi numbers) appears.Pausedstate reusing stop machinery: available, deferred (cheap middle step if API/agent ergonomics later want a distinct state).Review
Two independent passes (research brief + adversarial technical review). No P0s; recommendation verified sound. Two P1s (name the snapshot re-snapshot cost; sharpen the drifted-remote-state argument to the invariant-7 crash-equivalence) and two P2 citation tightenings are folded in. If accepted, the "no warm-pause primitive" decision is a candidate to promote to an immutable ADR.
Tier 3 (docs); the decision is Tier-1-significant → your call. markdownlint clean.
🤖 Generated with Claude Code