docs: design — live in-place hot-reload (processor + metadata) + conduit run --dev (§4)#2601
Merged
Merged
Conversation
…conduit run --dev Design doc for v0.17 execution-plan §4 (hot-reload) + §3 (dev verb). Roadmap: Phase 1 (Developer Experience Core). Scope decided in review: in-place-class changes apply WITHOUT restarting the pipeline (the real §4, not the always-restart shortcut). Invariant-safety line: - Live-swappable in place: processor config + pipeline Name/Description (no position/ack/connection state). - Restart-class via existing ApplyPlanLive: connector settings, DLQ, all topology (position/ack/connection state; live swap deferred to Phase 2 with the chaos harness). Core new work is a live, at-a-record-boundary ProcessorNode swap in pkg/lifecycle/stream, building on the existing ControlMessage/InjectControlMessage idiom; dev-mode is a consumer. Grounded against ApplyPlanLive, StopAndWait, ProcessorNode.Run, nodeBase.Receive, connectorService.Update, and the community prior art #2236 (superseded). Includes failure-mode + invariant (1-7) analysis and an honest accounting of the new crash surface (store commit is source of truth; node swaps in-memory), gated by a targeted mid-swap fault test. PR plan: PR1 (Tier 1, engine: swap primitive + classification + apply decision + fault test), PR2 (Tier 2, surface: run --dev + watcher + alias). Not yet implemented; opening for Tier-1 design sign-off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B3cLokwNJYLLBpdyt3cgGr
…ed PR1/PR2 acceptance criteria - Data-safety section: the swap drops zero events (control-signal trigger + backpressure + one-config-per-record boundary); DLQ only ever receives Nacks, never swap artifacts; the only exposure is crash-mid-swap -> at-least-once duplicates (never drops). - Batching/windowing guard as committed PR1 deliverables: invariant comment at the swap site + a contract test pinning the non-buffering Process model + a precondition on the future batching design (flush-first or restart-class). - Expanded the PR plan into detailed, testable PR1 (engine) and PR2 (surface) acceptance-criteria checklists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B3cLokwNJYLLBpdyt3cgGr
Contributor
Author
|
Tier-1 design sign-off from DeVaris (maintainer). Approved to implement. PR1 (engine) begins on |
This was referenced Jul 13, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Design doc for v0.17 execution-plan §4 (hot-reload) + §3 (dev verb). Roadmap: Phase 1 (Developer Experience Core). Design only — no implementation yet. Opening for Tier-1 design sign-off before code, mirroring how #2588's design landed as #2596 first.
The scope decision (made in review)
In-place-class changes must apply without restarting the pipeline — the real §4, not the always-restart shortcut (rejected). Invariant-safety line:
ApplyPlanLive): source/dest connector settings, DLQ (live destination w/ ack semantics), all topology. Live swap of these is invariant-1/2/3 territory → deferred to Phase 2 with the chaos harness.Core new work
A live, at-a-record-boundary
ProcessorNodeswap inpkg/lifecycle/stream, built on the existingControlMessage/InjectControlMessageidiom (already used to stop sources in-band). Open-new-before-teardown-old so a bad edit never drops the pipeline.ApplyPlanLiveInPlacedecides: all-swappable diff → in place; anything restart-class → delegate toApplyPlanLive. Dev-mode (run --dev+pipelines devalias) is a consumer.Grounding + rigor
Every mechanism claim verified against the tree:
ApplyPlanLive/StopAndWait,ProcessorNode.Run,nodeBase.Receive,connectorService.Update(store-only),buildNodes, and #2236 (superseded). Includes:Adversarial self-review (done inline; the async review agents stalled on infra)
Three lenses run against the code. Caught and fixed a real bug in an earlier draft:
ApplyPlanLive's!runningbranch imports without starting, so dev needed an explicit ensure-running step or pipelines would silently stay down after a failed apply / on a new file. Also corrected the DLQ classification (restart-class, not trivial metadata) and grounded the swap mechanism in the actualReceive/control-message design.Risk tier
Design doc for Tier-1 work. Requesting DeVaris sign-off on the design before I implement PR1.
PR plan
liveSwappable/LiveEligibleclassification +ApplyPlanLiveInPlace+ unit/property/mid-swap-fault tests. Fault test gates merge.run --dev+ runtime watcher + ensure-running +pipelines devalias +--json+ docs.🤖 Generated with Claude Code
https://claude.ai/code/session_01B3cLokwNJYLLBpdyt3cgGr