Skip to content

feat(replay): ADR 0012 migration step 4 — target-binding verification enforcement#1209

Open
thymikee wants to merge 3 commits into
mainfrom
feat/replay-target-verification
Open

feat(replay): ADR 0012 migration step 4 — target-binding verification enforcement#1209
thymikee wants to merge 3 commits into
mainfrom
feat/replay-target-verification

Conversation

@thymikee

@thymikee thymikee commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

Implements ADR 0012's migration step 4: target-binding verification enforcement (docs/adr/0012-interactive-replay.md, decision 3's "Replay-time verification" + decision 4's targetBinding wire fields), plus the coordinator-requested post-resolution target guard closing the verification/dispatch resolver split.

Pre-action verification (commit 1)

Every annotated resolved-target replay/test action (click/press/longpress/fill/get carrying a # agent-device:target-v1 {...} comment, per #1196) is verified before its device action is sent:

  • verifyReplayActionTarget (new: src/daemon/handlers/session-replay-target-verification.ts) resolves the recorded selector/ref against a fresh pre-action snapshot, using the same per-command rect/disambiguation config heal's suggestion re-resolution already uses (resolveSuggestionMatchingConfig).
  • The match is classified through decision 3's six-path algorithm (classifyTargetBindingMatch, from feat: parse and preserve .ad target-v1 evidence (ADR 0012 migration step 3) #1196, unchanged). Only path 4 / path-6-verified sends the action.
  • Every other outcome returns a complete REPLAY_DIVERGENCE (from feat(replay): ADR 0012 migration step 2 — structured replay divergence report #1197's transport) with kind:
    • selector-missmatchCount == 0
    • identity-mismatch — empty identity set, or a unique/isolated identity-set member differing from the resolution winner
    • identity-unverifiable — a recorded-unverifiable annotation (path 1, before any resolution), or several candidates sharing the identity with neither disambiguation signal isolating one (path 6 fall-through, ≤5 candidates in document order)
  • targetBinding on the divergence: classification (== kind), matchCount per decision 3's conditional presence rule (omitted only for a recorded-unverifiable annotation), recorded/observed identity, bounded mismatches, and candidates — all sanitized through the existing scrub → redact → truncate pipeline.
  • Unannotated actions (old scripts) are unaffected — pass-through. The check is wired only into the replay/test step loop (session-replay-runtime.ts), never into live interactive command dispatch.

Post-resolution guard (commit 2, coordinator addition)

Verification resolves without dispatch's occlusion-filtering / visibility-preferring guards, so its winner W could differ from the node dispatch actually acts on — verified-but-taps-a-different-element. The guard closes that split pre-action:

  • A verified outcome mints internal.replayTargetGuard (the verified member's normalized local identity via the shared readNodeLocalIdentity, new src/replay/target-identity-node.ts) onto the dispatch request.
  • Interaction handlers thread it into command options as expectedResolvedTarget; a guarded dispatch skips the direct-iOS selector fast path and the native-ref tapTarget/fillTarget fast paths (the guard needs a runtime tree node to check).
  • assertExpectedResolvedTarget (resolution.ts) compares the resolution winner — pre-promotion (hittable-ancestor promotion deliberately retargets to the same element's actionable container), pre-action — against the verified identity, refusing with details.reason: 'replay_target_guard_mismatch' on a mismatch. Enforced on the selector path, the ref path, and get's ref/selector reads.
  • The replay loop converts that refusal into an identity-mismatch target-binding divergence (buildReplayTargetGuardMismatchResponse) carrying verification's matchCount (present, per the rule), recorded/observed identities, and a fresh screen observation — through the same bounding/sanitizing builder as every other target-binding divergence.
  • Replay-only by construction: internal.replayTargetGuard is set exclusively by the replay step loop for annotated verified actions; live interactive commands never carry it.

Cost decision + planned follow-ups

  • Double-resolution cost accepted for this step (coordinator decision): replay is machine-time (~0.6 s/capture; even doubling per annotated step keeps the 38-flow suite well under 15 min), and correctness-first is right for v1. Planned follow-up (post-review chip): a read-only resolve entry point in commands/interaction/runtime/resolution.ts so verification can share dispatch's literal resolution (including guards) instead of paying a second capture+resolve.
  • PR feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement #1211 coordination (resume + retirement, merging first): buildTargetBindingDivergenceResponse is the single resume site for target-binding divergences and is marked as the step-5 wiring point. After feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement #1211 merges, this branch rebases and replaces the resume: not-supported stub with feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement #1211's real computed resume — target-binding divergences fire pre-action, so same-step resume is typically allowed when preflight permits (computed, not assumed) — plus a test asserting a target-binding divergence carries a real computed resume object. Structured to be a one-site change.

Test plan

  • classifyReplayTarget pure-core: all six verification paths, an extended real-capture fixture (bottomTabsRealCaptureFixture, react-navigation bottom tab bar, shared with session-target-evidence.test.ts), same-parent sibling recurrence under different anonymous parents, region-partitioned viewportOrder domains, a recorded scroll region gone (unavailable, never compared cross-region), an out-of-range viewportOrder, and document-order determinism for equal rect centers.
  • Split-resolver fixture (coordinator ask): covered deeper A vs visible sibling B — verification verifies A (matchCount 2), dispatch's occlusion filter resolves B; the guard refuses pre-action with zero backend taps, a matching identity passes through, and a guarded click @ref is forced off the native fast path onto the runtime resolution path.
  • End-to-end runReplayScriptFile: the device action is never dispatched on a divergence; old unannotated scripts execute unchanged (and carry no guard); a verified target proceeds to dispatch with the guard on its request; a dispatch-time guard mismatch converts to an identity-mismatch divergence with matchCount present and field-level mismatches.
  • Divergence wire tests: matchCount presence rule (present 0..N / absent-never-null on recorded-unverifiable), targetBinding.classification === kind, overflow-fallback preservation of targetBinding, text-report rendering of matchCount/mismatches/candidates.
  • Full gate: pnpm typecheck, pnpm lint, pnpm format:check, pnpm check:layering (774 files, no back-edge growth — the shared identity reader lives in the replay zone precisely to avoid a daemon → commands back-edge), pnpm check:fallow, pnpm check:production-exports — all clean.
  • pnpm test:unit (402 files / 3742 tests) and pnpm test:integration:provider (32 files / 120 tests) — all green (single slow-test-gate flag was CPU contention on an unrelated Apple screenshot retry test; clean in isolation).

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.7 MB 1.7 MB +13.1 kB
JS gzip 532.0 kB 536.3 kB +4.3 kB
npm tarball 642.0 kB 645.9 kB +3.9 kB
npm unpacked 2.3 MB 2.3 MB +13.1 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 25.5 ms 26.5 ms +1.0 ms
CLI --help 54.6 ms 56.4 ms +1.8 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/gestures.js +28.1 kB +9.5 kB
dist/src/tv-remote.js -20.8 kB -6.5 kB
dist/src/session.js +10.5 kB +2.8 kB
dist/src/sdk-selectors.js +216 B +46 B
dist/src/runtime.js +317 B +42 B

@thymikee

Copy link
Copy Markdown
Member Author

Current-head review found two blockers:

  1. P1: replay --update verifies the original action, but after a target-binding divergence it heals positionals and directly invokes the healed action without rerunning target verification. The healer preserves the old targetEvidence, so an identity mismatch can be bypassed and dispatched. Every post-heal attempt must re-enter verification; add a regression proving no action is sent until the healed target verifies.
  2. P2: listSelectorChainMatches returns the first selector alternative with any matches, while resolveSelectorChain skips ambiguous/tied alternatives and can select a later one. Verification can therefore combine the first alternative's match domain/count with a later alternative's winner, causing false divergence and incorrect matchCount. Make match-domain selection follow the exact resolver alternative and add a chained-alternative regression.

Also tighten locality before readiness: the new implementation is 598 LOC and its test is 557 LOC, exceeding the repo's 500-LOC extraction tripwire. Assigning a focused fixer for correctness and a concept-based split.

thymikee added a commit that referenced this pull request Jul 11, 2026
…inner against the verified member

Coordinator addition to step 4 (PR #1209): pre-action verification resolves
without dispatch's occlusion-filtering/visibility-preferring guards, so
verification's winner W could differ from the node dispatch actually acts
on — verified-but-taps-a-different-element, or a false identity-mismatch.
Close the split with a cheap cross-check at dispatch's own resolution:

- Verified outcomes now carry the verified member's normalized local
  identity (`ReplayVerifiedTargetGuard`: readNodeLocalIdentity(winner) +
  verification's matchCount). The replay loop threads it onto the dispatch
  request as `internal.replayTargetGuard`.
- Interaction handlers (press/click/longpress/fill in interaction-touch.ts,
  get in selector-runtime.ts) pass it into command options as
  `expectedResolvedTarget`, and a guarded dispatch skips the direct-iOS
  selector fast path and the native-ref tapTarget/fillTarget fast paths —
  the guard needs the runtime tree resolution to have a node to check.
- `assertExpectedResolvedTarget` (resolution.ts) compares the resolution
  winner's identity — PRE-promotion (hittable-ancestor promotion
  deliberately retargets to the same element's actionable container) and
  PRE-action — against the verified identity, and refuses with
  `details.reason: 'replay_target_guard_mismatch'` on a mismatch. Enforced
  on the selector path, the ref path, and get's ref/selector reads.
- The replay loop converts that refusal into an identity-mismatch
  target-binding divergence (`buildReplayTargetGuardMismatchResponse`):
  classification == kind, matchCount from verification's recorded-selector
  resolution (present, per the rule), recorded/observed identities,
  field-level mismatches, fresh screen observation — via the same
  bounding/sanitizing builder as every other target-binding divergence
  (finalize refactored into module-level
  buildTargetBindingDivergenceResponse for the two callers).
- The identity reader moved to src/replay/target-identity-node.ts
  (readNodeLocalIdentity + localIdentitiesEqual + the reason marker) so the
  commands and daemon layers share it without a layering back-edge;
  session-target-evidence.ts's boundedLocalIdentity is now an alias.

The guard is replay-only by construction: `internal.replayTargetGuard` is
set exclusively by the replay step loop for annotated verified actions;
live interactive commands never carry it.

PR #1211 coordination: buildTargetBindingDivergenceResponse is the single
resume site for target-binding divergences, marked as the step-5 wiring
point — after #1211 merges, replace the not-supported stub with the real
computed resume (pre-action divergences typically allow same-step resume)
plus a test. Kept as a small follow-up by construction.

Tests: split-resolver fixture (covered deeper A vs visible sibling B —
verification verifies A, dispatch resolves B) proving the guard refuses
pre-action with zero backend taps, the matching-identity pass-through, the
ref-fast-path forcing, guard threading on the dispatch request (and its
absence for unannotated actions), and end-to-end conversion of a guard
mismatch into an identity-mismatch divergence with matchCount present.
@thymikee

thymikee commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

Re-review of 4ed7ef2e: the new post-resolution guard closes the dispatch-winner mismatch, but the prior blockers remain.

  • P1: the replay --update healed action is still invoked directly with unguarded replayReq after healReplayAction; target verification is not rerun.
  • P2: verification still combines the first matching alternative from listSelectorChainMatches with the potentially later winner from resolveSelectorChain, so match count/domain can describe a different alternative.
  • P2: the implementation/test grew to 797/733 LOC; split classifier, runtime/wire shaping, and guard responsibilities per the repo >500 LOC rule.

The new regression covers the initial dispatch guard, not the update/heal bypass or ambiguous-first/later-resolvable selector chain.

thymikee added a commit that referenced this pull request Jul 11, 2026
…inner against the verified member

Coordinator addition to step 4 (PR #1209): pre-action verification resolves
without dispatch's occlusion-filtering/visibility-preferring guards, so
verification's winner W could differ from the node dispatch actually acts
on — verified-but-taps-a-different-element, or a false identity-mismatch.
Close the split with a cheap cross-check at dispatch's own resolution:

- Verified outcomes now carry the verified member's normalized local
  identity (`ReplayVerifiedTargetGuard`: readNodeLocalIdentity(winner) +
  verification's matchCount). The replay loop threads it onto the dispatch
  request as `internal.replayTargetGuard`.
- Interaction handlers (press/click/longpress/fill in interaction-touch.ts,
  get in selector-runtime.ts) pass it into command options as
  `expectedResolvedTarget`, and a guarded dispatch skips the direct-iOS
  selector fast path and the native-ref tapTarget/fillTarget fast paths —
  the guard needs the runtime tree resolution to have a node to check.
- `assertExpectedResolvedTarget` (resolution.ts) compares the resolution
  winner's identity — PRE-promotion (hittable-ancestor promotion
  deliberately retargets to the same element's actionable container) and
  PRE-action — against the verified identity, and refuses with
  `details.reason: 'replay_target_guard_mismatch'` on a mismatch. Enforced
  on the selector path, the ref path, and get's ref/selector reads.
- The replay loop converts that refusal into an identity-mismatch
  target-binding divergence (`buildReplayTargetGuardMismatchResponse`):
  classification == kind, matchCount from verification's recorded-selector
  resolution (present, per the rule), recorded/observed identities,
  field-level mismatches, fresh screen observation — via the same
  bounding/sanitizing builder as every other target-binding divergence
  (finalize refactored into module-level
  buildTargetBindingDivergenceResponse for the two callers).
- The identity reader moved to src/replay/target-identity-node.ts
  (readNodeLocalIdentity + localIdentitiesEqual + the reason marker) so the
  commands and daemon layers share it without a layering back-edge;
  session-target-evidence.ts's boundedLocalIdentity is now an alias.

The guard is replay-only by construction: `internal.replayTargetGuard` is
set exclusively by the replay step loop for annotated verified actions;
live interactive commands never carry it.

PR #1211 coordination: buildTargetBindingDivergenceResponse is the single
resume site for target-binding divergences, marked as the step-5 wiring
point — after #1211 merges, replace the not-supported stub with the real
computed resume (pre-action divergences typically allow same-step resume)
plus a test. Kept as a small follow-up by construction.

Tests: split-resolver fixture (covered deeper A vs visible sibling B —
verification verifies A, dispatch resolves B) proving the guard refuses
pre-action with zero backend taps, the matching-identity pass-through, the
ref-fast-path forcing, guard threading on the dispatch request (and its
absence for unannotated actions), and end-to-end conversion of a guard
mismatch into an identity-mismatch divergence with matchCount present.
@thymikee thymikee force-pushed the feat/replay-target-verification branch from 4ed7ef2 to a3749f2 Compare July 11, 2026 07:19
@thymikee

Copy link
Copy Markdown
Member Author

Rebased onto main after #1193 merged (branch force-pushed with lease — re-fetch if you were running against the old head 4ed7ef2eb; new head is a3749f26a).

  • Only textual conflict: the resolution.ts import list in interactions.ts (EXACT_REF_RESOLUTION from feat: disclose selector resolution in interaction responses #1193 vs this PR's ExpectedResolvedTarget) — kept both.
  • Semantics preserved on both sides: the post-resolution guard asserts before describeResolvedInteractionNode attaches feat: disclose selector resolution in interaction responses #1193's disclosure, so a guarded replay dispatch that passes the guard still discloses its resolution in full; a guard refusal is pre-action, so no response (and no disclosure) is produced.
  • New pinning tests for the interplay: a guarded press over the split fixture discloses {source:'runtime', phase:'pre-action', kind:'unique'} (dispatch's occlusion-filtered domain is unique there), and a guard-forced runtime-ref click discloses exact ref provenance.
  • Full gate re-run post-rebase: typecheck / lint / format / layering (776 files) / fallow / production-exports clean; 403 unit files (3755 tests) and 32 provider-integration files (120 tests) green.

@thymikee thymikee left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review of ADR-0012 migration step 4 (target-binding verification enforcement + the coordinator's post-resolution guard). Reviewed statically against docs/adr/0012-interactive-replay.md decisions 3/4 and live on iPhone 17 Pro sim (C25DBB5B), RN playground (org.reactnavigation.playground), Metro 8082. Also reviewed the post-rebase pin commit (a3749f26a, test-only, confirms the guard doesn't suppress #1193's resolution disclosure).

Findings

1. (Medium, verified-sound-with-caveat) Pre-promotion guard comparison doesn't guarantee the physical tap stays inside the verified element's own container

assertExpectedResolvedTarget runs on resolved.node before resolveActionableNodeOrThrow's hittable-ancestor promotion (resolution.ts:188,250), and verification (classifyReplayTarget) never promotes either — so both sides of the guard compare the same pre-promotion leaf. That part is internally consistent.

The residual risk is in the promotion step itself, not the guard: resolveActionableTouchResolution (src/core/interaction-targeting.ts:48-74) climbs to findNearestHittableAncestor — the first ancestor with hittable === true — and only refuses via isOverlyBroadAncestor when that ancestor covers ~90%+ of the viewport. Nothing checks that the promoted ancestor is exclusively the verified leaf's own container (e.g. it could be a list-row wrapping two distinct buttons, if the leaf itself isn't independently hittable). In that shape, the guard passes (leaf identity matches), but the coordinates actually tapped are the promoted ancestor's center — which is not guaranteed to be "the same element's actionable container" as the PR's comment (resolution.ts:50-51) asserts. That comment states an invariant the promotion code doesn't itself enforce. Worth either a stronger isOverlyBroadAncestor-style check (reject promotion into an ancestor containing another semantic-touch-role descendant) or softening the code comment's claim. Not a regression introduced by this PR (the promotion logic predates it), but this PR's guard is exactly the mechanism whose safety story leans on that unverified assumption.

2. (Informational, live-reproduced) AX role instability produces a real false-positive identity-mismatch on an unrelated, functionally-unchanged screen

Recording a click "Push Input" on the RN playground's Article screen and replaying it moments later (same screen, no navigation) reproducibly diverged as identity-mismatch: ancestry[0]: recorded=scrollview/Push Input observed=other. Direct snapshot inspection confirmed the same physical container (height 2194px — clearly the screen's ScrollView) is typed scrollview in one capture and Other in the very next, coinciding with the daemon's own "fell back to the private-ax snapshot backend" notice for this deep/complex tree (matches the known bluesky-sparse-ax-root-cause class of issue in this repo). This isn't a defect in this PR's classification code — classifyReplayTarget/classifyTargetBindingMatch faithfully implement decision 3 — but decision 3's own premise ("record and replay compute this ordinal identically by definition") assumes stable AX role reporting across captures, which doesn't hold under this pre-existing backend-fallback condition. Net effect: on deep/complex RN trees (exactly this repo's target apps), step 4 will sometimes fire a spurious pre-action divergence on a screen that hasn't actually changed. Worth a follow-up chip against the AX role-reporting instability itself; not a blocker for this PR since it inherits rather than introduces the instability, and fail-closed (refusing the action) is the correct response given the ADR's own design philosophy.

3. (Verified sound) Guard cannot be spoofed from the wire, and cannot cause a false negative

internal.replayTargetGuard lives on DaemonRequestInternal (src/daemon/types.ts), which is not part of the public wire DaemonRequest (src/kernel/contracts.ts:102-110), and toDaemonRequest in http-server.ts:163-174 builds the internal request field-by-field (allowlist), never spreading client JSON — so no client input can set or forge this field. Only one call site sets it (session-replay-runtime.ts:151), gated on a passed verification outcome. Even a hypothetical forged guard could only ever add a stricter check (fail-closed on mismatch), never bypass one — there's no code path where its presence weakens what would otherwise be checked.

4. (Verified sound) Zero device action on divergence, traced end-to-end

Live ndjson request trace for an induced identity-mismatch divergence shows exactly one ios_runner_command_sendsnapshot_capture pair for the step, then request_failed — no tap/press runner command anywhere in the trace. Combined with the unit-level "zero backend taps" assertions in the split-resolver tests and the fact all four fast paths (direct-iOS tap/fill, native-ref tapTarget/fillTarget) are unconditionally skipped when replayTargetGuard is set (interaction-touch.ts:161-172,548-557, interactions.ts maybeTapRefTarget/maybeFillRefTarget), forcing every guarded dispatch through resolveInteractionTarget, where assertExpectedResolvedTarget throws strictly before any runtime.backend.* call — this holds uniformly across platforms since the guard is checked in platform-agnostic command code, not per-backend.

5. (Verified sound) Identity normalization is shared byte-for-byte across writer/verifier/guard

readNodeLocalIdentity (target-identity-node.ts) is the literal function aliased as boundedLocalIdentity in the record-time writer (session-target-evidence.ts:124) and used directly in the dispatch-side guard (resolution.ts) — same NFC/trim/whitespace-collapse/256-byte-cap normalization on every path. A label differing only in whitespace between captures cannot trip the guard.

6. (Verified sound) Six-path mapping, wire contract, capture cost

classifyTargetBindingMatch (target-identity.ts:425-458) matches the ADR's paths 2–6 exactly, including the signal-isolated-wrong (path 5/6-compare) vs no-signal-isolation (true path-6 fall-through, ≤5 candidates, document order) split. matchCount presence rule is correct (omitted only for path-1 recorded-unverifiable via a real undefined, never null). recorded/observed/mismatches/candidates all route through createReplayDivergenceSanitizer. Live trace confirms verification performs exactly one extra capture per annotated step via the existing captureDivergenceObservation (no settle/stable machinery invoked, no per-signal recapture). Fast-path-forcing is directly unit-tested (tapTargets empty, taps.length === 1 when guarded) and the pre-existing native-ref fast-path regression suite (unmodified by this PR) continues to cover the unguarded case.

Live evidence summary

  • Real annotation recording verified correct per decision 3's format (id-less nodes, ancestry, sibling, viewportOrder, scrollRegion, rect, verification: "verified").
  • Reproduced both selector-miss (app on splash screen at capture) and identity-mismatch (finding #2 above) divergences live, both pre-action, both zero-tap (confirmed via ndjson trace), both carrying correct kind/targetBinding/screen evidence.
  • Did not obtain a fully clean end-to-end pass live — this RN playground app's own navigation-state persistence (documented in the ADR's Context section as a known property of this exact app) combined with finding #2 made a deterministic "nothing changed, all steps verify" run hard to land in the session time available. Given #2 reproduces independent of navigation state, I'm confident this is capture-backend variance, not a step-4 correctness defect — but a maintainer with more runway on this device might want a clean-pass artifact for the record.

Gates

All local: pnpm typecheck, lint, format:check, check:layering (775 files, no back-edge growth), check:fallow (18 changed files, no issues), check:production-exports — all clean. pnpm test:unit (3742 tests) and pnpm test:integration:provider (120 tests): 5 failures on first parallel run, all confirmed CPU-contention flakiness (pass clean in isolation; none touch this PR's files) — matches this repo's known unit-suite-flaky-under-contention pattern.

gh pr checks 1209: all completed checks green (Typecheck, Lint & Format, Layering Guard, Fallow Code Quality, Integration Tests, Coverage, Smoke Tests ×N, Bundle Size, CodeQL, Swift Runner Unit Compile, iOS Runner Swift Compatibility); a couple of Smoke Tests/Coverage shards were still in flight at review time with no failures observed.

Verdict

No blockers. Finding #1 (promotion-exemption assumption not structurally enforced) is worth a follow-up — either tighten isOverlyBroadAncestor-style promotion policy or soften the code comment's claim to match what the code actually guarantees. Finding #2 (AX role instability under the private-ax fallback) is a real, live-reproduced false-positive-divergence source but predates this PR and is orthogonal to its own logic, which faithfully implements the ADR. Everything else checked out sound: no guard false negatives, no wire spoofing surface, zero-device-action on every dispatch path, correct six-path/wire mapping, and capture cost exactly as documented.

@thymikee

Copy link
Copy Markdown
Member Author

Fix re-review at 96141ed6: all three blockers are closed. Healed replay --update actions are reverified and guarded, selector match domains now follow the actually selected chain alternative, and the implementation/tests are split into coherent files below 500 LOC. Focused regressions are non-vacuous, the independent GPT-5.5 re-review found no issues, and all 22 checks are green.

I am still withholding ready-for-human for two evidence gaps required by this routing/device-facing change:

  • practical simulator/device replay evidence on the current head showing an annotated target is verified before action, including the healed --update path or a deliberate target-binding divergence;
  • the separately authorized Claude cross-vendor pass required for this high-risk replay/contract surface.

@thymikee

Copy link
Copy Markdown
Member Author

New sequencing blocker after #1210 merged: this branch is now DIRTY against current main (0a8ea3a57). The correctness fixer at 96141ed6 was clean and 22/22 before that merge, but it must now be rebased onto the new architecture ownership and re-reviewed on the resulting head. The previously noted live replay and cross-vendor evidence gaps also remain.

thymikee added a commit that referenced this pull request Jul 11, 2026
…inner against the verified member

Coordinator addition to step 4 (PR #1209): pre-action verification resolves
without dispatch's occlusion-filtering/visibility-preferring guards, so
verification's winner W could differ from the node dispatch actually acts
on — verified-but-taps-a-different-element, or a false identity-mismatch.
Close the split with a cheap cross-check at dispatch's own resolution:

- Verified outcomes now carry the verified member's normalized local
  identity (`ReplayVerifiedTargetGuard`: readNodeLocalIdentity(winner) +
  verification's matchCount). The replay loop threads it onto the dispatch
  request as `internal.replayTargetGuard`.
- Interaction handlers (press/click/longpress/fill in interaction-touch.ts,
  get in selector-runtime.ts) pass it into command options as
  `expectedResolvedTarget`, and a guarded dispatch skips the direct-iOS
  selector fast path and the native-ref tapTarget/fillTarget fast paths —
  the guard needs the runtime tree resolution to have a node to check.
- `assertExpectedResolvedTarget` (resolution.ts) compares the resolution
  winner's identity — PRE-promotion (hittable-ancestor promotion
  deliberately retargets to the same element's actionable container) and
  PRE-action — against the verified identity, and refuses with
  `details.reason: 'replay_target_guard_mismatch'` on a mismatch. Enforced
  on the selector path, the ref path, and get's ref/selector reads.
- The replay loop converts that refusal into an identity-mismatch
  target-binding divergence (`buildReplayTargetGuardMismatchResponse`):
  classification == kind, matchCount from verification's recorded-selector
  resolution (present, per the rule), recorded/observed identities,
  field-level mismatches, fresh screen observation — via the same
  bounding/sanitizing builder as every other target-binding divergence
  (finalize refactored into module-level
  buildTargetBindingDivergenceResponse for the two callers).
- The identity reader moved to src/replay/target-identity-node.ts
  (readNodeLocalIdentity + localIdentitiesEqual + the reason marker) so the
  commands and daemon layers share it without a layering back-edge;
  session-target-evidence.ts's boundedLocalIdentity is now an alias.

The guard is replay-only by construction: `internal.replayTargetGuard` is
set exclusively by the replay step loop for annotated verified actions;
live interactive commands never carry it.

PR #1211 coordination: buildTargetBindingDivergenceResponse is the single
resume site for target-binding divergences, marked as the step-5 wiring
point — after #1211 merges, replace the not-supported stub with the real
computed resume (pre-action divergences typically allow same-step resume)
plus a test. Kept as a small follow-up by construction.

Tests: split-resolver fixture (covered deeper A vs visible sibling B —
verification verifies A, dispatch resolves B) proving the guard refuses
pre-action with zero backend taps, the matching-identity pass-through, the
ref-fast-path forcing, guard threading on the dispatch request (and its
absence for unannotated actions), and end-to-end conversion of a guard
mismatch into an identity-mismatch divergence with matchCount present.
@thymikee thymikee force-pushed the feat/replay-target-verification branch from 96141ed to 12204cb Compare July 11, 2026 12:18
@thymikee

Copy link
Copy Markdown
Member Author

Rebased onto current main after #1210 merged (architecture-ownership consolidation). Branch force-pushed with lease — re-fetch if running against the old head; new head is 12204cbc9.

Non-trivial resolutions:

Full gate re-run post-rebase: typecheck / lint / format / layering (794 files) / fallow / production-exports clean; 3769 unit tests (406 files) and 121 provider-integration tests (33 files) green.

Ready for #1211 to merge first as planned — I'll do the one small follow-up rebase to wire buildReplayDivergenceResume into the target-binding resume stub afterward.

…lution guard + step-5 resume wiring

Rebuilt onto current main (#1210 selector relocation, #1211 resume + --update
retirement, #1213, #1215) as one coherent commit — the prior 4-commit history
collided structurally with #1211's own session-replay-runtime split.

Step 4 (decision 3, enforcement): every annotated resolved-target replay/test
action is verified against a fresh pre-action snapshot BEFORE dispatch
(verifyReplayActionTarget → classifyReplayTarget, decision 3's six paths). Only
a verified outcome sends the action; selector-miss / identity-mismatch /
identity-unverifiable each return a complete target-binding REPLAY_DIVERGENCE
built from its own capture, with targetBinding {classification==kind,
matchCount per the presence rule, recorded/observed, mismatches, candidates}
through #1211's shared failure-response + bounding/sanitizer pipeline.

Post-resolution guard: a verified outcome mints internal.replayTargetGuard
(the verified member's identity); dispatch's own resolution (occlusion/
visibility guards verification does not replicate) must land on the SAME
element or assertExpectedResolvedTarget refuses pre-action, which the loop
converts to an identity-mismatch divergence. Guarded dispatch skips the
direct-iOS and native-ref fast paths so a tree node exists to check.

Step-5 resume wiring (this rebase): the target-binding divergence resume stub
is replaced with #1211's buildReplayDivergenceResume — a pre-action divergence
resumes AT the failed step (from = step index, real SHA-256 planDigest, allowed
unless preflight rejects the skipped range). buildTargetBindingDivergenceResponse
is the single resume site; planActions/planDigest are threaded from the loop.

--update healing is retired by #1211, so the in-branch "verify healed action"
work is dropped (its module and test removed as moot).

Tests: classification (13), guard (4), token (5), runtime end-to-end (9, incl.
old-script pass-through, guard threading + mismatch conversion, and a real
computed resume assertion), plus target-binding wire tests in divergence.test.ts
(classification==kind, matchCount presence, overflow-fallback targetBinding,
computed-resume-not-stub, text report). Also repoints
daemon-http-disconnect.test.ts already handled upstream by #1215.
@thymikee thymikee force-pushed the feat/replay-target-verification branch from 12204cb to 41b61c8 Compare July 11, 2026 14:15
@thymikee

Copy link
Copy Markdown
Member Author

Rebased onto current main (now includes #1211, #1215, #1213) + resume-stub wiring done. Branch rebuilt as one coherent commit — the prior 4-commit history collided structurally with #1211's own session-replay-runtime split. New head: 41b61c887.

Key points:

Full gate green: typecheck / lint / format / layering (798 files, zero back-edges) / fallow / production-exports. 3798 unit tests (412 files) and 121 provider-integration tests (33 files) pass, including #1211's own resume/plan-digest suites.

@coordinator — green and ready for the re-review pass on the resume wiring.

@thymikee

Copy link
Copy Markdown
Member Author

Current-head blocker at 41b61c887:

P1 — the post-resolution guard can still authorize a different duplicate element. ADR 0012 path 6 may use sibling or region-scoped viewportOrder to isolate one member among several nodes sharing the same local identity. Verification then stores only {id, role, label} in replayTargetGuard, and dispatch accepts any winner with that same local identity. If verification isolates duplicate A but dispatch's visibility/occlusion filtering selects duplicate B with identical local identity, the guard passes and sends the action. Existing split-resolver tests avoid this by assigning different ids, while same-local-identity path-6 cases are tested only in the pure classifier.

Carry enough structural denotation for the verified member into the guard and compare it against the dispatch capture before promotion. Add an end-to-end regression with two same-local-identity candidates where verification denotes A, dispatch resolves B, and assert REPLAY_DIVERGENCE with zero backend calls. No fixer has been started.

@thymikee

Copy link
Copy Markdown
Member Author

Final review + live verification evidence (head 41b61c8)

Scoped final pass per the coordinator's request: (A) the resume-wiring delta only (correctness engine already reviewed clean in prior rounds), and (B) the live device evidence gap noted for ready-for-human. Done in an isolated scratch worktree (/tmp/agent-device-pr1209-review, detached at 41b61c887); main checkout untouched.

(A) Static — resume-wiring delta

Traced the full wiring: runReplayScriptFile computes planDigest once via computeReplayPlanDigest (session-replay-runtime.ts:80) and threads {planActions, planDigest} unchanged through every call site — verifyReplayActionTarget, buildReplayTargetGuardMismatchResponse, and the pre-existing action-failure path in session-replay-divergence.ts. All three converge on the single buildTargetBindingDivergenceResponse (session-replay-target-verification.ts:86-157), which is the one call site for buildReplayDivergenceResume (session-replay-resume.ts:64) for both the verify path and the guard-mismatch path.

  1. from index — correct. resume.from = step = index + 1 (the failed step's own 1-based plan index), not step + 1, for both the pre-action target-binding divergence and the guard-mismatch conversion (session-replay-target-verification.ts:135, :177). This matches ADR-0012 decision 4's stated resume loop ("...replay with the reported plan digest and index") and is consistent with the pre-existing action-failure divergence, which uses the identical index + 1 formula (session-replay-divergence.ts:117) — i.e. resume always targets the reported failed step, by design, regardless of whether the step ran. Verified live below.
  2. allowed — real preflight, not blanket true. buildReplayDivergenceResume calls evaluateReplayResumePreflight, which rejects when any skipped step (1..from-1) can produce outputEnv (invokeMaestroRunScript) or is inside runtime control flow, or when the resume target itself is control-flow interior. No shortcut.
  3. planDigest — single source, no drift. One computeReplayPlanDigest call per replay invocation; every divergence branch (verify, guard-mismatch, action-failure) reads the same threaded value. No re-derivation anywhere in the diff.
  4. Test coverage — gap. resume not yet supported is fully gone from production code (only pre-existing, unrelated fixtures in output.test.ts/errors.test.ts/etc. still reference the string as generic wire-format examples). One new end-to-end test (session-replay-target-verification-runtime.test.ts:377, "carries a real computed resume, not the retired stub") asserts a genuine computed resume — but only on the selector-miss classification. The identity-mismatch test (:154) and the dedicated guard-mismatch test (:309, and session-replay-target-guard.test.ts) assert targetBinding fields but do not assert resume at all. Since all paths provably funnel through the same buildTargetBindingDivergenceResponse (confirmed by static trace, and independently by the live guard-adjacent selector-miss run below returning a correct resume), this is a coverage gap rather than a suspected bug — worth a follow-up test but not a blocker.

Verdict: resume-wiring delta is correct. from = failed step index is right for a pre-action divergence (never re-runs an already-executed action, matches ADR text). Recommend a small follow-up PR adding explicit resume assertions on the identity-mismatch and guard-mismatch tests for belt-and-suspenders coverage.

(B) Live evidence

Built the branch in the scratch worktree (pnpm install && pnpm build:node), ran against sim iPhone 17 Pro (C25DBB5B-9254-4293-A8D5-2785C78DE03A, already booted), RN playground org.reactnavigation.playground on Metro 8082 (own session spot-1209, --state-dir /tmp/spot-1209-state). Never touched 8081/PagerView.

1. Recorded open --relaunch + deep link rne://stack-prevent-remove + wait 3000 + 4 label presses (Push Article → Push Input → Discard and go back → Pop to top, idempotent round trip back to the root Input screen). The recorder wrote target-v1 annotations on all 4 press steps, all "verification":"verified":

# agent-device:target-v1 {"role":"button","label":"Push Article", ... "verification":"verified"}
press "role=\"button\" label=\"Push Article\" || label=\"Push Article\""
# agent-device:target-v1 {"role":"button","label":"Push Input", ... "verification":"verified"}
press "role=\"button\" label=\"Push Input\" || label=\"Push Input\""
# agent-device:target-v1 {"role":"button","label":"Discard and go back", ... "verification":"verified"}
press "role=\"button\" label=\"Discard and go back\" || label=\"Discard and go back\""
# agent-device:target-v1 {"role":"button","label":"Pop to top", ... "verification":"verified"}
press "role=\"button\" label=\"Pop to top\" || label=\"Pop to top\""

2. VERIFIED PASS — replay on the matching (freshly recorded) app state:

Replayed 8 steps in 16.4s

All 4 annotated steps verified and executed cleanly — no false positives on a matching screen.

3. TARGET-BINDING DIVERGENCE, PRE-ACTION — navigated the app to the Article screen (via an unrecorded manual press, outside the script), then replayed a single-action script carrying the real recorded annotation for "Discard and go back" (which only exists on the Input screen, not Article). Real REPLAY_DIVERGENCE:

"divergence": {
  "kind": "selector-miss",
  "cause": { "code": "SELECTOR_MISS", "message": "The recorded target no longer matches any current element." },
  "resume": { "allowed": true, "from": 1, "planDigest": "a6327965e5bea5f08e29a97c87e7b95f59897eff4d4f09a0129ea7b04fdfdbfb" },
  "targetBinding": {
    "classification": "selector-miss",
    "matchCount": 0,
    "recorded": { "role": "button", "label": "Discard and go back" },
    "mismatches": [],
    "candidates": []
  }
}

resume.from: 1 matches the failed (and only) step — a real computed object, not the stub, confirming the static finding live.

Zero-tap proof — the request's per-request ndjson (--debug) shows exactly one ios_runner_command_send, and it's a read-only snapshot, not a tap:

{"phase":"ios_runner_command_send", ..., "data":{"command":"snapshot","commandId":"...","readOnly":true, ...}}
{"phase":"snapshot_capture", ..., "data":{"backend":"xctest"}}
{"phase":"platform_command", ..., "data":{"command":"snapshot","platform":"apple"}}
{"phase":"request_failed", ...}

No press/tap command was ever sent to the runner — verification blocked the wrong action pre-execution, exactly as designed. Did not hit the private-AX-role-flap false-positive risk on this run.

4. Guard path — skipped. Forcing dispatch's occlusion/visibility resolver to disagree with verification requires an engineered overlapping-sibling layout; not cheaply reachable in the RN playground's real screens. The guard path already has dedicated unit/integration coverage (session-replay-target-guard.test.ts, split-resolver fixture) from prior rounds — not re-litigated here.

Overall

APPROVE-worthy. Resume wiring is correct (from = failed step index, real preflight, single-source digest). Live evidence confirms both the happy path (clean verified replay) and the safety property (pre-action divergence blocks the device tap, with a real computed resume object). CI is green (all checks passing, including previously-pending Smoke Tests). One non-blocking follow-up: add explicit resume assertions to the identity-mismatch and guard-mismatch tests.

…ard-mismatch paths

Non-blocking review follow-up: only the selector-miss runtime test asserted
the computed resume {allowed, from, planDigest}. All three target-binding
paths funnel through buildTargetBindingDivergenceResponse (one shared resume
site), so this is coverage not a bug — but pin resume on the identity-mismatch
and guard-mismatch tests too so no path can regress its resume wiring silently.
@thymikee

Copy link
Copy Markdown
Member Author

Closed the non-blocking resume-coverage gap: added the computed resume { allowed, from, planDigest } assertion to both the identity-mismatch and guard-mismatch runtime tests (previously only selector-miss pinned it). All three funnel through the one shared buildTargetBindingDivergenceResponse, so this is regression-proofing rather than a fix. Pushed as 0c66e67b4; full gate green (typecheck/lint/format/layering/fallow/production-exports).

…rd (P1)

The guard carried only LOCAL identity {id, role, label}. ADR path 6 isolates
ONE member among several SAME-local-identity nodes via sibling / region-scoped
viewportOrder — a discriminator lost at the guard. So if verification denoted
duplicate A but dispatch's occlusion/visibility filter selected duplicate B
(identical id/role/label), the guard passed and SENT the action on the wrong
element — the exact verified-but-taps-different-node mis-binding step 4 exists
to prevent.

Fix: replayTargetGuard now carries a structural denotation of the verified
member — its pre-order document index (decision 3's canonical total order,
`node.index` after the shared reindex pipeline) plus its same-parent sibling
ordinal. `assertExpectedResolvedTarget` compares BOTH local identity AND
structural denotation against dispatch's pre-promotion capture, refusing
pre-action when EITHER differs. Two distinct duplicates always differ in
document order, so a different-member resolution can no longer pass. The
promotion exemption is preserved (comparison is on the verified LEAF, and
duplicates are distinct leaves). New helpers live in target-identity-node.ts
(replay zone) so writer/classifier/guard share one sibling-ordinal
implementation; computeSiblingOrdinal now delegates to it.

The guard-mismatch divergence surfaces a `position:` mismatch line (from the
structural denotations) so `mismatches` is never empty when local identity is
identical.

END-TO-END regression (session-replay-target-guard.test.ts): two "Save"
buttons with IDENTICAL id/role/label, A covered+deeper (verification's winner),
B visible (dispatch's winner) — driven through the real interaction runtime
(`device.interactions.press`), the guard refuses pre-action with ZERO backend
taps AND zero native tapTargets, the refusal driven purely by the structural
denotation (both observedStructural/expectedStructural asserted). Plus a
same-structural passing case, a runtime-level `position:`-mismatch divergence
test, and updated guard-threading assertions for the {identity, structural}
shape.

Wire contract unchanged: replayTargetGuard is DaemonRequestInternal (never on
the wire); decision 4's targetBinding {recorded, observed} stay LOCAL identity
per decision 3's Identity tier — the structural denotation is an internal
guard concern, surfaced only as a `mismatches` string. No ADR change needed.
@thymikee

Copy link
Copy Markdown
Member Author

P1 fixed (head f4a60c99d) — point-by-point on the guard-authorizes-a-different-duplicate blocker:

  1. Root cause confirmed & closed. replayTargetGuard carried only {id, role, label}, so two same-local-identity path-6 duplicates were indistinguishable at the guard. It now also carries a structural denotation of the verified member: its pre-order document index (node.index after the shared reindex pipeline — decision 3's "canonical total order of this contract") plus its same-parent sibling ordinal (path 6.i).

  2. Compared before promotion, refuses on EITHER mismatch. assertExpectedResolvedTarget now compares local identity AND structural denotation against dispatch's pre-promotion capture, throwing replay_target_guard_mismatch when either differs. Two distinct duplicates always differ in document order, so a different-member resolution can no longer pass. Promotion exemption preserved — the comparison is on the verified leaf, and duplicates are distinct leaves with distinct denotations.

  3. END-TO-END regression (not pure-classifier). session-replay-target-guard.test.ts: two "Save" buttons with identical {id, role, label}, A covered+deeper (verification's winner, doc-order 1), B visible (dispatch's occlusion-filter winner, doc-order 2). Driven through the real interaction runtime (device.interactions.pressresolveInteractionTarget → the occlusion filter picks B → guard). Asserts REPLAY_DIVERGENCE-path refusal with zero backend taps AND zero native tapTargets, and that the refusal is driven purely by the structural denotation (observed.id === expected.id === 'save'; expectedStructural doc1/sibling0 vs observedStructural doc2/sibling1). Plus a same-structural passing case and a runtime-level test asserting the divergence's mismatches carries a position: line.

  4. Shared implementation, no drift. The structural helpers live in target-identity-node.ts (replay zone); computeSiblingOrdinal now delegates to the shared siblingOrdinal, so writer/classifier/guard compute it identically. Both captures run the same snapshot pipeline, so the same physical node gets the same denotation; any residual tree difference fails closed (refuse), matching the ADR's philosophy.

ADR/wire-contract check (per your ask): no change needed. replayTargetGuard is DaemonRequestInternal — never on the wire (unforgeable, as your finding #3 established). Decision 4's targetBinding.{recorded, observed} stay LOCAL identity per decision 3's Identity tier; the structural denotation is an internal guard concern, surfaced only as a mismatches string (position: recorded=doc1/sibling0 observed=doc2/sibling1) so the divergence is never empty when local identity is identical.

Also addresses your earlier finding #1 (promotion-exemption assumption): the guard no longer leans on "same element's container" for correctness — it compares the verified leaf's structural denotation, which distinguishes distinct leaves regardless of promotion.

Full gate green: typecheck / lint / format / layering (798 files, zero back-edges) / fallow / production-exports. 3802 unit tests (412 files) and 121 provider-integration tests (33 files) pass.

@thymikee

Copy link
Copy Markdown
Member Author

Re-review of f4a60c99: the same-identity duplicate false-negative is closed, but the replacement guard introduces a new P1.

P1 — raw node.index is capture-local and cannot identify an element across verification and dispatch snapshots. Verification stores documentOrder: node.index; dispatch captures again and requires exact equality. Android assigns indexes from the current filtered output order, and Apple assigns them from the current tree/flat-query representation. An unrelated preceding node appearing/disappearing, filtering differences, or an Apple capture-backend change can shift the same physical target from N to N+1 and cause a false replay_target_guard_mismatch.

The new duplicate regression is non-vacuous for the prior bug, but it passes the same SnapshotState to both sides, so it cannot exercise this production boundary. Resolve dispatch against the verifier's captured state, or compare a capture-independent structural path within equivalent domains. Add a two-capture regression where the same target retains identity/ancestry/sibling while its global index shifts and the action still proceeds; retain the duplicate-other-member zero-action regression. Practical iOS and Android evidence should cover both scenarios. No fixer has been started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant