Fix locator-healing pipeline issues from reporter capture to diagnosis#201
Merged
Conversation
Capture (reporter + dogfooding fixtures):
- Stop capturing live input `value` — filled secrets (passwords, tokens)
leaked into snapshots, the wire payload, and locator_snapshots storage,
and no alternative generator ever used it. Capture `multiple` instead.
- Map plain <select> to combobox (listbox only with `multiple`), matching
ARIA and the server's implicitRoleForTag — getByRole('listbox')
suggestions never matched real dropdowns.
- Gate getByLabel on a real <label>/aria-label (new hasLabel probe):
names approximated from placeholder/title produced broken getByLabel
suggestions ranked above the working getByPlaceholder.
- Probe candidate selectors for uniqueness (querySelectorAll counts) and
drop ambiguous testid/id/name/class alternatives — they were
strict-mode violations waiting to be pasted.
- Selector hygiene: attribute-selector fallback for non-CSS-safe ids
(React useId `:r3:`), skip Tailwind-variant classes as bare selectors,
detect radix-/headlessui-/mui-/useId auto-generated ids, collapse
whitespace so getByText can't contain literal newlines.
- Add pressSequentially/waitFor/dispatchEvent/selectText to the action
surface (type was covered but its replacement wasn't).
- Dedupe repeated call sites before attaching; drop unused stepIndex;
clear watchdog timers; sync the dogfooding fixture (incl. the
version-tolerant ariaSnapshotBestEffort it had drifted away from).
Server lookup:
- Replace raw NUL bytes in the upsert key with \x00 escapes — git/grep
treated the whole file as binary.
- extractErrorLocation now handles named stack frames and skips
node_modules/node: frames; the dead file-only fallback (which could
never match a stored location) is gone. sameFileLine matches path
suffixes so absolute error frames still hit cwd-relative captures.
- New cross-test ladder (2.5): the same locator signature captured by
another test in the project heals assert-only locators; freshest
snapshot wins. Backed by a new used_args_fp index (both dialects).
- suggestAddTestId is only kept for stability-scored sources; ARIA
relevance scores no longer trigger the misleading "add a data-testid"
advice (UI alert + AI prompt rule inherit the fix).
- Expose capturedAt (snapshot lastSeenAt) for freshness display.
AI context:
- parseLocatorFromError returned 'Text' instead of 'getByText',
rendering a malformed "Requested:" line; the nearest-ARIA section
parsed a [role=...] format ariaSnapshot never emits (dead section) —
both now reuse the shared parseAriaCandidates/textSimilarity, and the
section is skipped when locator healing already provides evidence.
Cleanup:
- Remove dead normalizeAndHashArgs (+ tests); share LocatorHealingResult
via shared/locator-healing.types (server/panel/MCP now agree); new
drift-guard test pins the reporter's hand-mirrored helpers to the
shared implementations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HcMZHXdxwYAhQgosxueCz6
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens Piwi’s locator-healing pipeline end-to-end (reporter capture → server storage/lookup → UI + AI diagnosis context) to reduce secret leakage, eliminate invalid/ambiguous suggestions, and improve matching accuracy (including a new cross-test reuse ladder).
Changes:
- Reporter capture: stop capturing live input
value, add uniqueness probing for candidate selectors, improve role/name handling (e.g.<select>combobox vs listbox), and dedupe repeated call sites. - Server lookup & API payload: improve stack-frame location extraction, add cross-test healing via
used_args_fpindex, exposecapturedAt, and centralizeLocatorHealingResultin shared types. - AI + UI: fix locator parsing in AI context, reuse shared ARIA parsing/similarity, skip redundant sections when healing exists, and display snapshot freshness in the panel.
Reviewed changes
Copilot reviewed 23 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| reporter/tests/locator-healing.spec.ts | Expands unit coverage for new capture/alternative-generation behaviors (select role mapping, secret guard, whitespace collapse, uniqueness gating, dedupe). |
| reporter/src/internal/capture/locator-healing.ts | Adds dedupe-by-location helper, uniqueness-aware alternative generation, safer selector generation, expanded action surface, and exports helper functions for drift guarding. |
| reporter/src/internal/capture/capture-fixtures.ts | Adds hasLabel + selector uniqueness probing, collapses whitespace in captured text, clears watchdog timers, and dedupes snapshots before attaching. |
| docs/reporter.md | Documents uniqueness probing, no live-value capture, and adds cross-test lookup step to the server ladder. |
| application/types/api.ts | Switches locatorHealing.source coverage typing to the shared LocatorHealingSource union. |
| application/tests/unit/reporter-shared-drift.test.ts | Adds a unit “drift guard” to pin reporter-mirrored helpers to shared implementations. |
| application/tests/unit/locator-healing.test.ts | Removes tests for deleted normalizeAndHashArgs. |
| application/tests/fixtures.ts | Syncs dogfooding fixtures with reporter capture changes (dedupe, bounded ariaSnapshot, uniqueness probe, shape changes). |
| application/shared/locator-healing.types.ts | Adds SelectorCounts, removes stepIndex, and defines shared LocatorHealingSource + LocatorHealingResult (+ capturedAt). |
| application/shared/locator-healing.ts | Removes dead normalizeAndHashArgs implementation. |
| application/server/utils/mcp/tools.ts | Includes capturedAt in MCP locator-healing payload. |
| application/server/utils/locator-healing.ts | Adds cross-test healing ladder, improves location parsing & matching, escapes NUL in map keys, clears misleading suggestAddTestId for non-stability-scored sources, and returns capturedAt. |
| application/server/utils/ai-context.ts | Fixes locator parsing, reuses shared ARIA parsing/similarity, skips redundant nearest-ARIA section when healing exists, and includes capturedAt in the healing section. |
| application/server/database/schema.sqlite.ts | Adds idx_locator_snapshots_args_fp for cross-test lookup. |
| application/server/database/schema.pg.ts | Adds idx_locator_snapshots_args_fp for cross-test lookup. |
| application/server/database/migrations/meta/0026_snapshot.json | Captures migration snapshot metadata for sqlite migration 0026. |
| application/server/database/migrations/meta/_journal.json | Records sqlite migration 0026 in the journal. |
| application/server/database/migrations/0026_careful_wendigo.sql | Adds sqlite index migration for locator_snapshots.used_args_fp. |
| application/server/database/migrations-pg/meta/_journal.json | Records Postgres migration 0024 in the PG journal. |
| application/server/database/migrations-pg/0024_naive_black_widow.sql | Adds PG index migration for locator_snapshots.used_args_fp. |
| application/server/api/test-runs/[id]/cases/[caseId]/locator-healing.get.ts | Updates OpenAPI description to reflect cross-test snapshot reuse. |
| application/app/utils/help-content.ts | Updates help copy to mention cross-test locator reuse. |
| application/app/components/shared/LocatorHealingPanel.vue | Uses shared LocatorHealingResult, adds cross-test source note, and displays relative “captured” freshness when available. |
| AGENTS.md | Updates locator-healing architectural notes to reflect the new capture probes, dedupe, cross-test ladder, and suggestion semantics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
npm run app:test:unit must pass without a prior reporter build — dist/ is not committed and only CI builds it before the unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HcMZHXdxwYAhQgosxueCz6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Capture (reporter + dogfooding fixtures):
value— filled secrets (passwords, tokens)leaked into snapshots, the wire payload, and locator_snapshots storage,
and no alternative generator ever used it. Capture
multipleinstead.