Skip to content

Fix locator-healing pipeline issues from reporter capture to diagnosis#201

Merged
PhenX merged 2 commits into
mainfrom
claude/locator-heal-tools-inspect-ole4z5
Jul 4, 2026
Merged

Fix locator-healing pipeline issues from reporter capture to diagnosis#201
PhenX merged 2 commits into
mainfrom
claude/locator-heal-tools-inspect-ole4z5

Conversation

@PhenX

@PhenX PhenX commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Capture (reporter + dogfooding fixtures):

  • Stop capturing live input value — filled secrets (passwords, tokens)
    leaked into snapshots, the wire payload, and locator_snapshots storage,
    and no alternative generator ever used it. Capture multiple instead.
  • Map plain to combobox (listbox only with multiple), matching ARIA and the server's implicitRoleForTag — getByRole('listbox') suggestions never matched real dropdowns. Gate getByLabel on a real /aria-label (new hasLabel probe): names approximated from placeholder/title produced broken getByLabel suggestions ranked above the working getByPlaceholder. Probe candidate selectors for uniqueness (querySelectorAll counts) and drop ambiguous testid/id/name/class alternatives — they were strict-mode violations waiting to be pasted. Selector hygiene: attribute-selector fallback for non-CSS-safe ids (React useId :r3:), skip Tailwind-variant classes as bare selectors, detect radix-/headlessui-/mui-/useId auto-generated ids, collapse whitespace so getByText can't contain literal newlines. Add pressSequentially/waitFor/dispatchEvent/selectText to the action surface (type was covered but its replacement wasn't). Dedupe repeated call sites before attaching; drop unused stepIndex; clear watchdog timers; sync the dogfooding fixture (incl. the version-tolerant ariaSnapshotBestEffort it had drifted away from). Server lookup: Replace raw NUL bytes in the upsert key with \x00 escapes — git/grep treated the whole file as binary. extractErrorLocation now handles named stack frames and skips node_modules/node: frames; the dead file-only fallback (which could never match a stored location) is gone. sameFileLine matches path suffixes so absolute error frames still hit cwd-relative captures. New cross-test ladder (2.5): the same locator signature captured by another test in the project heals assert-only locators; freshest snapshot wins. Backed by a new used_args_fp index (both dialects). suggestAddTestId is only kept for stability-scored sources; ARIA relevance scores no longer trigger the misleading "add a data-testid" advice (UI alert + AI prompt rule inherit the fix). Expose capturedAt (snapshot lastSeenAt) for freshness display. AI context: parseLocatorFromError returned 'Text' instead of 'getByText', rendering a malformed "Requested:" line; the nearest-ARIA section parsed a [role=...] format ariaSnapshot never emits (dead section) — both now reuse the shared parseAriaCandidates/textSimilarity, and the section is skipped when locator healing already provides evidence. Cleanup: Remove dead normalizeAndHashArgs (+ tests); share LocatorHealingResult via shared/locator-healing.types (server/panel/MCP now agree); new drift-guard test pins the reporter's hand-mirrored helpers to the shared implementations. Co-Authored-By: Claude Fable 5 noreply@anthropic.com Claude-Session: https://claude.ai/code/session_01HcMZHXdxwYAhQgosxueCz6

Capture (reporter + dogfooding fixtures):
- Stop capturing live input `value` — filled secrets (passwords, tokens)
  leaked into snapshots, the wire payload, and locator_snapshots storage,
  and no alternative generator ever used it. Capture `multiple` instead.
- Map plain <select> to combobox (listbox only with `multiple`), matching
  ARIA and the server's implicitRoleForTag — getByRole('listbox')
  suggestions never matched real dropdowns.
- Gate getByLabel on a real <label>/aria-label (new hasLabel probe):
  names approximated from placeholder/title produced broken getByLabel
  suggestions ranked above the working getByPlaceholder.
- Probe candidate selectors for uniqueness (querySelectorAll counts) and
  drop ambiguous testid/id/name/class alternatives — they were
  strict-mode violations waiting to be pasted.
- Selector hygiene: attribute-selector fallback for non-CSS-safe ids
  (React useId `:r3:`), skip Tailwind-variant classes as bare selectors,
  detect radix-/headlessui-/mui-/useId auto-generated ids, collapse
  whitespace so getByText can't contain literal newlines.
- Add pressSequentially/waitFor/dispatchEvent/selectText to the action
  surface (type was covered but its replacement wasn't).
- Dedupe repeated call sites before attaching; drop unused stepIndex;
  clear watchdog timers; sync the dogfooding fixture (incl. the
  version-tolerant ariaSnapshotBestEffort it had drifted away from).

Server lookup:
- Replace raw NUL bytes in the upsert key with \x00 escapes — git/grep
  treated the whole file as binary.
- extractErrorLocation now handles named stack frames and skips
  node_modules/node: frames; the dead file-only fallback (which could
  never match a stored location) is gone. sameFileLine matches path
  suffixes so absolute error frames still hit cwd-relative captures.
- New cross-test ladder (2.5): the same locator signature captured by
  another test in the project heals assert-only locators; freshest
  snapshot wins. Backed by a new used_args_fp index (both dialects).
- suggestAddTestId is only kept for stability-scored sources; ARIA
  relevance scores no longer trigger the misleading "add a data-testid"
  advice (UI alert + AI prompt rule inherit the fix).
- Expose capturedAt (snapshot lastSeenAt) for freshness display.

AI context:
- parseLocatorFromError returned 'Text' instead of 'getByText',
  rendering a malformed "Requested:" line; the nearest-ARIA section
  parsed a [role=...] format ariaSnapshot never emits (dead section) —
  both now reuse the shared parseAriaCandidates/textSimilarity, and the
  section is skipped when locator healing already provides evidence.

Cleanup:
- Remove dead normalizeAndHashArgs (+ tests); share LocatorHealingResult
  via shared/locator-healing.types (server/panel/MCP now agree); new
  drift-guard test pins the reporter's hand-mirrored helpers to the
  shared implementations.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HcMZHXdxwYAhQgosxueCz6

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens Piwi’s locator-healing pipeline end-to-end (reporter capture → server storage/lookup → UI + AI diagnosis context) to reduce secret leakage, eliminate invalid/ambiguous suggestions, and improve matching accuracy (including a new cross-test reuse ladder).

Changes:

  • Reporter capture: stop capturing live input value, add uniqueness probing for candidate selectors, improve role/name handling (e.g. <select> combobox vs listbox), and dedupe repeated call sites.
  • Server lookup & API payload: improve stack-frame location extraction, add cross-test healing via used_args_fp index, expose capturedAt, and centralize LocatorHealingResult in shared types.
  • AI + UI: fix locator parsing in AI context, reuse shared ARIA parsing/similarity, skip redundant sections when healing exists, and display snapshot freshness in the panel.

Reviewed changes

Copilot reviewed 23 out of 25 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
reporter/tests/locator-healing.spec.ts Expands unit coverage for new capture/alternative-generation behaviors (select role mapping, secret guard, whitespace collapse, uniqueness gating, dedupe).
reporter/src/internal/capture/locator-healing.ts Adds dedupe-by-location helper, uniqueness-aware alternative generation, safer selector generation, expanded action surface, and exports helper functions for drift guarding.
reporter/src/internal/capture/capture-fixtures.ts Adds hasLabel + selector uniqueness probing, collapses whitespace in captured text, clears watchdog timers, and dedupes snapshots before attaching.
docs/reporter.md Documents uniqueness probing, no live-value capture, and adds cross-test lookup step to the server ladder.
application/types/api.ts Switches locatorHealing.source coverage typing to the shared LocatorHealingSource union.
application/tests/unit/reporter-shared-drift.test.ts Adds a unit “drift guard” to pin reporter-mirrored helpers to shared implementations.
application/tests/unit/locator-healing.test.ts Removes tests for deleted normalizeAndHashArgs.
application/tests/fixtures.ts Syncs dogfooding fixtures with reporter capture changes (dedupe, bounded ariaSnapshot, uniqueness probe, shape changes).
application/shared/locator-healing.types.ts Adds SelectorCounts, removes stepIndex, and defines shared LocatorHealingSource + LocatorHealingResult (+ capturedAt).
application/shared/locator-healing.ts Removes dead normalizeAndHashArgs implementation.
application/server/utils/mcp/tools.ts Includes capturedAt in MCP locator-healing payload.
application/server/utils/locator-healing.ts Adds cross-test healing ladder, improves location parsing & matching, escapes NUL in map keys, clears misleading suggestAddTestId for non-stability-scored sources, and returns capturedAt.
application/server/utils/ai-context.ts Fixes locator parsing, reuses shared ARIA parsing/similarity, skips redundant nearest-ARIA section when healing exists, and includes capturedAt in the healing section.
application/server/database/schema.sqlite.ts Adds idx_locator_snapshots_args_fp for cross-test lookup.
application/server/database/schema.pg.ts Adds idx_locator_snapshots_args_fp for cross-test lookup.
application/server/database/migrations/meta/0026_snapshot.json Captures migration snapshot metadata for sqlite migration 0026.
application/server/database/migrations/meta/_journal.json Records sqlite migration 0026 in the journal.
application/server/database/migrations/0026_careful_wendigo.sql Adds sqlite index migration for locator_snapshots.used_args_fp.
application/server/database/migrations-pg/meta/_journal.json Records Postgres migration 0024 in the PG journal.
application/server/database/migrations-pg/0024_naive_black_widow.sql Adds PG index migration for locator_snapshots.used_args_fp.
application/server/api/test-runs/[id]/cases/[caseId]/locator-healing.get.ts Updates OpenAPI description to reflect cross-test snapshot reuse.
application/app/utils/help-content.ts Updates help copy to mention cross-test locator reuse.
application/app/components/shared/LocatorHealingPanel.vue Uses shared LocatorHealingResult, adds cross-test source note, and displays relative “captured” freshness when available.
AGENTS.md Updates locator-healing architectural notes to reflect the new capture probes, dedupe, cross-test ladder, and suggestion semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread application/tests/unit/reporter-shared-drift.test.ts Outdated
npm run app:test:unit must pass without a prior reporter build — dist/
is not committed and only CI builds it before the unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HcMZHXdxwYAhQgosxueCz6
@PhenX PhenX merged commit a29718d into main Jul 4, 2026
2 checks passed
@PhenX PhenX deleted the claude/locator-heal-tools-inspect-ole4z5 branch July 4, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants