Skip to content

fix: add curated suggestions for open-url and close-session#1175

Merged
thymikee merged 1 commit into
mainfrom
fix/vocab-suggestions
Jul 10, 2026
Merged

fix: add curated suggestions for open-url and close-session#1175
thymikee merged 1 commit into
mainfrom
fix/vocab-suggestions

Conversation

@thymikee

Copy link
Copy Markdown
Member

Summary

  • The 2026-07-09 help-conformance benchmark run showed agents guessing open-url and close-session as command names (1x each), and the CLI produced no "Did you mean" hint for either — the Levenshtein nearest-name fallback doesn't cover them because both tokens exceed the edit-distance threshold for their length versus the real command names (open, close).
  • Verified pre-fix behavior directly: node bin/agent-device.mjs open-url and node bin/agent-device.mjs close-session both returned a bare Unknown command: ... with no hint.
  • Added curated entries in src/cli/parser/command-suggestions.ts following the existing convention: open-urlopen <url>, close-sessionclose.
  • Added unit tests mirroring the existing per-token test pattern in src/cli/parser/__tests__/command-suggestions.test.ts.

Test plan

  • node bin/agent-device.mjs open-url now prints Unknown command: open-url. Did you mean open <url>?
  • node bin/agent-device.mjs close-session now prints Unknown command: close-session. Did you mean close?
  • npx vitest run src/cli/parser — 170/170 passing (includes the two new cases plus the registry-drift guards)
  • pnpm lint, pnpm typecheck, pnpm format:check, pnpm check:layering all clean
  • npx fallow audit --base origin/main — no issues in the 2 changed files

Last night's help-conformance benchmark showed agents guessing
open-url and close-session with no "did you mean" hint (the
Levenshtein fallback doesn't cover either token — both exceed the
edit-distance threshold for their length). Add curated entries
pointing them at `open <url>` and `close`.
@github-actions

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.6 MB 1.6 MB +99 B
JS gzip 517.3 kB 517.3 kB +29 B
npm tarball 623.4 kB 623.5 kB +41 B
npm unpacked 2.2 MB 2.2 MB +99 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.0 ms 28.7 ms +0.7 ms
CLI --help 57.1 ms 57.4 ms +0.3 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/cli.js +99 B +29 B

@thymikee thymikee left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review, done in a scratch worktree off origin/fix/vocab-suggestions (44c1cff). Tiny, well-scoped change — quick pass as requested.

Findings

No blockers. One nit:

  • NIT: the "last-wins"/precedence conventions aren't in question here since these are simple 1:1 curated map entries, but worth a beat noting the PR doesn't need a code comment update — the existing file-level doc comment on COMMAND_ALIAS_SUGGESTIONS already generically covers new entries ("Keys must be lowercase... each example must parse as a valid invocation").

Verified sound

  • Pre-fix "no suggestion" claim: confirmed by tracing getNearestCommandNames. For open-url (length 8 → threshold 3 per nearestMatchThreshold), commandNameDistance takes the startsWith fast path against open and returns abs(8-4) = 4 > 3, so it's filtered out; no prefix match either (no registered command name starts with open-url). Same for close-session (length 13, distance abs(13-5)=8 vs threshold 3). Both guesses got zero suggestion before this PR — the stated bug is real.
  • Curated-map conventions: both new entries ({ command: 'open', example: 'open <url>' }, { command: 'close', example: 'close' }) follow the existing shape and match the file's own drift-test contract.
  • Registry-drift tests cover the new entries: every curated alias suggestion target resolves to a registered command and every curated alias suggestion example parses as a valid invocation both iterate listCommandAliasSuggestionEntries(), so they automatically cover open-url/close-session without any test changes needed.
  • <url> placeholder tokenization: the drift test substitutes any token starting with < with com.example.app before calling parseArgs. For open <url>, that's not just syntactically convenient — open's real schema (positionalArgs: ['appOrUrl?', 'url?'], app: stringField('App name, bundle id, package, or URL.')) genuinely accepts a bare URL/string as its first positional, so the substitution exercises a real, valid invocation shape, not a coincidental parse.
  • Semantic correctness of the mappings: close with no args resolves to client.sessions.close(...) (per closeCommandDefinition in src/commands/management/app.ts), which is exactly what "close-session" should mean — not just closing an app.
  • Tests: all 22 tests in command-suggestions.test.ts pass (including the 2 new ones) in an isolated worktree build.
  • Gates: lint, typecheck, format:check, check:layering, fallow audit --base origin/main all pass clean.
  • CI: all green except one Smoke Tests job. Pulled its log — it failed in the Prepare iOS runner step with "Daemon request timed out" / xcodebuild runner-prep timeout, matching the known pre-existing Apple process-spawn-under-CPU-contention flake, not anything related to this PR's CLI-parser-only diff.

Verdict

APPROVE-worthy. Small, correct, well-tested; the one CI failure is an unrelated known flake.

@thymikee thymikee merged commit b3b0f59 into main Jul 10, 2026
20 of 22 checks passed
@thymikee thymikee deleted the fix/vocab-suggestions branch July 10, 2026 08:06
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-10 08:07 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant