feat: provider discovery as a shipped tool - engine, CLI, MCP tools, JSON entries - #10
Open
MrJ55 wants to merge 10 commits into
Open
feat: provider discovery as a shipped tool - engine, CLI, MCP tools, JSON entries#10MrJ55 wants to merge 10 commits into
MrJ55 wants to merge 10 commits into
Conversation
getAgentStatus truncated long answers to their final fragment and could get stuck reporting WORKING after the answer finished streaming. Fixes: - Join ALL [class*="prose"] blocks with containment dedup instead of taking only the last element - long answers are returned in full, without duplicated nested content. - Raise the response cap from 8000 to 30000 chars. - Completion detection: "Ask a follow-up" + prose wins over the working-text heuristic, so answers whose text contains words like "Working"/"Searching"/"Analyzing" no longer leave comet_poll stuck on WORKING. - Harden getAgentStatus against undefined CDP evaluate results (dead tab, navigate race, closed browser) so callers never crash on status.toUpperCase(). Verified end-to-end against live Perplexity threads (search mode) through the MCP gateway: comet_poll returns the complete, deduplicated answer.
…indings, Grok discovery runbook
…ntries for Perplexity and Grok
…ry types Port the discovery harness into the tool: src/core/discovery.ts (runDiscovery, verifyProvider, diffEntry, pickPrompt with per-run prompt rotation) and src/core/registry.ts (load/validate ProviderEntry JSON, packageRoot path resolution). Canonical ProviderEntry/ProviderControl types move into src/types/provider.ts so entries are data, not code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes provider discovery a shipped, triggerable capability of comet-mcp instead of a test artifact, and adds self-healing provider controls (ADR 0003) so the harness is the rare repair, not the routine check.
Two capabilities:
Discovery as a shipped tool — when a provider changes its DOM and comet-mcp breaks, re-running discovery is repeatable and on-demand. Provider entries are data (JSON), not code, written directly by the discovery engine — so DOM-drift repair is
discover → commit new JSON, no code changes.Self-healing controls (ADR 0003) — inspired by Bladebro's Live Page Model (
src/knowledge.rs,src/page/refs.rs).provider_verifyis now a learning loop: per-control confidence (success +0.05, failure −0.15 asymmetric, evict <0.3, trust ≥0.7 hot-path resolve, learn-only-from-success). On a selector miss, a structural fingerprint rebind (in-page FNV-1a of ancestor chain + tag + children + identity attrs) absorbs React/Vue re-renders before anything escalates — only a genuine DOM change degrades and suggestsprovider_discover --diff.What's included
src/core/discovery.ts— the engine (ported from the discovery harness):runDiscovery(provider, {write, deadlineMs})— inventory → composer/send detection → one varied validation prompt → streaming/completed observation → writes entry JSON + fixturesverifyProvider(provider)— cheap selector check, sends NO prompt (the drift gate); now a confidence-learning loop with fingerprint rebind on missdiffEntry(provider, next)— selector/heuristic changes vs the committed entrypickPrompt— per-run prompt rotation (persisted state) so no provider repeats a probe promptsrc/core/registry.ts— loads + validatessrc/providers/entries/*.json; confidence model (recordSuccess/recordFailure/resolveWithConfidence/persistControlUpdate);packageRoot()walk-up so paths work from compileddist/.src/core/fingerprint.ts— in-page FNV-1a structural fingerprint (ancestor chain ≤10 + tag + first-3 children + type/name/data-testid, per Bladebroperception.rs) +resolveWithRebind: known selector → fingerprint rebind on miss → null escalates to discovery. Resolution order: known → fingerprint-rebind → heuristic → discovery.src/cli.ts+ bin dispatch — the primary on-demand trigger:MCP tools (
provider_discover,provider_verify) — opt-in operational tools per ADR 0001 (discovery is not a hot-path dependency).Provider entries as JSON —
src/providers/entries/{perplexity,grok,gemini,chatgpt,claude}.json, all HIGH confidence from live verification (2026-08-06/07, ACK/PONG/ALPHA/OK/BRAVO validations). Each control carries confidence/fingerprint fields; conditional controls (send buttons rendered only after typing) are flagged and skipped by idle verify. The hand-written.tsentries are retired.Docs — ADR 0003 (self-healing provider controls); build plan phase status (P0 done, P1/P2 types+discovery done, runtime pending); generalized discovery runbook; Turn-02 checklist updated.
Fixtures —
test/fixtures/{perplexity,gemini,chatgpt,claude}/state snapshots.Notes for reviewers
tsc --noEmit+npm run buildpass.comet-mcp verify --provider perplexity→ HEALTHY with confidence climbing (0.70→0.80 across verifies) and fingerprints captured;verify --provider gemini→ HEALTHY (conditional sendButton correctly skipped, unconditional controls at 1.00);verify --provider grokgracefully reports "no grok tab found" (exit 1); plaincomet-mcpstill starts the server.provider_discover.test/integration/discover.mjsharness removed (superseded by the engine).test/integration/out/are gitignored as runtime artifacts.Drift workflow this enables
Related
src/knowledge.rs,src/page/refs.rs,src/page/perception.rs