feat(p3+p1h2): tab registry, per-target CDP session pool, event store, reconnect-dedup — independent provider sessions - #11
Open
MrJ55 wants to merge 37 commits into
Open
Conversation
getAgentStatus truncated long answers to their final fragment and could get stuck reporting WORKING after the answer finished streaming. Fixes: - Join ALL [class*="prose"] blocks with containment dedup instead of taking only the last element - long answers are returned in full, without duplicated nested content. - Raise the response cap from 8000 to 30000 chars. - Completion detection: "Ask a follow-up" + prose wins over the working-text heuristic, so answers whose text contains words like "Working"/"Searching"/"Analyzing" no longer leave comet_poll stuck on WORKING. - Harden getAgentStatus against undefined CDP evaluate results (dead tab, navigate race, closed browser) so callers never crash on status.toUpperCase(). Verified end-to-end against live Perplexity threads (search mode) through the MCP gateway: comet_poll returns the complete, deduplicated answer.
…indings, Grok discovery runbook
…ntries for Perplexity and Grok
…ry types Port the discovery harness into the tool: src/core/discovery.ts (runDiscovery, verifyProvider, diffEntry, pickPrompt with per-run prompt rotation) and src/core/registry.ts (load/validate ProviderEntry JSON, packageRoot path resolution). Canonical ProviderEntry/ProviderControl types move into src/types/provider.ts so entries are data, not code.
…ase status, runbook, and Turn-02 checklist to reflect shipped discovery and all-5-provider verification
…ral fingerprint rebind A: registry confidence model (success +0.05 / failure -0.15 asymmetric, evict <0.3, trust >=0.7 hot-path resolve, learn-only-from-success); verifyProvider is now a learning loop persisting per-control confidence. B: src/core/fingerprint.ts — in-page FNV-1a structural fingerprint (ancestor chain + tag + children + identity attrs, per Bladebro refs.rs) with resolveWithRebind: known selector -> fingerprint rebind on miss -> escalate. Conditional controls (send buttons rendered only after typing) are SKIPPED by verify (no confidence penalty for idle absence, not a health failure) — a false-negative bug caught live (gemini sendButton drained 0.9->0.6). Entries backfilled: sendButton conditional=true, confidence seeded 0.9.
…y-related items done in Turn-02 checklist
… not a DOM node
Runtime.evaluate runs with returnByValue:true — a DOM element cannot cross
the boundary, so the fingerprint rebind silently returned null and every
re-render was recorded as a MISS (confidence drain). Caught by the live
DOM-mutation heal test: change #ask-input id -> verify -> rebind works,
restore -> normal OK. rebindSearchJs now returns {id,testid,aria,name,tag,cls}
and selectorFromElement rebuilds the selector from it.
… extraction testable (P1 Half 1) Refactors Perplexity behavior into the provider contract without changing user-visible behavior (P1): src/drivers/perplexity.ts implements ChatDriver — controls resolve through src/providers/entries/perplexity.json + ADR 0003 fingerprint rebind (replacing CometAI's hardcoded selector list); response extraction moves from the injected template-literal into src/providers/ extraction.ts (pure, Node-side, unit-testable). The in-page poll script only COLLECTS raw prose + signals; extraction happens in Node. Server wired via a compat layer (legacySendPrompt/legacyGetAgentStatus/ legacyStopAgent) so comet_ask/comet_poll/comet_stop keep identical external behavior — the migration path from comet_* to provider_*. Validation: 9 extraction unit tests pass (Bug hanzili#1 join+containment-dedupe, Bug hanzili#1 v2 whitespace order, Bug hanzili#2 keep-newest slice, filtering, steps, status ordering); live P1 gate smoke passed — ask -> poll -> completed with real answer extracted ("The capital of France is Paris."). Short-token prompts (OK/PONG) are intentionally filtered by the preserved len>5 rule (UI noise); real-length prompts validate correctly. Note: old src/comet-ai.ts remains for now (comet_* compat), to be retired after the migration path is fully proven.
…flects live smoke; build plan P1/P2 status updated
Grok driver mirroring the Perplexity driver: controls resolve through src/providers/entries/grok.json + ADR 0003 fingerprint rebind; extraction in src/providers/extraction.ts (testable). Grok-specific findings from live discovery encoded: composer contenteditable chat-input (execCommand typing), send button conditional (chat-submit renders only after text), NO stop button ever on the Fast model (stop() is a no-op false), streaming/completion via the "Working for Xs"->"Worked for Xs" timing line which Grok renders INSIDE the assistant-message (extractGrokResponse strips it). Entry gained the missing responseContainer control ([data-testid="assistant-message"]). Extraction seams (filters/clean) parameterized for provider-specific UI lists without changing Perplexity behavior. Validation: 8 new Grok extraction unit tests pass; 9 Perplexity tests still pass (regression-clean). Live P2 gate (ask/poll/stop against grok.com) PENDING — needs the grok tab open in Comet.
…across all providers (P2) Implements the handoff doc's markdown-preservation option (b): capture the response container's innerHTML and convert via turndown in Node (lower risk than an in-page HTML->MD walker). Provider-neutral by design — every driver collects innerHTML, src/providers/markdown.ts converts, with provider-specific pre-cleanup (perplexity citation badges stripped). - src/providers/markdown.ts: htmlToMarkdown(provider, html) — turndown with atx headings, fenced code, pre/code rule; preClean strips perplexity <sup> citation badges and <a class="citation">. - PollResult gains optional `markdown` field (non-breaking; text stays primary). - perplexity + grok drivers collect innerHTML in POLL_SCRIPT, populate markdown on completed. - Added turndown@^7.2.4 dependency + @types/turndown devDep. - npm note: this machine's global npm config is `omit=dev`, so typescript (devDep) was silently skipped on install — use `npm install --include=dev`. Validation: 5 new markdown unit tests + 9 perplexity + 8 grok all pass (22 total). Live P2 Grok gate PASSED: ask -> poll -> completed, text "Paris\nBerlin\nRome" + markdown "- Paris\n- Berlin\n- Rome" from the real DOM. execCommand typing into Grok's contenteditable confirmed live (P2 "correct CDP insertion/key behavior").
…2 complete (P2 finish) Fixture-driven tests (test/unit/fixture-driven.test.ts) run the extraction and markdown pipelines against the REAL sanitized DOM fixtures captured from live conversations, not synthetic strings — protects against extraction drifting from the actual DOM shape the drivers encounter. Fixtures rewritten from live answers with real structure: grok/completed.html (Paris/Berlin/Rome markdown bullet list from the live gate run), perplexity/completed.html (capital-of-France answer, citation <sup> stripped). Docs: Turn-02 P2 checklist fully checked (composer/send/stop/extraction/reset/ health, CDP insertion, markdown strategy, fixture-driven tests, capability evidence); build-plan P2 status updated. All 28 unit tests pass: 9 extraction + 8 grok extraction + 5 markdown + 6 fixture-driven.
… comet_* aliases (P2 wiring) The cleanest path to Grok-from-MCP: src/drivers/index.ts is a driver registry (getDriver/listDrivers) plus provider-neutral helpers (askAndWait, renderPoll, renderInProgress, normalizePrompt) implementing the ask->wait->respond loop ONCE over the ChatDriver contract. - New MCP tools: provider_ask/provider_poll/provider_stop with a `provider` param (perplexity | grok), dispatching via the registry; markdown appended to responses. - comet_ask/comet_poll/comet_stop reimplemented as thin Perplexity aliases over the same helpers (identical external behavior — the P1 comet_*->provider_* migration path fully realized). Dropped the old Perplexity-navigation/old-state cruft from comet_ask (driver.open() already targets the right tab). - Removed dead code: the legacy compat layer in drivers/perplexity.ts and src/comet-ai.ts (superseded by the drivers; extraction lives in src/providers/extraction.ts). Validation: tsc + build pass; all 28 unit tests pass; live dispatcher smoke PASSED (provider_ask path against grok.com: "Python\nJavaScript" + markdown bullets); server starts clean; comet-mcp list works. grok.json now carries responseContainer.
…runbook runtime usage; Turn-02 P2 gate passed Adds ADR 0004 recording the two P2 material decisions (innerHTML + turndown markdown extraction; provider dispatcher + provider_ask/poll/stop MCP tools with comet_* aliases). Build plan: P2 row marked DONE, new "Multi-provider runtime is wired" section. Runbook: new Runtime usage section documenting the 11 MCP tools + markdown. Turn-02: P2 gate marked PASSED with the live evidence.
Live provider_verify (perplexity) and provider_ask (grok) through pi bumped control confidence and captured structural fingerprints (grok composer/ modelPicker 0.9->0.95 with fingerprints; perplexity counters incremented). This is the self-healing learning loop persisting — committing the learned state so the entries stay current.
Test-drove the multi-provider backbone: asked Perplexity and Grok (both tabs loaded with 02-turn-02) to critique the P0-P8 plan against current state. Full critiques saved to docs/reference/06-provider-critiques/. Both converged: finish the P1 event store BEFORE full P3 (it's a P3 dependency — reconnect-dedup needs a durable extraction cursor, not just P4's); re-scope P6 to driver wiring only (discovery is done); minimal path = event store -> P3 -> P4 -> P5. Integrated decisions (Turn-02): - P3: audit dispatcher tab-addressing first (confirmed: Perplexity open() uses cometClient.connect() with no target = best-target fallback; both drivers hardcode cdpSessionId 'comet-client' — genuine singleton assumption); add per-tab poll backoff + circuit breaker. - P5 split: P5a wait_any in minimal release, P5b plan machinery deferred. - P4: approval binds to hash of exact envelope (single-use, expiring); redaction as first-class config; unknown-delivery reconciliation via read-only re-extraction; receipts append-only. - P1 gate honesty: replay-safety criterion cannot pass without event-store runtime — gate partially met. - Type contracts (Perplexity): schemaVersion; idempotencyKey/id split with attempt numbers; extraction evidence on receipt; taint propagation (trusted:false inherits); expired deadlineAt -> blocked. - P8: surface last successful high-confidence verification early. Also: test-drive surfaced two integration findings — MCP gateway truncates large provider_ask outputs (use the drivers to capture full text), and long asks can time out the gateway RPC (send via driver with longer wait).
…capture); perplexity entry learned confidence from the ask
…ateway result cap
Investigated the "long outputs truncate" report. Findings:
- The comet-mcp server returns FULL responses (verified: 208-char rivers answer, 5KB critique — nothing truncated server-side).
- The pi MCP gateway caps tool-result content at a small size (~a few hundred bytes / tokens); provider_ask/poll inline responses were cut there (text and markdown both truncated at the same byte budget).
- The gateway cap is not configurable from pi's mcp.json and not present in the pi agent package.
Fix: file-backed responses. provider_ask/provider_poll now write the complete
response (text + markdown) to C:\Dev\comet-mcp\responses\<provider>-<stamp>.md
and return a COMPACT result that fits the gateway budget: status + 200-char
preview + the full-content path. Full functionality is always available at the
path; no content is lost regardless of gateway limits.
Also hardened askAndWait completion detection (found during the same test):
- sawNewResponse now keys on content-hash change / length growth past the
pre-send snapshot, not mere presence of text (follow-up turns already have
prior text in the DOM).
- completed requires TWO identical poll readings (stability), so a single
'completed' that catches the DOM mid-render no longer returns a partial.
Verified live via pi: provider_ask {provider: grok, 10 rivers} returns
"Completed (208 chars). Preview: … Full response: <path>" with all 10 rivers
in the file (615 bytes, text + markdown).
…nked retrieval + retention Integrates the Perplexity + Grok critiques of commit 4b5dd56 (file-backed responses). Both providers validated the diagnosis (gateway caps tool results at ~500 bytes) and the workaround, but flagged it as path-coupled, non-atomic, unbounded, and concurrency-unsafe. This commit addresses their recommendations: - ID-based store: storeResponse() writes responses/<id>.md, returns an opaque responseId (NOT a filesystem path). Registry tracks {id, provider, path, contentHash, fullChars, markdownChars, createdAt, expiresAt}. - Structured compact result (JSON, not a formatted string): {status, responseId, preview, previewChars, fullChars, markdownChars, contentHash, expiresAt} — fits the gateway budget, machine-readable (Perplexity's structured-result recommendation). - provider_response MCP tool: chunked retrieval by responseId (offset/limit), with lazy re-index of existing files after restart (durability). Decouples clients from filesystem paths (Grok+Perplexity: path coupling is the main downside of the workaround). - Retention: enforceRetention() on startup + after each write — 24h TTL and max 100 responses, deleting expired/overflow files (both critiques: unbounded growth). - renderPoll/compactAskResult return structuredCompact; paths are internal. Verified live via pi: provider_ask grok -> structured compact JSON with responseId; provider_response {responseId} -> full text + markdown retrieved by ID. No truncation through the gateway.
…ate independently Audit (docs/reference/08-p3-dispatcher-tab-audit.md) confirmed the P2 dispatcher encoded a one-tab-per-provider singleton — worse, a one-tab-GLOBAL singleton (cometClient holds a single CDP connection; a second provider's open() silently killed the first's session, and comet_connect destroyed sibling tabs). P3 implementation: - src/cdp-pool.ts: per-target CDP session pool (Map<targetId, handle>), cap=5 (P0 measured), TabCapExceededError, per-tab health/reconnect, closeAll - src/tab-registry.ts: Map<tabId, TabSession> + providerKey→tabId addressing, last-tab protection (last tab reset not closed), scoped reset - drivers refactored: all CDP ops route through the pooled per-tab handle (evaluate/safeEvaluate/pressKey/navigate), open() via tabRegistry, real cdpSessionId (per-target wsUrl) replacing hardcoded 'comet-client' - askAndWait keeps the session across ask+poll (no per-poll reconnect) - per-tab poll backoff (2s→15s) + circuit breaker (5 failures → 30s cooldown) - dedup anchors populate on poll (lastContentHash, lastCompletedAt, ...) - MCP tools added: provider_open/list/close/health/override; provider_ask/poll/ stop accept tabId; comet_connect no longer closes all tabs (audit F5) - tests: 4 new unit tests (32 total), p3-live-gate.mjs (16 checks) Live gate PASSED 2026-08-07: perplexity+grok pooled concurrently (2/5), grok asked while perplexity session survived, scoped close isolated. Reconnect-dedup remains blocked on P1 event-store runtime (durable cursor).
…tency index, durable cursors
Both provider critiques (2026-08-07) converged: the P1 event-store runtime is the
silent dependency — P3's reconnect-dedup gate and P4's delivery receipts are
impossible without durable envelopes/receipts/cursors. Minimal scope as agreed
('days not weeks'): append-only JSONL + idempotency index + durable cursor
checkpoints.
Implementation (src/core/event-store.ts):
- append-only event log (ConversationEvent, monotonic seq, rebuilt watermark on
startup, corrupt lines skipped not fatal); data/ gitignored, COMET_DATA_DIR override
- idempotency index: replay/retry with the same idempotencyKey returns the PRIOR
outcome — the P1 gate's replay-safety criterion ('no duplicate send')
- durable cursor checkpoints per provider:tab (atomic temp+rename rewrite)
- response dedup substrate (hasResponseHash) for P3 reconnect-dedup
- receipts as an APPEND-ONLY stream (critique L37), each attempt its own row;
DeliveryReceipt extended with attempt + contentHash/providerMessageId/cursor
- fabric lifecycle helpers: envelope.created → send.queued/accepted/unknown →
response.received/deduplicated → delivery.receipt
Wiring (drivers/index.ts, index.ts):
- askAndWaitOn runs the full durable lifecycle; makeEnvelope builds native-ask
envelopes; replayOutcomeIfRecorded short-circuits recorded keys BEFORE any send
- provider_ask/comet_ask accept idempotencyKey; compact result carries
correlationId/idempotencyKey/replayed for safe client retries
- response.received checkpoints the extraction cursor per tab (P3 substrate)
Tests: 10 new unit tests (42 total) incl. the P1 gate replay-safety criterion;
p1-replay-smoke.mjs live PASSED 2026-08-07 — first ask sent, replay returned the
prior answer (replayed=true), event log shows exactly ONE send.queued + ONE
response.received for the correlation.
…response event Closes P3's last gate. The P1 Half 2 store (357f7ea) provided the substrate (durable cursor checkpoints + correlation-scoped hasResponseHash); this wires it into the reconnect path so a dropped/re-established session cannot re-emit a response event for content that was already recorded. - tab-registry: poolTab + reconnect() hydrate dedup anchors (extractionCursor, lastContentHash) from the durable store on (re)open; reconnect() forces a fresh pooled CDP session and falls back to a new tab if the old target is gone - drivers: updateSessionAnchors checkpoints the extraction cursor to the store on completed polls (durable even for poll-only flows); askAndWaitOn's completed branch dedups via hasResponseHash(correlationId, hash) → response.deduplicated instead of a second response.received; receipt carries 'reconnect-dedup' detail; AskOutcome.deduped surfaced in the compact result - MCP: provider_reconnect tool (re-establish session + re-hydrate anchors) Tests: 5 new unit tests (47 total); p3-reconnect-dedup.mjs live gate PASSED 12/12: ask → response.received + durable cursor checkpointed; reconnect re-hydrated anchors from the store; retry with same key returned the prior answer (replayed=true) with exactly ONE response.received + ONE send.queued in the log. P3 is now fully complete: registry, CDP pool, scoped reset, last-tab protection, 5 provider_* tools, per-tab backoff + circuit breaker, reconnect-dedup.
…rride accept any entry provider; agentBrowsingUrl excludes sibling provider tabs
Integration test through pi's MCP bridge surfaced two P3-era gaps (2026-08-07):
1. provider_open/close/reconnect/override gated on getDriver(), but the tab
registry + CDP pool are provider-neutral — only ask/poll/health need a
ChatDriver. With all 5 entries shipped (discovery), opening gemini/chatgpt/
claude failed 'Unknown provider'. Now: any provider with a registry entry is
addressable at the registry level (knownProvider = driver OR entry); health
falls back to a pointer to provider_verify for pre-driver providers.
2. Perplexity poll's agentBrowsingUrl used the legacy listTabsCategorized
classification ('any non-Perplexity page = agent browsing'), which mislabels
sibling provider tabs (grok/claude/...) as the agent's browsing target in the
multi-tab world. Poll now excludes tabs registered in the tab registry.
Verified live through pi (18 tools, fresh bridge):
- pool at 5/5 with all five real provider tabs (P0 cap re-measured, holds)
- perplexity+grok provider_health HEALTHY (all controls known-selector)
- gemini+chatgpt provider_verify HEALTHY; claude composer MISS on /recents →
navigated to /new → OK (empty-state finding: responseContainer conditional)
- provider_ask grok full lifecycle in event log (envelope→queued→accepted→
received→receipt), replay with same idempotencyKey → replayed=true, no dup
- provider_reconnect re-hydrates durable cursor
- agentBrowsingUrl bogus 'Browsing: claude.ai' gone after fix
- ADR 0003 learning loop observed live: chatgpt composer 0.9→0.95 learned;
claude composer 0.9→0.65 (2 fails) then success_count=1
…_cap_exceeded Found live through pi (2026-08-07): opening a 6th provider tab with the pool at 5/5 threw tab_cap_exceeded correctly, but the new-tab path creates the browser tab via cometClient.newTab() BEFORE the pool acquire, so the rejected open left an orphan unregistered claude.ai/new tab. The orphan then polluted Perplexity's agentBrowsingUrl heuristic (unregistered tab looked like 'agent browsing'), and cometClient.closeTab could not close it reliably (Target.closeTarget is bound to the current session's target; HTTP /json/close 404'd mid-close). Fix: registry.open() now checks sessionPool.size >= cap BEFORE openNewProviderTab — the N+1 open fails cleanly with zero browser side effects. Tests: new unit test (48 total) monkeypatches the pool to report full and asserts open() rejects with tab_cap_exceeded without creating anything.
…responseContainer, health delegation
Four bugs found during integration testing through pi (2026-08-07):
1. GROK EARLY-LATCH TRUNCATION (highest impact): askAndWait's completion
stability check returned after TWO identical readings, but Grok pauses
2-4s between phases ('Worked for Xs' marks the research trail done while
the ANSWER is still streaming) — two 2s-apart polls caught a pause and
latched 1592 chars of a 10205-char answer. Fix: duration-based stability
(completionStability helper + MIN_COMPLETION_STABILITY_MS = 8s): the hash
must hold the full wall-clock window, not just two readings. Live-verified:
a 706-char answer polled stable (no truncation).
2. DEFAULT TAB SELECTION: getProviderTab returned the FIRST registered tab,
so default (no-tabId) asks hit a stale tab and gateway-timed-out while a
fresher tab would answer. Fix: prefer most-recently-completed tab
(lastCompletedAt), tie-break by newest openedAt. Live-verified: default
ask targeted the completed grok tab, not the fresh one.
3. responseContainer EMPTY-STATE-CONDITIONAL: verify punished idle absence —
claude's responseContainer took 3 unfair fail_count hits (conf 0.9→0.45).
Fix: marked responseContainer conditional across all 5 entries (exists
only after a first response), reset claude's penalty history. Live-verified:
claude verify shows [OK] responseContainer (conditional).
4. provider_health DELEGATION for pre-driver providers: instead of erroring
'no ChatDriver (P6)', provider_health now runs entry-level verify (no
prompt). Live-verified: provider_health gemini → HEALTHY.
Tests: 50 → 54 (stability window x2, tab-selection x4). All fixes
live-verified through the pi bridge. Claude modelPicker remains genuinely
drifted (React-internal id) — needs provider_discover claude when convenient.
…phemeral-id inline
Two bugs surfaced while fixing the self-healing pipeline (2026-08-07):
1. DOWNGRADE GUARD: a low-confidence / partial discovery run could overwrite a
strictly-better existing entry (found live: a claude run ended streaming/low,
lost sendButton entirely, dropped conditional flags, flattened confidence to
0.3 — clobbering the committed HIGH entry). runDiscovery now compares the new
entry against the existing one before writing: refuses when the existing entry
has sendButton the new run lost, has more controls, or has higher confidence.
Result carries guarded {existingBetter, reason}; CLI + provider_discover
surface 'NOT overwritten (downgrade guard)'. Live-verified: claude run refused,
HIGH entry preserved.
2. VISIBLE-COMPOSER RANKING: the INVENTORY picked the first textarea in DOM
order, which on claude.ai is a hidden 0x0 accessibility textarea
(#static-composer-input) — discovery typed into it, Enter never submitted,
probe saw no response, run ended streaming/low. Inventory now ranks composers
by visibility + contenteditable (visible contenteditable/role=textbox first,
hidden last); the composer-selection chain prefers visible contenteditable
over the old textarea-first preference. Live-verified: discovery now picks
the real contenteditable div.
3. In-page isEphemeralId was referenced inside a CDP page string where imports
don't exist (ReferenceError at runtime) — inlined the regex into the send
button scan.
KNOWN LIMITATION (documented, not fixed): claude.ai's submission path on this
account/UI shows NO send button in the DOM and Enter does NOT submit via CDP
dispatchKeyEvent into the contenteditable — claude discovery cannot observe a
completed response yet. The downgrade guard protects the good entry from these
partial runs; the claude driver (P6) will need the real submit contract.
Tests: 57 → 65 (guard x5, composer ranking x3, fingerprint-fix x3).
…ection + marker stripping + guard extension The claude 'no submit path' limitation was WRONG — the send button exists and is a normal element. Root causes found via the user's observation (the send glyph span data-cds=Icon with the Anthropicons arrow char) + CDP probes: 1. The send button IS a <button aria-label='Send message'> (CDS group/btn class) that appears ~250ms after typing into the contenteditable composer, and clicking it submits. Discovery's own flow (focus → a/Delete dance → Input.insertText → scan → click) completes: state=completed, confidence=high, submit=button-click via [aria-label=Send message], all 3 fixtures captured. The earlier 'streaming/low' runs were a STALE BUILD — the composer-ranking + inlined-ephemeral-id fixes had not been compiled when they ran, and the in-page isEphemeralId ReferenceError crashed discovery after typing, before the scan click, silently falling back to enter-key (which does NOT submit on claude's contenteditable). 2. Discovery now marks conditional controls by observation: sendButton (absent at idle, present after typing) and responseContainer (exists only after a first response) get conditional:true + condition, so verify skips idle absence instead of punishing it. 3. Strip internal ranking markers (__visible/__editable) before persisting entries (they leaked into claude.json from the composer spread). 4. Downgrade guard extended: also refuse when the existing entry marks controls conditional that the new run would leave unmarked (verify would punish idle absence). Live-verified: guard refused a write with 'existing marks 2 controls conditional, new marks 0'. Claude discovery is now self-sufficient: run → HIGH entry with stable selectors + seeded fingerprints + correct conditional flags; verify HEALTHY 4/4 through the pi bridge. Tests: 65 → 66 (guard conditional-flag case).
…teway RPC window
Found live during the review drive (2026-08-07): the pi MCP gateway caps the RPC
round-trip (~150s). provider_ask used to block inside askAndWaitOn for the whole
window; the gateway abandoned the call (-32001) MID-ask, stranding the typed
prompt in the composer (observed: 1047-char review prompt in the composer,
never submitted, tab left dirty). The file-backed response store only helped
AFTER completion — it never solved the RPC-window problem.
Fix (async ask registry in drivers/index.ts):
- dispatchAsk(driver, session, prompt): runs the durable lifecycle up to
send.accepted (envelope.created → queued → snapshot → ask → accepted), then
registers a PendingAsk keyed by idempotencyKey and returns IMMEDIATELY with
{correlationId, idempotencyKey, status:'in_progress'} — no RPC blocking.
- advanceAsk(key): one poll step driven by provider_poll. Applies the same 8s
completion-stability window, per-tab backoff/circuit, response dedup, cursor
checkpoint, and delivery receipt; on completion stores the response
server-side (fetched via provider_response). Removes the pending entry on
completion/timeout.
- lastDispatchedFor(provider): provider_poll finds the pending ask to advance.
MCP wiring (index.ts):
- provider_ask now dispatches and returns the in_progress compact JSON
immediately. Replay guard (same idempotencyKey) returns the prior outcome.
- provider_poll advances the pending ask when one exists; falls back to a plain
poll otherwise.
- provider_response unchanged (chunked fetch of the stored response).
Verified live through the MCP server: provider_ask → immediate in_progress
(no -32001), provider_poll advances. Also surfaced: Grok tab on a project-chat
view doesn't submit reliably (needs plain grok.com chat), and the Grok account
hit its rate limit ('18 minutes before limit is gone') — environmental, not
code. Tests: 66 → 70 (async-ask dispatch/advance/stability).
Cherry-picked from main (88258f9) onto the PR branch, EXCLUDING the root README.md (upstream-facing; will be rewritten when the project is done). Build plan phase table corrected + new sections (tab registry/CDP pool P3, async ask dispatch, discovery hardening); ADR 0005 (async ask dispatch — gateway RPC survival); Turn-02 P1 gate PASSED + sequencing; reference README index; runbooks operational notes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Delivers P1 Half 2 (the event store) and P3 (concurrent tab registry + CDP session pool + reconnect-dedup) — the two phases both provider critiques (Perplexity + Grok, 2026-08-07) confirmed as the next dependencies. Together they make comet-mcp a true multi-provider conversation backbone: independent provider sessions, per-tab isolation, durable idempotent event log, replay safety, and reconnect-dedup.
What's included
P3 — tab registry + per-target CDP session pool (src/cdp-pool.ts, src/tab-registry.ts)
The audit (docs/reference/08-p3-dispatcher-tab-audit.md) confirmed the P2 dispatcher encoded a one-tab-global singleton:
cometClientholds a single CDP connection, so a second provider'sopen()silently killed the first's session — andcomet_connectclosed ALL tabs except one.Map<targetId, handle>), cap=5 (P0 measured safe limit),TabCapExceededErroron the N+1 open, per-tab health + reconnect,closeAll.Map<tabId, TabSession>with providerKey→tabId addressing; last-tab protection (the last tab of a provider is reset, never closed); scoped reset (never touches sibling provider tabs — ADR 0001 §Safeguards 2).cdpSessionId(per-target wsUrl) replaces the hardcoded'comet-client';askAndWaitkeeps the session across ask+poll (no per-poll reconnect).provider_ask/poll/stopaccepttabId;comet_connectno longer destroys tabs.P1 Half 2 — conversation event store (src/core/event-store.ts)
Minimal store as the critiques specified ("days not weeks"): append-only JSONL + idempotency index + durable cursor checkpoints.
ConversationEventlog with monotonicseq(watermark rebuilt on startup, corrupt lines skipped);data/gitignored,COMET_DATA_DIRoverridable.idempotencyKeyreturns the PRIOR outcome — the P1 gate's replay-safety criterion ("no duplicate send") is structural.hasResponseHashdedup substrate.DeliveryReceiptextended withattempt+contentHash/providerMessageId/cursorextraction evidence (critique L35/L37).askAndWaitOnruns the full durable lifecycle: envelope.created → send.queued → send.accepted/unknown → response.received/deduplicated → delivery.receipt;provider_ask/comet_askacceptidempotencyKey.Reconnect-dedup (P3's last gate)
"Unchanged content produces no new response event": sessions re-hydrate their dedup anchors (
extractionCursor,lastContentHash) from the durable store on (re)open; the completion path recordsresponse.deduplicatedinstead of a secondresponse.receivedwhen the correlation already holds that content hash.Notes for reviewers
tsc --noEmit+npm run buildpass; 47 unit tests pass.response.received+ durable cursor checkpointed;reconnect()re-hydrated anchors from the store; retry with the same key returned the prior answer (replayed=true) with exactly ONEresponse.received+ ONEsend.queuedin the log.Related