T8: standing testing modalities — localhost playground + LARP-mode agent user testing

[mzargham]

Owner requirement (2026-08-02): two testing modalities woven into EVERY T8 phase, early and often, to catch drift from design intent before human user testing.

1. Localhost playground — docs/playground.md: always-current, hand-verified quickstart to run the current increment locally and play with it. Extended at each phase (P1: cds-serve + cds-oracle + MCP config over a cds init scratch root).
2. LARP-mode agent user testing — docs/testing/larp-user-testing.md: LLM agents role-play concrete personas aggressively against public surfaces only; explicit doctrine-drift verdicts; findings filed with severity + repro. Cadence table covers P1–P6 (incl. P4 no-fabrication baiting and P5/P6 hostile-user probing for a code surface).

Acceptance per phase: playground page updated + verified; a LARP pass run with findings filed. Closes only when feat/t8-concept-definition-app merges to main.

[mzargham]

First LARP session — P1 facilitator + oracle (2026-08-02)

Persona: "Priya Raman", systems engineer, municipal drone-delivery pilot; HTTP surfaces only, zero docs. Built a full front end (synthesis, mission, problem, 4 stakeholders, 4 goals, 5 linked needs, tension, parked item, queue item), made deliberate mistakes, probed boundaries, compiled a presentable brief.

Findings

• [F-1] major | facilitator OpenAPI — cds_new/cds_edit schemas omit kind-specific link fields (for_stakeholder, serves_goal, addresses, segment/interest/influence); validated server-side but undiscoverable from the contract. Contract lies by omission.
• [F-2] major | cds_new — duplicate slug silently upserts (overwrote label/description, dropped links, 200). Expected 409 / "exists — use cds_edit".
• [F-3] major | cds_waive — accepts unknown rule names and rules whose findings would be T1-class; "T1 refused" only manifests against live findings. Dead waivers enter an append-only ledger.
• [F-4] major | staging UX — no way to retract a mistaken candidate (junk goal now permanent in the brief). Design question: a retraction affordance touches the K1 whitelist (CLI has rm; the whitelist deliberately (?) doesn't) — owner call needed.
• [F-5] major | session isolation — a prior session's record (the operator smoke test) surfaced in Priya's verify. Root cause: P1 binds a shared project root; per-session scratch staging is P2 scope (ADR-5) — this finding validates that P2 priority.
• [F-6] minor — cds_list {"kind":"synthesis"} → raw KeyError detail. [F-7] minor — commit 403 message is roadmap jargon, no escalation path. [F-8] minor — oracle: vacuous turtle "conforms"; parse errors leak rdflib byte noise. [F-9] minor — /rules has names only (no tier/description → can't tell waivable from blocking). [F-10] minor — extras posture inconsistent (cds_new accepts junk fields; typoed link fields silently dropped). [F-11/12/13] papercuts — cds_explain speaks CLI syntax + null on unknown names; bare-URI/null returns; nothing teaches synthesis-first except a 422.

Doctrine verdicts

candidates-only mostly held (eroded by F-2/F-4/F-5) · commit refused firmly, explained poorly · verification advisory + actionable — the standout · escalation path partly discoverable · errors teach vocabulary, not order.

Top fixes before human user testing

1. Kind-specific fields into the OpenAPI schemas (F-1)
2. cds_new refuses existing slugs; retraction affordance (owner decision re: whitelist) (F-2, F-4)
3. cds_waive validates rule names + refuses T1-class at waive time (F-3)
4. End-user-grade refusal/explain text; fix cds_list synthesis KeyError (F-6, F-7, F-11)
5. Per-session clean staging (P2) + /rules enriched with tier/description (F-5, F-9)

Full protocol: docs/testing/larp-user-testing.md. Refs: #50 (facilitator), #48 (oracle), #46 (whitelist design question in F-4).

[mzargham]

LARP pass #2 — regression + ADR-9/positions (2026-08-02)

Same persona (Priya), fresh services at d9ab7f4-adjacent build. All seven P1 findings resolved: F-2 FIXED (teaching refusal on duplicate slug), F-4 FIXED (cds_discard + referrer report), F-3 FIXED (unknown rule → known-rules list; T1-class → 403), F-6 FIXED, F-7 FIXED (role/safety/next-step), F-1 FIXED (all link fields in the schema), F-11 PARTIAL (suggestions yes, but generic list, no fuzzy match). New edit flow works incl. 422 on absent.

New-feature verdicts: divergence exactly right — conforms:true, T3, "all retained; divergence is valid"; brief's Convergence & divergence section judged director-ready; scratch mode "usable without chafing"; errors "consistently teach".

New findings:

• G-2/G-7 (moderate) — supersedes accepted bare slugs into broken IRIs, and superseded records stayed in the live brief. FIXED post-session in d9ab7f4: one reference rule (bare slug = same-kind) + eager supersededBy materialization; superseded records now leave the current view immediately.
• G-1 (moderate) — cds_explain doesn't know position or the lifecycle verbs; the newest concepts are the least explained.
• G-5 (moderate) — compile has no per-synthesis scoping; records from a second synthesis bleed into the brief.
• G-6 (minor) — cds_show doesn't surface the retracted marker/reason; append-only is a claim the HTTP user takes on faith.
• G-8 (minor) — oracle /rules is names-only and lists fewer rules than the waiver error; no tier/description.
• Nits: 422-where-409/404-expected; OpenAPI fields lack per-field descriptions.

Remaining fixes queue: G-1, G-5, G-6, G-8 + status-code/description polish — candidates for P1.6 or fold into P2. Refs: #52, #50, #48.