Chat-flow analysis + promise-detection baseline (docs + WIP scaffold)

[edwh]

This branch carries two related pieces of work on understanding Freegle member-to-member chats. Docs + a work-in-progress ML scaffold; no production code paths are touched.

---

Part 1 — Empirical chat-conversation flowcharts (Offer & Wanted)

State machines of User2User chats, derived from the live DB (V2 live API tunnel) over ~40k new conversations in one month, read against ~260 full conversations. Offer and Wanted behave differently, so each has its own flow.

Key finding: Completed only closes the post — it does not prove the replier in this chat got the item. A bare Interested → Completed, or a two-sided chat ending "sorry, gone to someone else", is not a collection. Counting only genuine handover-to-this-replier:

Outcome	OFFER	WANTED
✅ Collected / received	~43%	~24%
❌ Gone elsewhere / declined	~29%	~6%*
👻 Ghosted	~11%	~38%
⏸️ Stalled	~15%	~30%
↩️ Reneged	~2%	~0%

Files: plans/freegle-chat-flow.md, …-offer.{png,dot} + …-offer-workflow.json, …-wanted.{png,dot} + …-wanted-workflow.json (ai-flower workflow definitions, % on every transition).

---

Part 2 — Promise-detection: linear baseline scaffold (WIP) + session brief

Detecting the promised state (the point an exchange is committed to) from natural-language dialogue, per the working-notes spec. This is the pragmatic §5.2 baseline: a linear classifier over TF-IDF word + char n-grams on windowed, speaker-tagged, PII-normalised spans, as Laravel artisan commands in iznik-batch.

• plans/promise-detection-session-brief.md — full brief: spec interpretation, locked decisions (PHP/Rubix-ML, static CSV, address-as-signal not pure leakage, charset/utf8mb4), live-DB findings (~16.5% positive; role resolution), sample sizing, the dataset CSV contract, and the open alternatives (embedding classifier, DST/generative, zero-shot NLI) so another explorer can build a comparable model off the same span→label contract.
• Code (iznik-batch/app/Services/Promise, …/Console/Commands/Promise): PiiNormaliser, DatasetExtractor, CharNgramTokenizer, DatasetReader, KeywordBaseline, FeaturePipeline, Evaluator (P/R/F1, ROC/PR-AUC, threshold sweep, promise-timing offsets), promise:extract + promise:train commands, 6 unit-test files (53 cases), the CSV CONTRACT.md + synthetic fixture, and rubix/ml in composer.

Status — runs end-to-end on real data; WIP, not merge-ready. Defects fixed along the way: real probabilities (the Rubix Pipeline::proba() double-transformed — replaced with fit-once/transform-only transformers), top n-grams implemented, and a CSV data-corruption bug (fputcsv/fgetcsv default \-escaping silently merged ~30% of rows on read — spans are full of \u.. artefacts + quotes; fixed with RFC-4180 escape: ''). Clean result on a 768-room sample (2,516 train / 640 test, split by conversation, 13% positive): linear F1 0.206 / ROC-AUC 0.591 / PR-AUC 0.198 — near chance, and the keyword baseline beats it on F1 (0.333). (An earlier "F1 0.357 / ROC-AUC 0.751" was an artefact of the corrupted data.)

Success criterion is cost-based, and the baseline fails it. A false positive (wrongly claiming a promise) is stated as 20× worse than a false negative. That makes the target precision ≥ ~95% (minimise expected cost 20·FP + 1·FN; fire only at P>~0.95); below ~95% precision a model is net-negative versus never firing. On clean data the linear baseline tops out ~33% precision (and ROC-AUC 0.591, near chance), so under 20:1 it's not deployable — and it isn't even a convincing proof-of-signal. It sets the bar (precision ≥95%) the embedding/DST alternatives must clear. See the brief for the full reasoning and design implications (two-stage triage / human-in-loop).

Remaining issues: Rubix's dense vectoriser OOMs on char-n-grams at scale (capped for now); char-grams dominate the feature importances and add noise (a word-grams-only pass is the cheap next experiment). This part is pushed to enable parallel exploration, not to merge as-is.

---

Branch hygiene + test-suite fixes

This branch had accumulated unrelated commits from several sessions (monitor-fsm, mail-retry, freegle tooling) — all of which had already landed in master — plus one duplicate mail commit. It has been rebased onto current master, dropping every one of those; it now contains only the chat-flow docs (Part 1) and the promise scaffold (Part 2).

The committed unit tests also carried PHPUnit-validity bugs that the rebase surfaced, now fixed so the suite runs cleanly:

• assertEqual() (17 uses) — not a method in PHPUnit 11 → replaced with assertEquals().
• assertStringNotContainsString() called with a single argument (4 uses) → missing $result haystack supplied.
• the DatasetExtractor window/tolerance logic and a fixture/assertion drift (EvaluatorTest "random" ROC data, fixture row count, a KeywordBaseline acceptance phrase).
• the \r\n\t-collapse test used digit+word input (Line 1 … Line 2) that the normaliser's address regex redacts to <ADDRESS>; switched to digit-free words.

---

Notes

• PII-free: raw conversation samples (names, addresses, phones) were deliberately not persisted; only aggregate analysis, the synthetic fixture, and the PII-normalising pipeline are committed.
• Independent of other in-flight work; branched off master.

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for golden-caramel-d2c3a7 canceled.

Name	Link
🔨 Latest commit	ed6f31f
🔍 Latest deploy log	https://app.netlify.com/projects/golden-caramel-d2c3a7/deploys/6a53cf3831105b000834ee27