Context
Rescoped. This issue originally proposed building one consolidated threshold page. #222 already does most of that with Getting-Started/thresholds-and-metrics.md, so building it here would mean writing a competing page around an open contributor PR.
This issue is now the remainder: what #222 does not cover, plus the rename. It depends on #222 landing first.
Already covered by #222
- all four thresholds and what each gates
- threshold precedence and the rule that a comparator's own threshold is ignored in normal evaluation
- the five-category confusion matrix and the
FP = FA + FD split
recall_with_fd, both formulas, and when to use each
- a worked example end to end (numbers being corrected on that PR)
Remaining gaps
1. FN → FD raises recall; TP → FD lowers it
Two opposite moves, same vocabulary, and nothing distinguishes them anywhere in the docs:
| move |
numerator |
denominator |
default recall |
TP → FD (raise match_threshold) |
-1 |
-1 |
falls, or stays equal |
| FN → FD (what #224 changed for 1-vs-1 lists) |
unchanged |
-1 |
rises |
This is worth calling out explicitly because it is genuinely easy to get backwards: a paragraph on #225 stated it inverted, and catching it needed an exhaustive check over tp0 ∈ 1..4, fn ∈ 0..3 (result: 0 cases where a TP→FD move raised recall, 6 equal, 34 fell). hungarian-matching.md now carries a corrected version of this; it belongs wherever recall_with_fd is explained.
2. The threshold=0.0 trap
A threshold of exactly 0.0 is satisfied by every score, so every pair becomes a TP and a wholly incorrect prediction reports perfect precision, recall, F1 and accuracy. #236 adds a runtime UserWarning whose message links to the threshold docs, so the docs need a section for it to land on.
It is a cliff rather than a slope, which is the part worth writing down: 0.01 classifies correctly, only 0.0 misbehaves.
3. Rename "Threshold-Gated Recursive Evaluation"
docs/docs/Advanced/threshold-gated-evaluation.md names the mechanism rather than the reader's question, and undersells itself: the page is about classification, not only recursion.
Candidate: "How Thresholds Affect Your Metrics". Alternatives: "Thresholds and Classification", or keep it as a mechanism reference and let #222's Getting-Started page be the explainer it links to.
Worth deciding deliberately, since the name becomes the link target from #236's warning message and from CHANGELOG entries.
4. Reduce duplication between the three pages
With #222 landed, hungarian-matching.md, threshold-gated-evaluation.md, and the new Getting-Started page will each partly explain thresholds. The Advanced pages should become mechanism references that link to the explainer rather than restating it.
Acceptance criteria
Notes
Depends on #222. Gaps 1 and 2 came out of the #225 review; both were symptoms of threshold behavior being spread thin across three pages with no single owner.
Context
Rescoped. This issue originally proposed building one consolidated threshold page. #222 already does most of that with
Getting-Started/thresholds-and-metrics.md, so building it here would mean writing a competing page around an open contributor PR.This issue is now the remainder: what #222 does not cover, plus the rename. It depends on #222 landing first.
Already covered by #222
FP = FA + FDsplitrecall_with_fd, both formulas, and when to use eachRemaining gaps
1. FN → FD raises recall; TP → FD lowers it
Two opposite moves, same vocabulary, and nothing distinguishes them anywhere in the docs:
match_threshold)-1-1-1This is worth calling out explicitly because it is genuinely easy to get backwards: a paragraph on #225 stated it inverted, and catching it needed an exhaustive check over
tp0 ∈ 1..4,fn ∈ 0..3(result: 0 cases where a TP→FD move raised recall, 6 equal, 34 fell).hungarian-matching.mdnow carries a corrected version of this; it belongs whereverrecall_with_fdis explained.2. The
threshold=0.0trapA threshold of exactly
0.0is satisfied by every score, so every pair becomes a TP and a wholly incorrect prediction reports perfect precision, recall, F1 and accuracy. #236 adds a runtimeUserWarningwhose message links to the threshold docs, so the docs need a section for it to land on.It is a cliff rather than a slope, which is the part worth writing down:
0.01classifies correctly, only0.0misbehaves.3. Rename "Threshold-Gated Recursive Evaluation"
docs/docs/Advanced/threshold-gated-evaluation.mdnames the mechanism rather than the reader's question, and undersells itself: the page is about classification, not only recursion.Candidate: "How Thresholds Affect Your Metrics". Alternatives: "Thresholds and Classification", or keep it as a mechanism reference and let #222's Getting-Started page be the explainer it links to.
Worth deciding deliberately, since the name becomes the link target from #236's warning message and from CHANGELOG entries.
4. Reduce duplication between the three pages
With #222 landed,
hungarian-matching.md,threshold-gated-evaluation.md, and the new Getting-Started page will each partly explain thresholds. The Advanced pages should become mechanism references that link to the explainer rather than restating it.Acceptance criteria
recall_with_fdis explained, with the direction for eachthreshold=0.0trap is documented, and feat(thresholds): warn when a threshold is exactly 0.0 #236's warning message links to itthreshold-gated-evaluation.mdis renamed, with.nav.ymland inbound links updatedmkdocs buildwarning count does not increase (currently 9, all pre-existing)Notes
Depends on #222. Gaps 1 and 2 came out of the #225 review; both were symptoms of threshold behavior being spread thin across three pages with no single owner.