Skip to content

[DOCS]: fill the gaps left after #222 and rename "threshold-gated recursion" #235

Description

@sromoam

Context

Rescoped. This issue originally proposed building one consolidated threshold page. #222 already does most of that with Getting-Started/thresholds-and-metrics.md, so building it here would mean writing a competing page around an open contributor PR.

This issue is now the remainder: what #222 does not cover, plus the rename. It depends on #222 landing first.

Already covered by #222

  • all four thresholds and what each gates
  • threshold precedence and the rule that a comparator's own threshold is ignored in normal evaluation
  • the five-category confusion matrix and the FP = FA + FD split
  • recall_with_fd, both formulas, and when to use each
  • a worked example end to end (numbers being corrected on that PR)

Remaining gaps

1. FN → FD raises recall; TP → FD lowers it

Two opposite moves, same vocabulary, and nothing distinguishes them anywhere in the docs:

move numerator denominator default recall
TP → FD (raise match_threshold) -1 -1 falls, or stays equal
FN → FD (what #224 changed for 1-vs-1 lists) unchanged -1 rises

This is worth calling out explicitly because it is genuinely easy to get backwards: a paragraph on #225 stated it inverted, and catching it needed an exhaustive check over tp0 ∈ 1..4, fn ∈ 0..3 (result: 0 cases where a TP→FD move raised recall, 6 equal, 34 fell). hungarian-matching.md now carries a corrected version of this; it belongs wherever recall_with_fd is explained.

2. The threshold=0.0 trap

A threshold of exactly 0.0 is satisfied by every score, so every pair becomes a TP and a wholly incorrect prediction reports perfect precision, recall, F1 and accuracy. #236 adds a runtime UserWarning whose message links to the threshold docs, so the docs need a section for it to land on.

It is a cliff rather than a slope, which is the part worth writing down: 0.01 classifies correctly, only 0.0 misbehaves.

3. Rename "Threshold-Gated Recursive Evaluation"

docs/docs/Advanced/threshold-gated-evaluation.md names the mechanism rather than the reader's question, and undersells itself: the page is about classification, not only recursion.

Candidate: "How Thresholds Affect Your Metrics". Alternatives: "Thresholds and Classification", or keep it as a mechanism reference and let #222's Getting-Started page be the explainer it links to.

Worth deciding deliberately, since the name becomes the link target from #236's warning message and from CHANGELOG entries.

4. Reduce duplication between the three pages

With #222 landed, hungarian-matching.md, threshold-gated-evaluation.md, and the new Getting-Started page will each partly explain thresholds. The Advanced pages should become mechanism references that link to the explainer rather than restating it.

Acceptance criteria

  • The FN→FD versus TP→FD contrast appears wherever recall_with_fd is explained, with the direction for each
  • The threshold=0.0 trap is documented, and feat(thresholds): warn when a threshold is exactly 0.0 #236's warning message links to it
  • threshold-gated-evaluation.md is renamed, with .nav.yml and inbound links updated
  • The Advanced pages link to the explainer instead of restating threshold semantics
  • mkdocs build warning count does not increase (currently 9, all pre-existing)

Notes

Depends on #222. Gaps 1 and 2 came out of the #225 review; both were symptoms of threshold behavior being spread thin across three pages with no single owner.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    Status
    In progress

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions