Implement AUROC, Rand index, V-measure and Kendall tau in numpy; drop the scikit-learn dependency

[sromoam]

Summary

Four statistical functions from scikit-learn and scipy can be implemented in numpy, which is already a core dependency. Doing so removes scikit-learn from the project entirely and reduces docsplit to pandas alone.

I verified all four numerically before filing, so this is a scoped refactor rather than a research task.

The four functions

function	source	used by
roc_auc_score	sklearn.metrics	AUROCMetric (confidence/metrics.py:102)
rand_score	sklearn.metrics	packet_evaluation_metrics.py:135
homogeneity_completeness_v_measure	sklearn.metrics	packet_evaluation_metrics.py:134
kendalltau	scipy.stats	packet_evaluation_metrics.py:167

That is the entire scikit-learn and scipy surface outside comparators/semantic.py (which uses scipy.spatial and stays behind the semantic extra).

All four are rank or contingency-table statistics with closed forms. AUROC is the normalized Mann-Whitney U statistic; Rand index and V-measure are contingency-table sums; Kendall tau is a concordant/discordant pair count.

Verified equivalence

numpy implementations compared against the reference implementations:

function	cases	max absolute difference
rand_score	1500 random clusterings	0.0 (bitwise identical)
homogeneity_completeness_v_measure	1500 random clusterings	7.2e-16
kendalltau	1500 random rankings	1.1e-16
roc_auc_score	4420 (random, heavy ties, all-equal scores, perfect and inverted separation)	1.1e-16

The nonzero differences are floating-point summation order, at most 3 ULPs. Rounded to 12 decimals, AUROC was identical in 1960/1960 cases, which is far beyond any precision these metrics are reported at.

Tie handling is the only subtle part in AUROC and Kendall tau, which is why the AUROC sweep specifically included score vectors with only three distinct values, and all-identical score vectors.

Payoff

	today	after
scikit-learn in the project	core dependency	removed
scipy	core (via sklearn) + semantic	semantic only
docsplit extra	pandas + scipy + scikit-learn	pandas only
install size	~217 MB with docsplit	~87 MB

It also removes the reason the scikit-learn floor was relaxed to >=1.7.2 (1.8.0 requires Python 3.11+, which conflicted with the 3.10 floor from #201). With sklearn gone, that constraint disappears rather than being worked around.

Approach

1. Add the four numpy implementations, probably in a small internal module (stickler/utils/statistics.py or similar) so both confidence/metrics.py and doc_split/ use one copy.
2. Keep the reference implementations as test oracles. Where scikit-learn/scipy are installed (the dev group), assert the numpy version matches to a tight tolerance across randomized inputs including ties and degenerate cases. That is how the equivalence above was established, and it makes a strong regression guard.
3. Remove scikit-learn from [project.dependencies] and from the docsplit extra.
4. pandas stays for docsplit: it is used for real input handling (_to_dataframe, CSV reading, groupby), not one function, so replacing it is a genuine refactor and not worth it here.

Notes and caveats

• The Kendall tau implementation I validated is O(n²) pairwise. That is fine for pages within a document group, but a merge-sort variant is needed if it ever sees large inputs. Worth deciding deliberately rather than by accident.
• Degenerate cases already handled by the current code must keep working: AUROC returns {"value": None} for a single-class input, and kendalltau returning NaN is coerced to 0 at packet_evaluation_metrics.py:170.
• This changes metric implementations, so it wants its own PR and its own review rather than riding along with a packaging change.

Sequencing

This follows the decision to make confidence metrics core (see #207): scikit-learn returns to [project.dependencies] now so calibration metrics work on a bare pip install stickler-eval. This issue then removes that dependency by implementing the maths directly, and the same work lets document-packet evaluation move into the core with only pandas behind an extra.

Related: #201 (dependency slimming), #207 (the PR that scoped the extras).