Gamut clamping in apply_gamut_matrix masks codec gamut-clipping regressions

[lilith]

Summary

apply_gamut_matrix (zensim/src/color.rs:73-85) clamps gamut-converted RGB to [0, 1] after multiplying by the BT.2020→sRGB / P3→sRGB matrix. The comment at line 71 calls this "acceptable for SDR" because a real sRGB display can't show out-of-gamut colors anyway.

This is correct for the post-display-clamp interpretation of the metric. But it has a non-obvious failure mode: it masks codec gamut-clipping regressions.

Failure case

A codec (or pre-encoding pipeline) might clip out-of-gamut colors to sRGB gamut as a destructive operation. Currently zensim's gamut conversion clips the reference identically before XYB, so the destructive clip in the encoded output produces zero metric difference — even though information was lost.

Concretely: suppose the source has a BT.2020-saturated red region. A faithful encoder preserves the wide-gamut signal; a lossy encoder clips to sRGB before encoding. If the user feeds both decoded outputs into zensim with ColorPrimaries::Bt2020, both go through apply_gamut_matrix and clip to (1, 0, 0). zensim returns 100.0 — same result for the lossless and lossy encoder. The metric cannot distinguish them.

This is exactly what bt2020_saturated_colors_differ_from_srgb and p3_green_outside_srgb_gamut_differs in tests/icc_coverage.rs are measuring (and failing on, because the design as currently shipped doesn't promise to detect this).

Adjacent test that does work

bt2020_vs_srgb_interpretation_differs uses non-saturated colors that are in-gamut on both sides, and passes. So the metric correctly distinguishes BT.2020 vs sRGB metadata for any color that's within both gamuts; only the saturation boundary is invisible.

Options

A. Keep clamping, document explicitly

Add a clear note in the Zensim / apply_gamut_matrix rustdoc that the metric measures post-display-clamp perception, and that out-of-gamut clipping during encoding is below the metric's measurement floor for SDR output. Update the failing tests in icc_coverage.rs to use in-gamut colors. Pros: matches existing behavior; low churn. Cons: leaves a real codec regression class undetected.

B. Drop the clamp; let out-of-gamut linear flow into XYB

Remove .clamp(0.0, 1.0) in apply_gamut_matrix. The downstream opsin matrix can handle negative / >1 linear values; the existing .max(0.0) clamp on mixed (color.rs:244-250) on the post-opsin sum still keeps the cube-root domain valid. Codec clip vs. preserve becomes a measurable XYB difference. Pros: detects gamut-clipping regressions. Cons: produces non-zero metric scores for content that's visually identical on an sRGB display. Score values shift for any wide-gamut content with saturated colors — a metric behavior change worth a minor version bump and a Changed changelog note.

C. Two-pass scoring

Compute the metric twice: once with the current clamping (post-display fidelity), once without (gamut-loss detection). Combine or expose both. Pros: best of both worlds. Cons: higher compute cost; new public API.

D. Configurable

Zensim gains a constructor option / profile knob to choose clamp vs. no-clamp. Pros: explicit. Cons: another knob; users have to know to set it.

Recommendation

I'd start with B in a 0.3.x → 0.4.0 release with a Changed note, and file follow-up tickets to verify the new behavior on real wide-gamut codec corpus pairs. The clipping protection is more valuable than the post-display-clamp simplification, and downstream consumers tracking exact scores already have to rebase on the cbrt_midp swap.

A is acceptable as an interim if (B) needs more validation work.

C/D feel like over-engineering for a binary decision the metric should make once.

Tasks

• Decide on option (A/B/C/D)
• Implement
• Update tests/icc_coverage.rs (bt2020_saturated_colors_differ_from_srgb, p3_green_outside_srgb_gamut_differs) — either to assert the new behavior, or to use in-gamut colors per option A
• Verify against real wide-gamut codec output if possible (mozjpeg / jpegli / zenavif at saturated content)
• CHANGELOG entry under Changed if option B/C/D

Related

• The two icc_coverage tests have been failing in CI since they were added (commit 62aa350). They've been the load-bearing red light pointing at this design question.
• Companion to: precise score targets noted in docs/scale-invariance.md — wide-gamut sources are a different drift axis but adjacent.

[lilith]

Implemented as the least-breaking variant of option D in 16958ab — an opt-in GamutMapping mode rather than a global behavior flip:

• New GamutMapping enum: Clip (default — post-display-clamp semantics, byte-identical to every release to date) and Preserve (opt-in — out-of-gamut values flow into XYB, making codec gamut clipping detectable). Exposed as ImageSource::gamut_mapping() (provided method, additive) + StridedBytes::with_gamut_mapping(GamutMapping::Preserve).
• Implementation found a second masking layer beyond apply_gamut_matrix: the SIMD linear→XYB kernels clamp their input to [0,1] (.max(zero).min(one) in every arch variant), so removing the gamut-matrix clamp alone re-masks the difference downstream. Preserve therefore routes gamut-converted rows through a new unclamped scalar converter (per-pixel math mirrors the clamped kernels' scalar remainder exactly, bit-parity-gated; the opsin max(0) on the post-mix sum keeps the cube-root domain valid, as the issue predicted). Also: reflect_pad_to_min's OwnedImage and SubsetView forward the flag (the reflect-pad initially dropped it — caught by the sub-64px test path).
• Tests: the two formerly-#[ignore]d icc_coverage tests (bt2020_saturated_colors_differ_from_srgb, p3_green_outside_srgb_gamut_differs) now run and pass under Preserve; new gamut_clip_regression_masked_by_clip_detected_by_preserve reproduces the issue's failure case end-to-end (faithful BT.2020 red vs sRGB-gamut-clipped-then-encoded: Clip scores ≈100 = masked, Preserve detects it, >1 point separation); in-gamut content agrees across modes to 0.045 score points (cbrt-precision only — Preserve uses cbrtf_fast ~20-bit scalar vs the SIMD cbrt_midp ~15-bit); a bit-parity gate pins the unclamped mirror to the clamped scalar remainder so it can't drift.

Semver: additive only — new non_exhaustive enum, new provided trait method, new builder; the default is bit-unchanged (preserve_oog=false short-circuits to the untouched SIMD kernel and the Clip arm's arithmetic is identical). Trailing-component territory for the unpublished 0.3.0.

Not done from the issue's task list: verification against real wide-gamut codec output (mozjpeg / jpegli / zenavif saturated content) — the synthetic end-to-end test models the clip-before-encode case, but a real-codec corpus pass is still worth doing before recommending Preserve in CI pipelines. Option B (flipping the default) remains available later as a 0.4.0 Changed if Preserve proves out; the enum makes that a one-line default swap.

Leaving open for the real-codec verification task + the default-flip decision.