Skip to content

feat(task): per-artifact output equivalence predicate (Closes #246) - #264

Open
noahgift wants to merge 2 commits into
mainfrom
feat/output-equivalence
Open

feat(task): per-artifact output equivalence predicate (Closes #246)#264
noahgift wants to merge 2 commits into
mainfrom
feat/output-equivalence

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

Closes #246.

Byte-identity is exactly right for source and object files — recompiling to identical bytes correctly does not relink. It is wrong for producers that cannot reach byte-identity, and wrong in the dangerous direction: such an artifact is not "uncached", it is content-addressed with the wrong key.

The case, reproduced then fixed

An ASR pass writes narration.srt; a person corrects it. Verified end to end through the binary, differentially — "the artifact survived" proves nothing unless the default destroys it:

2nd apply narration.srt
no declaration 1 converged ASR draft — human work destroyed
narration.srt: external 1 unchanged Human corrected transcript

Both are tests, not just a transcript — the "destroyed" case is kept as the precondition that makes the fix meaningful.

--force still regenerates

The issue names force as the escape. I built a script-level guard that blocked it first, then removed it: it contradicted the spec, and it tripped the repo's own C3-FORCE-DISTINGUISHABLE invariant, which was right to complain. A guard that made --force mean something different here than everywhere else in forjar would be worse than the problem.

Three predicates, cheapest first, as the issue asks

  • none — excluded from the staleness predicate; a missing artifact still is
  • external — a modification is an improvement, not staleness
  • command — compare a declared normaliser's stdout instead of the bytes, covering structural equivalence without teaching forjar any media formats

A failing normaliser is an error, never a silent fallback to bytes — that would reintroduce exactly the spurious staleness the author declared it to avoid.

Absent declarations mean bytes, so every existing project hashes exactly as before (FALSIFY-OE-002 pins it).

Two things the repo's own guards caught

  1. every_resource_field_is_declarable_in_yaml failed — a field on Resource that isn't in RESOURCE_FIELDS makes any forjar.yaml using it fail with unknown field. The feature would have been unreachable from config while every unit test passed.
  2. The declaration is now part of the hash. Flipping an artifact from external back to bytes must not look identical to never having declared it, or it stays keyed by the old predicate forever.

Also: a pre-existing flake, fixed in passing

cli::structured_log::tests::test_set_and_get_level asserts on a process-global log level while two sibling tests set it concurrently. It failed once in 13k — the worst frequency: rare enough to look like noise, common enough to erode trust in every other result. Serialised with a mutex that ignores poisoning.

Verified

13013 lib tests
8 equivalence tests (6 unit + 2 end-to-end through the binary)
clippy --all-features --all-targets -D warnings clean
io_tracking.rs 99.1 (A+) — above its own baseline, after splitting the new
                            code into its own module rather than inlining it

🤖 Generated with Claude Code

noahgift and others added 2 commits August 17, 2026 16:16
Byte-identity is exactly right for source and object files — recompiling to
identical bytes correctly does not relink. It is wrong for producers that
CANNOT reach byte-identity, and wrong in the dangerous direction: such an
artifact is not "uncached", it is content-addressed with the wrong key.

THE CASE, REPRODUCED THEN FIXED

An ASR pass writes narration.srt; a person corrects it. Verified end to end
through the binary, DIFFERENTIALLY — because "the artifact survived" proves
nothing unless the default destroys it:

  without the declaration   2nd apply: 1 converged  -> "ASR draft"
                                                       human work DESTROYED
  narration.srt: external   2nd apply: 1 unchanged  -> "Human corrected
                                                        transcript"

`--force` still regenerates. The issue names force as the escape, and a guard
that blocked it would make the flag mean something different here than
everywhere else in forjar. I built such a guard first and removed it: it
contradicted the spec, and it tripped the C3-FORCE-DISTINGUISHABLE invariant,
which was right to complain.

THREE PREDICATES, cheapest first, per the issue

  none       excluded from the staleness predicate; a MISSING artifact still is
  external   a modification is an improvement, not staleness
  command    compare a declared normaliser's stdout instead of the bytes,
             covering structural equivalence without teaching forjar any media
             formats. A failing normaliser is an ERROR, never a silent fallback
             to bytes — that would reintroduce the spurious staleness the
             author declared it to avoid, and silently.

Absent declarations mean `bytes`, so every existing project hashes exactly as
before. FALSIFY-OE-002 pins that.

TWO THINGS THE REPO'S OWN GUARDS CAUGHT

  1. `every_resource_field_is_declarable_in_yaml` failed: a field on `Resource`
     that is not in RESOURCE_FIELDS makes any forjar.yaml using it fail
     validation with `unknown field`. The feature would have been unreachable
     from config while every unit test passed.
  2. The declaration is now part of the hash. Flipping an artifact from
     `external` back to `bytes` must not look identical to never having
     declared it, or it stays keyed by the old predicate forever.

The equivalence hashing lives in its own module rather than in io_tracking:
inlining it dropped that file A+ -> A, and "how an artifact is compared" is a
different concern from "which files are tracked". io_tracking is now 99.1 (A+),
above its own baseline.

ALSO: fixed a pre-existing flake found in passing

`cli::structured_log::tests::test_set_and_get_level` asserts on a
PROCESS-GLOBAL log level while two sibling tests set it concurrently. Failed
once in 13k, which is the worst frequency — rare enough to look like noise,
common enough to erode trust in every other result. Serialised with a mutex
that ignores poisoning, so a panic in one cannot cascade.

Verified: 13013 lib tests, 8 equivalence tests (6 unit + 2 end-to-end),
clippy clean, contracts/output-equivalence-v1.yaml validates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@noahgift
noahgift enabled auto-merge (squash) August 19, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant