Skip to content

Remove duplicate chunk discovery paths - #75

Open
asinghvi17 wants to merge 2 commits into
claude/e1-graph-backed-lazyfrom
claude/e2-delete-legacy-discovery
Open

Remove duplicate chunk discovery paths#75
asinghvi17 wants to merge 2 commits into
claude/e1-graph-backed-lazyfrom
claude/e2-delete-legacy-discovery

Conversation

@asinghvi17

Copy link
Copy Markdown
Member

Task E2, the whole of Phase 4 of regrid-notes/2026-08-21-regridding-simplification-plan.md. Fifth in the stack, on top of #74 (which is on #72, on #71, on #70). Record: regrid-notes/2026-08-23-e2-delete-legacy-discovery.md.

Phase 4's gate: one query implementation defines graph edges, and neither the executor nor the interface translates that relation back through a compatibility tree. Both halves are tests, in test_chunkgraph.jl, testset one query implementation defines every edge.

Deleted, each after a repo-wide audit

  • connectedchunks / connectedchunks! — all four methods. Not exported, not public, and after E1 no caller outside test_lazy.jl. The two srcindex in-place forms were the card's "compatibility source-index state": a prebuilt source index carried around outside any relation and re-queried.
  • The whole chunktree bridge — the export, the declaration, the chunktree-collecting chunkextents fallback with _collectextents!, RasterFlatTree and its fifteen trait methods, and chunktree(::RasterGrid).

The chunktree verdict: remove

Nothing implemented it but the bridge itself, a TileCells forwarder, and three test toys. DGGSpace never had one — it specializes chunkextents/chunkindex/candidatechunks! directly. DGG's private _cachedchunktree is an unrelated name (it caches a HierarchicalGridCursor's caps) and is untouched, as is the local variable named chunktree in crosssystem/regrid.jl. ext/ has no reference and docs/ does not build GlobalRegridding.

So the only thing that still needed chunktree was chunktree: a space packed its caps into a flat tree so the generic chunkextents could walk it and collect the same caps straight back out. chunkextents is now a required hook with no fallback, asserted as !hasmethod(chunkextents, Tuple{RegridSpace}). RasterFlatTree went with it, closing the note at rastergrid.jl:969-971.

Kept, because the audit says they are not dead

  • chunkextents — many real consumers that want the caps as values: spacestamp, _builddependencies (both sides), subspace_dependencies, the generic chunkindex, and three production scripts.
  • chunkextent — after connectedchunks went it had zero callers and zero specializations, but it is public, documented, and asserted by the qualified-contract testset. The card says keep a cheap one, and the generic chunkextents(space)[chunk] was not cheap, so RasterGrid now specializes it at O(1) in the chunk count.
  • The gates harness's archived :latjoin arm — there is no latitude join left in src/ at all (G2 deleted it). What remains carries an explicit sunset condition tied to G2's waiver, which has not been retired; deleting it would have destroyed both the waiver's audit trail and half of this PR's own before/after evidence.
  • The generic candidatechunks! STI fallback — unreached in-repo after the above, but it is one method of the one query function rather than a second relation, and it is what makes the documented chunkindex promise true.
  • Nothing to do on "post-plan graph builder" (G4 removed it; api.jl documents that chunk_dependency_graph has no plan method) or "redundant cap vectors" (E1 moved the graph's two onto the relation on purpose; in production they alias the two DGGSpaces' own arrays).

Gate: the relation is identical

benchmark/chunk_graph_gates.jl at the branch point and on this branch. 13 cases including the production copdem90-igeo7-l12 pair from a local tile list, both arms, 26 ndjson rows each, -t 8 --gcthreads=4, 5 samples.

Every relation field identical on all 26 rowsedges, demanded_pairs, demand_missing, oracle_pairs, oracle_missing, only_here, missing_here, destination_chunks, source_chunks, radius, identity_bytes, graph_allocated_bytes, graph_summarysize_bytes. graph_allocated_bytes byte-identical on every row is the direct evidence the bridge was never on the build path. Both runs print verdict: PASS on 9 oracle-checked case(s); 4 case(s) unchecked — a real geometric verdict, not NOT CHECKED. Production :indexed +1.8 %, inside the band G3/G4/E1 all recorded on unchanged code.

Values and residency: byte-identical

before (0e3fb70) after
LazyStats loads 328, hits 3654, peak 2 073 600 B identical
source readblock! calls 400 400
repeated read / slices / retiled plan true true
max abs(eager − lazy) 2.44249e-15 2.44249e-15
live: plan / array 82 072 B / 43 376 B 82 072 B / 43 376 B

Suites

suite before after delta
lib/GlobalRegridding/test 3 999 / 1 broken / 0 fail 4 021 / 1 / 0 +22 pass
crosssystem/regrid.jl 236 236 none
crosssystem/regrid_acceptance.jl 22 22 none
scripts/copdem_policy.jl 89 89 none
scripts/copdem_source_mode.jl 7 7 none
CopernicusDEM/runtests.jl 16 258 / 3 broken / 0 fail 16 258 / 3 / 0 none

No count dropped. Every assertion that named a deleted function was rewritten onto the relation rather than removed. The +22 is itemized assertion by assertion in the record: 19 in the new Phase 4 gate testset, 2 in discovery, 1 in the wave-failure testset.

🤖 Generated with Claude Code

https://claude.ai/code/session_019DuGKTmvs5B4EynwZddKMg

asinghvi17 and others added 2 commits August 23, 2026 22:46
Phase 4's gate: one query implementation defines graph edges, and neither
the executor nor the interface translates that relation back through a
compatibility tree. Both halves are now tests.

Deleted, each after a repo-wide audit recorded in the note:

  - `connectedchunks` and `connectedchunks!` (all four methods). Not
    exported, not `public`, and after E1 with no caller outside
    `test_lazy.jl`. The two `srcindex` in-place forms were the card's
    "compatibility source-index state": a prebuilt source index carried
    around outside any relation and re-queried, which is what E1 removed
    from `LazyRegridArray`.

  - The whole `chunktree` bridge: the export, the declaration, the
    `chunktree`-collecting `chunkextents` fallback with
    `_collectextents!`, `RasterFlatTree` and its fifteen trait methods,
    and `chunktree(::RasterGrid)`. Nothing implemented it but the bridge
    itself and three test toys; `DGGSpace` never had one, and DGG's
    private `_cachedchunktree` is an unrelated name and is untouched.
    `chunkextents` is now a required hook with no fallback, so a space
    no longer has two ways to answer one question.

Kept, because the audit says they are not dead: `chunkextents`, which
has many real consumers that want the caps as values rather than as a
query; `chunkextent`, which is `public`, documented and part of the C1
contract, and which `RasterGrid` now specializes at O(1) in the chunk
count rather than materializing the whole vector; the gates harness's
archived `:latjoin` arm, whose sunset condition (G2's waiver retired)
has not fired; and the generic `candidatechunks!` fallback, which is one
method of the one query rather than a second relation.

`TileCells` forwards `chunkextents`/`chunkextent`/`chunkindex` where it
used to forward `chunktree`, which also makes it answer over a
`DGGSpace`.

The oracle tests now read every claim off `sourcesof(dependencies(plan),
d)`, and the shared oracles in `graphoracles.jl` are reused rather than
re-spelled. `G4ProbeSpace`'s counter moved up one level from `chunktree`
to `chunkextents` — the same funnel, since `chunktree` had no caller of
its own.

Evidence, all in regrid-notes/2026-08-23-e2-delete-legacy-discovery.md:

  - `benchmark/chunk_graph_gates.jl` before/after, 13 cases including the
    production pair from a local tile list, 26 rows each: every relation
    field identical on every row, `graph_allocated_bytes` byte-identical
    on every row, and both runs "PASS on 9 oracle-checked case(s)".
  - Residency and values byte-identical: LazyStats peak 2073600 B, 400
    readblock! calls, repeated reads/slices/retiled plan stable, and the
    pre-existing max |eager - lazy| = 2.44249e-15 unchanged.
  - Suites: GlobalRegridding 3999 -> 4021 pass (+22, itemized in the
    note; nothing was deleted along with the code it covered), 1 broken,
    0 fail; crosssystem regrid 236, acceptance 22, copdem_policy 89,
    copdem_source_mode 7 and CopernicusDEM 16258/3 all unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019DuGKTmvs5B4EynwZddKMg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant