Nine grid systems from the DGG literature, with an audit - #25
Nine grid systems from the DGG literature, with an audit#25asinghvi17 wants to merge 17 commits into
Conversation
# Conflicts: # README.md # docs/src/all_dggs.md
`MOC_SWEEP` and `CONGRUENT` still listed the six systems the file was written against, so `member_neighbors` — a registry-wide law — was untested on all nine systems this branch adds. Extend both to all fifteen entries, and add the coverage guard the subset sweep above already carries so the list cannot silently fall behind again. `CONGRUENT` becomes a predicate rather than a type tuple: an `AuthalicSystem` wrap keeps the wrapped system's refinement, so AusPIX has to answer the geometric law as rHEALPix does. Levels are chosen per system so the coverage is genuinely mixed-level on every aperture — ISEA3H needs 10, ISEA4H and the aperture-4 rhombics 8, rHEALPix and the aperture-9 rhombics 6. All nine pass, including the geometric boundary-sharing law. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six defects, none of them an algorithm that had to be slow. Measured at
the level nearest 200 000 cells, every result stored so no call is
eliminated as dead.
* The IVEA/RTEA inverse projection bisected 58 times and called
`slerp` inside the loop, recomputing arc BC and its sine on every
one. The map is smooth, strictly increasing, and [0,1] brackets the
root, so Illinois keeps the bracket and converges superlinearly.
`cell_centroid` 2.9µs -> 547ns, and the area residual at the
level-16 apex probe is 1.0e-25 against the old 1.6e-18.
* IVEA/RTEA `node_extent` sampled eight segments per chart edge. It is
a cap, so wider is conservative: four widens it 6.7% against halving
the projections at every node of every descent. 97µs -> 9.6µs.
* Four systems sorted rings with `sort!(...; by = ...centroid...)`,
which runs `by` per comparison — about 25 inverse projections to
order nine cells. Keys once, then `sortperm`.
* rHEALPix re-derived seam topology from the projection on every
crossing, four forward and two inverse projections per cut, nine
cuts for a vertex star. The topology depends on the two polar-square
placements alone, so it is a 384-entry table built at load and
checked against the derivation for 16 placements at three longitude
origins. `edge_neighbors` 146ns, allocation-free.
* ISEA4T's vertex star filtered candidates with a FLATTENED generator,
so each candidate's densified 24-point boundary was rebuilt once per
vertex of the subject's ring. Vertex incidence is about the three
corners. `neighbors` V 785µs -> 33.3µs, k=2 10.3ms -> 382µs.
* ISEA3H/4H `_locate_in_root` rebuilt each candidate's whole
development-plane offset from digit one, with an allocation each
time. The offset is a prefix sum. Carry it, tabulate the directions
and radial scales, hoist the query angle. `cellat` 150µs -> 33.5µs,
`neighbors` 832µs -> 205µs.
The obvious seventh fix is deliberately absent. Densifying the IVEA/RTEA
boundary to a fixed chart STEP rather than a fixed segment count is 372µs
-> 2.2µs, and it opens a lens between a parent's polyline and its
children's that a mixed-level MultiOrderCoverage reads as a sliver — five
sampled interior points of California in no cell at all. Exact nesting
needs `nseg(l) == s * nseg(l+1)` to the bottom of the tree, which is 2^25
segments at a root. The count stays level-independent; the reasoning is
in the comment so the next reader does not rediscover it the same way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`docs/research/literature-grid-performance.md` joins the four research docs already tracked past `.gitignore`'s `docs/*`: the fifteen-system table, the six fixes with mechanism and before/after, the reverted seventh with why it cannot be had, and what is left slow with the reason it is architecture rather than oversight. Two claims in README were left behind by this branch and are corrected rather than restated. `subtree_border` is not an O(rim) automaton on every system but A5 — it is one on IGEO7, H3, HEALPix, S2 and ISEA4R, and the nine systems added here all take the fallback. A5 is not the only system without `has_sorted_subtrees` — the four IVEA/RTEA rhombics are row-major within a root rather than curve-ordered, so `level_ranges` throws there too and `member_neighbors` falls back to a per-call member dictionary. Morton order would give them contiguous ranges, which makes it a change they have not made rather than a property they lack. The `systems()` docstring gains the reason IVEA/RTEA `cell_boundary` draws 128 points per cell at every level: the count is level-independent on purpose, and the mixed-level tiling law is why. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rewrite the node_extent paragraph in the performance audit: the central-place refinement does force the covering cap past the cell, but only to a measured ~2.0 (ISEA3H) / ~1.68 (ISEA4H); the declared 4.5/3.5 come from a magnitude-sum limit that ignores the rotation of the digit directions, and the hemisphere- crossing extents (ISEA3H levels 0-1, ISEA4H level 0) that forced require_convex_extents=false are the constant's doing, not the refinement's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cap_inflation` was 4.5 and 3.5. The covering factor of a descendant `n` levels down is `sqrt(3)*|sum_j q^j u_j| + q^n`, and those came from bounding `|sum|` by `sum |.|` — a triangle inequality that assumes every level's step points the same way — plus margin. Aperture 4's digit directions do not depend on depth or on the preceding digit, so a single repeated direction really is its worst case; but that is `sqrt(3) = 1.7321`, and `1 + sqrt(3)` had added a whole ancestor circumradius for a descendant that shrinks to nothing as the subtree deepens. Aperture 3's directions rotate — odd depths use a fixed gauge, even depths are gauged by the preceding digit — so consecutive steps largely cancel. Taking the support function of the reachable displacement set (a DP over `(depth, previous digit)`, swept over direction with a `sec(pi/M)` enclosure and a geometric tail bound) gives a supremum of exactly 2, approached by the alternating sequence `1,2,1,2,...` and only from an even-depth ancestor; odd-depth ancestors reach only `sqrt(3)`. Brute force over every admissible sequence to depth 13 gives 1.99942 with 0.0023 of tail left, and the displacement model is checked digit-by-digit against `_dev_step` itself. The sphere adds ~1%, not the 59% the chart's worst-case anisotropy (1.5864) implies: an ancestor and its subtree share one local Jacobian, so the distortion nearly cancels in a ratio rather than multiplying it. Exhaustively over every ancestor at levels 0:4 with seven-level subtrees the sphere-side worst is 2.0211 for ISEA3H — on the odd ancestor levels, as predicted — and 1.721 for ISEA4H over levels 0:3. So: 2.3 and 2.0, carrying 14% and 16%. The widest extent goes from 168 and 131 degrees to 86 and 75, every cap is convex, and both systems now take the conformance harness's convex-extent assertions instead of opting out — which was never a property of the refinement, only of the constant. A test reproduces the measurement in both directions: it fails if the factor is lowered under the geometry, and if the geometry moves out from under the factor. Nothing reproduced either number before, which is how 4.5 went unchallenged. Full suite: 1 205 622 passing, 0 failing, 60 broken — up 160 passes and down the two `require_convex_extents` skips. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Main's subset-halo work and this branch's nine literature systems were written against different registries: main sweeps six systems, this branch registers fifteen. The textual conflicts were the easy half — nine files, every hunk main-tightened-prose against branch-added-prose, resolved by adopting main's register and re-expressing the measured substance in it. The rest is what the two halves cost each other. `coarse_probe_rings`, a new system trait. Main's subset prune descends a coarse node only when a one-ring neighbour touches the subset, which is sound exactly when a node's descendants stay inside its own footprint. Central-place refinement does not honour that, and ISEA3H breaks it: counting the neighbours whose ancestor is neither the cell's own nor in its ancestor's one-ring gives 20 escapes at target level 2, 180 at 4, 1620 at 6, and zero at two rings throughout. The consequence was silent — at level 9 the walk returned 596 of the 649 halo cells. The escapes appear only from ODD coarse levels, which is the parity the alternating-digit maximiser predicts, the even-depth ancestors being the ones driven to the full covering factor 2 rather than to sqrt(3). ISEA4H does not need it and does not get it: zero escapes at one ring at every coarse level through target 7, its overhang being sqrt(3), short of the one-ring reach. Widening it would have restored exactly the pruning slack this branch removed from `cap_inflation`. Two rings costs 4.85x the queries — 43603 against 8987 on a level-9 subset — which main's "the walk inspects only a fraction of members" bound cannot absorb. The one-ring walk met that bound because it was wrong. The asymptotic law, that query growth follows halo size, still holds for every system; the absolute fraction is now pinned per probe width, so a third ring still fails. `law_root`, replacing `cellindex(levelgrid(sys, 0), 1)` in the cross-system laws. ISEA3H and ISEA4H seal their two polar roots as single-child chains, so root 1 has a one-cell subtree at every depth and every growth law read the pentagon's five-cell one-ring at every level. It resolves to root 1 on the other thirteen systems, so the constants pinned against them are undisturbed — their subset question totals come back byte-identical. AusPIX is a registered `AuthalicSystem`, so main's re-wrap sweeps now skip the systems that are already wrapped rather than throwing on them. The suite went to 230 minutes on the merge, 222 of them in the halo file and 170 in a single @test_broken testset. The cause is not the tests: the four aperture-9 systems have no specialized halo engine, so their walk costs what the full-grid reference scan costs — 4.46s each at depth 7 on rHEALPix against 0.006s on S2 — and level 7 is a 16k-cell subtree at aperture 4 but a 4.8M-cell one at aperture 9. The depth ladders are now capped by subtree size at the largest one these laws already walked, and stretched at the coarse end where aperture 3 could not reach an aperture-4 growth ratio. Every system main swept keeps its exact depths. That missing engine is recorded rather than hidden: the subset question totals now pin all fifteen systems, and IVEA9R and RTEA9R sit at 535818 where rHEALPix, at the same aperture and subset size, asks 9612. They are the systems with neither a specialized halo engine nor sorted subtrees, so the walk cannot prune by descendant range at all. Closing that gap will fail these pins, which is the intent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The eager-engine testset is the negative control for the three allocation laws above it: wrap the real engine in one that materializes, and check the laws refuse it. Without it they could be passing because nothing measurable is allocated either way, which is the failure mode a positive law cannot see. It does not need the whole registry to say that. Eagerness is a property of the wrapper, not of the system underneath — an eager engine over IVEA4R fails for the same reason it fails over S2 — so the only axis worth spanning is the engine being wrapped. Three systems cover it: a square specialization, a hex one, and a system on the generic walk. Sweeping all fifteen re-proved that at depth 7 on 4.8M-cell aperture-9 subtrees. It was 170 of the 230 minutes this suite took when the file first met the literature registry, and 24 of the 41 that survived sizing the depth ladders. The file now runs in 16m39s and the suite in 25m32s, with the 48 dropped assertions all being the twelfth through fifteenth re-proof. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The oracle sets are 73k of the 80k lines this branch adds, which buries the 2.5k lines of implementation under them in any review of the whole branch. They are not large where it costs: coordinate text compresses 7.3x, so the 13 MB tree is about 1.75 MB of a 28 MB pack, and a Float64 binary form would save only a fifth of that — mantissa bits do not compress — while making the contract unreadable. So this marks them rather than reformatting them. `linguist-generated` and deliberately not `-diff`. Collapsing them in the PR view is the goal; suppressing `git diff` is not. Regenerating a family with `scripts/oracles/` and reading exactly which cells moved is a normal thing to want, and `-diff` would answer "binary files differ". The eol pin covers the other half: these are parsed as coordinate text and read as fixed-width fields in places, so a CRLF checkout would shift every digit position. `git add --renormalize .` is a no-op today, so this pins the current state rather than changing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eighty-three commits, carrying a store-IO layer, the neighborhood forms, a `GlobalRegridding` workspace package, and `CopernicusDEMSystem`. Every textual conflict was additive on both sides — main adding CopernicusDEM in the same places this branch adds the literature families — so all four kept both sides. What did not survive verbatim is main's arithmetic about itself. Three statements were true when written and are not true on the merged tree, so taking main's side unchanged would have committed a falsehood: `interface/system.jl` had tightened "none of the shipped here does" to "none of the SEVEN shipped"; the registry is fifteen. The number-free form is back. `DiscreteGlobalGrids.jl` called CopernicusDEM "the seventh system, included but not registered". It is no longer the seventh, and the ordinal bought nothing the sentence did not already say. The README claimed "All seven return `HierarchicalLevelGrid`" and that A5 is "the one system without `has_sorted_subtrees`". The branch's counterparts are the post-merge truths — fifteen entries, and A5 plus the four IVEA/RTEA systems. One of main's counts was already stale on main: the README says six tutorials under `docs/src/tutorials/` and there are seven, `store_io.jl` having arrived without the count following it. Dropped rather than corrected, since it is the third count in this file to go stale in two merges. Main's new `neighborhood.jl` sweep failed on nine systems, which is its own coverage guard working: the list was written for main's seven. The nine are added with levels chosen per aperture rather than shared, because `covlvl` is a level number and level 10 is not the same work at aperture 3 as at aperture 9. Calibrated against main's own rows — 779-1035 coverage cells over 48-80 windows — the nine land at 213-873 over 19-49, and cost 0.7-2.0s each against main's 0.6-4.7s. Suite: 1,255,104 passing, 0 failing, 63 broken, 28m31s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The literature systems follow main's spellings — `maxlevel`, `maxneighbors`, and `border`/`interior`/`halo` over a subtree region — and `coarse_probe_rings`, which this branch adds, joins `cap_inflation` in the `public` tier rather than the export list. Congruence is what two of the cross-system laws actually branch on, and it is now wider than main's `isquadface`: ISEA4T's four triangles, rHEALPix's nine quads and the IVEA/RTEA rhombi all tile their parent. `iscongruent` states that in `test/helpers.jl`, beside the trait it generalises, and the sweep tables go back to main's shape. `sweepcovers` grows an `except` for the two systems that reject the budget mode outright. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SwnXToXzkTdstjqfp8ovhH
`_multi_order_query` moved to `Engine` with the rest of the query planner, so the two central-place hexagon systems were calling a name that no longer existed and every coverage query on them threw. The fast conservative path reaches all nine literature systems once it resolves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SwnXToXzkTdstjqfp8ovhH
… break
Four selectors read true on main's six systems and false on the reality the
nine literature ones create:
* `coarse_probe_rings` was reachable from `Fallbacks` but not from `Engine`,
where the subset halo walk now lives, so ISEA3H's two-ring probe threw.
* The coarse-containment law computed `k` and then walked one ring anyway;
it walks `k` again, which is what ISEA3H's escape counts pin.
* `sortedsubtrees` stood for "ships a specialized halo engine" only while A5
was the sole exception. `hashalowalk` asks the question directly, and
selects the same five systems `runtests.jl` names as automata.
* The eager negative control needs a ladder it can watch grow. The shallow
one is A5's cost dodge, and A5 is not one of the three controls.
`mapneighbors.jl` takes `neighborhood.jl`'s literature rows — it is the same
sweep, split. And the rotational-adjacency law is asked only of a shape with
rows enough to answer it: the every-fifth-cell subset leaves ISEA4H two such
rows and the aperture-9 rhombi none. A new testset makes each system witness
the law over its shapes together, so the skip cannot hide a `sort!`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SwnXToXzkTdstjqfp8ovhH
…f work `the subset walk's cost follows the halo, not the subset` materializes the whole descendant range through `holed_subtree`, and its second rung was a bare level 9. That is a 262k-cell subtree at aperture 4 and a 387-MILLION-cell one at aperture 9, so registering rHEALPix and AusPIX took the file's peak resident size from 0.88 GB to 15.9 GB — past what a CI runner has, which is why the 1.12 ubuntu job was killed rather than failed. `law_depths` is the cap the rest of the file already uses. Every system main swept keeps its exact ladder — the aperture-4 three stay at `(4, 9)` — and the two aperture-9 newcomers come back to `(4, 6)`, a 531k-cell subtree. Measured at 2 threads: 24620 pass, peak 0.88 GB. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SwnXToXzkTdstjqfp8ovhH
|
Merged Conflicts (11). Four selectors read true on main's six systems and false on the fifteen. Each was found by a failing assertion, not by inspection:
Why 1.12-ubuntu was red. Suite. 1256419 pass / 0 fail / 63 broken, 39 min at Fast-to-regrid check. Every one of the nine reaches the specialized
No system needed a new hook. The one thing that was actually broken was The IVEA/RTEA four are ~2× the rest at equal cell counts, and that is the declared Not merging — over to you. |
…he cone `_hex_boundary` asked `dev_angle_deg` whether a corner had run past the five-face cut. That helper exists for the decoder, where an angle above 345 degrees is the floating-point shadow of angle 0 and must not be read as a continuation; it folds such an angle to a small negative. A hexagon corner, though, genuinely reaches into `(345, 360)` — the six corners of a cell whose centre sits on the cut straddle it — and the guard exempted exactly those from the wedge test, so they stayed on the wrong side. One corner per cell was wrong, on the five southern roots, at every level from 2 down: 5 cells at ISEA3H level 2 and 75 at ISEA4H level 4, up to 0.216 degrees off the sealed DGGRID rings. The paired-edge construction below it then drew the same physical edge twice, once per incident cell, so 20 ISEA3H and 60 ISEA4H neighbour pairs at level 2 shared one sampled point or none instead of all nine. The polygons did not tile, and the DGGS-as-source direction of a conservative regrid lost 0.7% (ISEA3H) and 1.1% (ISEA4H) of a field of ones. The raw modular angle is what `_dev_fold` and `_hex_neighbors1` already use, and the fold it enables is the cone's own gluing, so a corner on the cut ray maps to the same sphere point either way. With it every committed boundary vector matches to 6.2e-7 degrees through ISEA3H level 5 and ISEA4H level 4, every level-2 neighbour pair shares its nine samples, and both families conserve as a regrid source to 1.3e-13 — the other thirteen systems' figure. The sealed `*-boundaries.geojson` vectors were committed but never read; they are now the corner oracle, alongside a shared-edge pin that catches a fold error below the acquired oracle depth. The `@test_broken` arms that pinned the old behaviour in `regridding_conservation.jl` become ordinary assertions; the DGGS-as-destination direction stays broken for these two, as it is for every non-convex ring, on the upstream Sutherland-Hodgman limitation the file's header describes. Also: `cellposition` on ISEA4T and ISEA4H answered for levels outside `levels(sys)` — 1 and a `BoundsError` respectively, where every other system answers `nothing`; compact `show` methods for the four systems that carry constructor parameters, so `systems()` prints as its docstring says; and `member_neighbors`'s congruence lists, which still named only the six systems that existed before this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SwnXToXzkTdstjqfp8ovhH
Codex CLI review, and the disposition of it
1. Reproduced and root-caused. One corner per cell, on the five southern roots, at every level from 2 down: Codex's downstream observation is right too: the paired-edge construction then drew the same physical edge twice, once per incident cell. Level-2 shared-sample histogram, over every neighbour pair: The polygons therefore did not tile, and that is what the "finite approximation" story in The fix is the raw modular angle, which The 2. Codex diagnosed the cause correctly and then declined to draw the conclusion. The destination direction fails for non-convex rings, which is every curvilinear system in the package, six of which predate this branch:
3. The out-of-range
The level-bound half is real and is fixed: 4.
5. The four systems carrying constructor parameters printed their internals. They now print compactly, as Negative results worth recording. Codex found no seam, antimeridian or polar covering misses on any of the nine; exact poles longitude-independent; cell counts matching every aperture formula and areas summing to Suite. Julia 1.12, |
1 participant
Adds nine grid systems from the DGG literature and brings them up to the
package's contracts: ISEA3H, ISEA4H, ISEA4T, rHEALPix, AusPIX, IVEA4R,
IVEA9R, RTEA4R, RTEA9R. The registry goes from six systems to fifteen.
Each family is pinned against sealed black-box oracle vectors — DGGRID 9.0b for
the ISEA families, DGGAL for the rhombic ones — committed under
test/oracles/rather than regenerated, so the vectors are the contract and not a run of
whatever happened to be installed.
What is in here
The systems, with centre/boundary/area/neighbour implementations, the
conformance harness passing on all nine, and
member_neighborsswept overevery registered system (it had been untested on nine of the fifteen).
A performance audit (
docs/research/literature-grid-performance.md): amethod, a fifteen-system table, six fixed defects, one documented dead end, and
the residues that remain. The six fixes were repeated work rather than
architecture — a 58-iteration bisection in the rhombic inverse projection, a
node extent sampling twice the perimeter it needed, ring sorts recomputing keys
per comparison, rHEALPix re-deriving seam topology on every crossing, ISEA4T
rebuilding boundaries per vertex, and the ISEA3H/4H inverse re-walking every
prefix it extended.
A corrected
cap_inflationfor ISEA3H/4H, 4.5 and 3.5 down to 2.3 and 2.0.The old values came from a triangle inequality that assumes every level's
overhang points the same way; the central-place digit directions rotate and
mostly cancel. The true planar suprema are exactly 2 at aperture 3 (alternating
digits, even-depth ancestors only) and sqrt(3) at aperture 4, derived by a
support-function DP and cross-checked by brute force, with the sphere adding
about 1%. The old 4.5 put a level-0 extent at 168 degrees — past a hemisphere,
which is the only reason these two systems used to opt out of the harness's
convex-extent check. They no longer do.
What the merge with main cost
Main's subset-halo work and this branch were written against different
registries. Three things broke where they met, all detailed in
18961c2:when a one-ring neighbour touches the subset, which is sound when descendants
stay inside their ancestor's footprint. Central-place refinement does not:
ISEA3H had 20 escaping neighbours at target level 2, 180 at 4, 1620 at 6, and
at level 9 the walk returned 596 of 649 halo cells. New
coarse_probe_ringstrait, 2 on ISEA3H and 1 everywhere else — ISEA4H does not need it, having
zero escapes at one ring through target level 7.
chains, so the cross-system laws — which hang subtrees off root 1 — were
reading the pentagon's five-cell ring at every depth.
AuthalicSystem, so main's re-wrap sweeps tried todouble-wrap it.
Known gaps, deliberately not closed here
None of the nine has a specialized halo engine, so their walk is the generic
one, which costs what the brute-force reference scan costs: 4.46s against 4.46s
at depth 7 on rHEALPix, where S2 does 0.006s against its own 0.72s. Worst on
IVEA9R/RTEA9R, which also lack
descendant_range— one subset walk asks 535,818questions where rHEALPix asks 9,612 at the same aperture and subset size. Those
counts are pinned in
subtree_halos.jl, so closing the gap fails the pin.Two open decisions are in the audit's final section: the rhombic boundary
density (128 points per cell where every other congruent system draws 32) and
Morton reindexing for IVEA4R/RTEA4R, which would unlock
descendant_rangeandthe engines above but changes canonical cell identity.
Testing
Full suite green locally: 1,230,578 passing, 0 failing, 60 broken (the
pre-existing count), 24 minutes.
Worth noting the suite runtime, since it is a merge artifact rather than
something either side had alone: the cross-system halo file went to 222 minutes
when it first met the fifteen-system registry, because its depth ladders are
written as level numbers and level 7 is a 16k-cell subtree at aperture 4 but a
4.8M-cell one at aperture 9. The ladders are now capped by subtree size at the
largest one those laws already walked, and the eager-engine negative control is
restricted to three systems spanning the engine families rather than sweeping
all fifteen to re-prove a property of the wrapper. That file now runs in 17
minutes; every system main already swept keeps its exact depths.
CI has not run on any of this — the workflow is
pull_request-only and this isthe first PR. Julia 1.11 is untested locally.
Note that
mainhas advanced 18 commits past the merge point (9ae1f4e);none of them touch the halo API or these systems.
🤖 Generated with Claude Code