Shapes: non-blocking off-thread loading + vertex-pulling FlatPolygonLayer; live picking + shared tooltip

.changeset/shapes-nonblocking-flat-polygon-layer.md

@@ -0,0 +1,48 @@
+---
+"@spatialdata/core": minor
+"@spatialdata/layers": minor
+"@spatialdata/vis": minor
+---
+
+Non-blocking shapes loading + a vertex-pulling `FlatPolygonLayer`.
+
+Shapes no longer gate first paint. The geometry column is decoded (WKB → flat buffers)
+**and tessellated** into render topology inside the geometry worker and transferred back
+zero-copy; `ShapesResolver.blockingResources` is now `[]`, and a main-thread tessellation
+fallback covers the no-worker path. This removes the full-element main-thread WKB decode
+that previously blocked behind the "Loading layer data…" overlay, and it lets Visium HD
+`square_002um` (~2.7M polygons) load without running out of memory.
+
+Polygon shapes now render through a new hand-rolled `FlatPolygonLayer`
+(`@spatialdata/layers`) instead of deck's `SolidPolygonLayer` + a `PathLayer` outline:
+
+- **Vertex pulling** — an attribute-less draw where the vertex shader reconstructs each
+  vertex's position and a boundary edge-distance from two shared geometry textures via
+  `gl_VertexID`, and imputes an anti-aliased outline with `fwidth` in the fragment shader
+  (no separate outline layer). Per-frame cost ≈ the fill; geometry memory ≈ the stock
+  indexed fill. Works on arbitrary polygons (cell segmentation, not just grids).
+- **Feature state via a per-feature colour texture** (the reusable "table column →
+  buffer" primitive): colour-by-column, hide, and fade re-upload only a small texture,
+  never the geometry buffers. Picking colours are computed in-shader from the feature
+  index.
+- **Outline** is a lightened derivation of the fill, width-capped to a fraction of each
+  shape's on-screen size and faded out for sub-pixel shapes — clear when zoomed in,
+  non-dominating (no moiré) when zoomed out.
+
+New/changed public surface: `@spatialdata/core` exports `tessellateFlatPolygons` /
+`TessellatedPolygons` and carries the tessellated topology on `ShapesRenderData`;
+`@spatialdata/layers` exports `FlatPolygonLayer`. `@spatialdata/vis` decouples the
+one-shot auto-fit from the shapes-blocking transition so a shapes-only view still frames
+correctly.
+
+Also fixes a fill-colour "one column behind" bug (the feature-state runtime cache is now
+keyed on the fill-colour entry identity, not just its column signature); the hover/pan
+buffer-thrash from unstable deck `updateTrigger` arrays; and a per-feature colour-buffer
+thrash where two shapes layers sharing the default feature-state runtime rebuilt each
+other's (million-element) colour buffer on every frame — the `FlatPolygonLayer` colour
+cache is now keyed per layer.
+
+Known follow-ups: the main-thread GPU texture upload (~seconds on the largest elements)
+is not yet off the main thread — a WGSL/WebGPU variant (storage buffers instead of
+texture-packing) is the intended fix; explicit per-feature stroke override on the polygon
+path; non-blocking associated-table load.

.changeset/spatialcanvas-picking-perf-and-rules-of-react.md

@@ -9,20 +9,21 @@ SpatialCanvas hover/picking performance and Rules-of-React cleanup.
 Picking/tooltip performance:
 
 - New `hoverTooltipMode` prop (`'off' | 'simple' | 'aggregate'`, default
-  `'simple'`) on `SpatialCanvas` and `SpatialCanvasViewer`, with a matching
-  selector in the `SpatialCanvas` UI. `'simple'` resolves the tooltip from the
-  single top-most pick deck.gl already does for hover/highlight; `'aggregate'`
-  adds `pickMultipleObjects` GPU passes to include every layer under the cursor
-  (more expensive); `'off'` makes shape layers non-pickable entirely (no
-  autoHighlight, no picking-buffer render) — the cheapest mode. Replaces the
-  earlier boolean `aggregateHoverTooltips`.
-- Shape layers are made non-pickable (and `autoHighlight` disabled) while the
-  camera is being panned/zoomed, so deck.gl does not re-render the shape
-  geometry into the picking buffer during gestures. New `pickingEnabled` option
-  on the shapes layer (`@spatialdata/layers`) drives this.
+  `'aggregate'`) on `SpatialCanvas` and `SpatialCanvasViewer`, with a matching
+  selector in the `SpatialCanvas` UI. `'aggregate'` reports every feature under
+  the cursor across layers (`pickMultipleObjects` GPU passes); `'simple'`
+  resolves the single top-most pick deck.gl already does for hover/highlight;
+  `'off'` makes shape layers non-pickable entirely (no autoHighlight, no
+  picking-buffer render) — the cheapest mode. Replaces the earlier boolean
+  `aggregateHoverTooltips`.
+- Picking stays live through pan/zoom. The shapes layer keeps a `pickingEnabled`
+  option (`@spatialdata/layers`) that `'off'` mode uses to drop picking, but it
+  is no longer toggled by camera gestures — the `FlatPolygonLayer` pick pass is a
+  single cheap vertex-pulled draw, so no gesture gate is needed.
 - Hover tooltip resolution is suppressed while a pointer button is held (drag),
   and the per-missing-layer supplemental aggregation pick is collapsed into a
-  single batched pick.
+  single batched pick. The hover-tooltip machinery (pick → tooltip → portal) is a
+  single `useHoverFeatureTooltip` hook shared by both canvas surfaces.
 
 Rules-of-React fixes (eslint-plugin-react-hooks, `pnpm lint:react` now clean and
 the `react-lint` CI job is required): removed ref reads/writes during render and

docs/docs/vis/layer-prop-flow.mdx

@@ -272,22 +272,38 @@ geometry, bounds, shape prebuilds, or feature-state runtimes.
 | Putting `featureState` wholesale in `updateTriggers` | Any new object identity invalidates all colour attributes |
 | Per-feature accessors for layer-wide opacity | Use deck `opacity` on the layer instead |
 
-### Direction: binary attributes (not implemented)
-
-deck.gl's fastest path is **precomputed attribute buffers** (optionally built in
-a worker/WASM) passed via `data.attributes`, bypassing accessor calls entirely.
-Our shapes path still uses `PolygonLayer` / `ScatterplotLayer` with per-feature
-accessors for fill/stroke lookup by `featureId`. That is acceptable for current
-scale targets but is not the long-term ceiling.
-
-Future work (document-only for now):
-
-1. Columnar fill/stroke colours aligned with `featureIds` / `rowIndexByFeatureIndex`
-2. Push colours into typed arrays when encodings change, not when opacity changes
-3. Optional worker/WASM attribute generation for multi-million feature sets
-
-Until then, keep hot paths allocation-free across cosmetic renders and rebuild
-feature-state runtimes only when filtering or encodings actually change.
+### Shapes render path: vertex-pulling `FlatPolygonLayer`
+
+Circles/points still use `ScatterplotLayer` with per-feature colour accessors (their
+counts are modest). **Polygon** shapes — up to millions of features (Visium HD
+`square_002um` ≈ 2.7M) — render through a hand-rolled `FlatPolygonLayer`
+(`@spatialdata/layers`) instead of deck's `SolidPolygonLayer`/`PolygonLayer`, because
+accessor re-execution and a separate outline layer do not scale to that count.
+
+- **Non-blocking, off-thread decode + tessellation.** `VShapesSource` decodes the WKB
+  geometry column into flat buffers **and** tessellates it into render topology inside
+  the geometry worker (`shapesPolygonTessellate`), transferring both back zero-copy.
+  Shapes never gate first paint (`ShapesResolver.blockingResources = []`).
+- **Vertex pulling — no per-vertex attributes.** The layer draws an attribute-less
+  triangle list: `gl_VertexID` selects the triangle + corner, and the vertex shader
+  fetches the topology and shared ring positions from two data textures, computing each
+  vertex's position and a boundary edge-distance on the fly. This keeps geometry memory
+  to two shared textures instead of large de-indexed attribute buffers, and imputes an
+  anti-aliased outline with `fwidth` in the fragment shader — no separate outline layer.
+- **Feature state = "table column → buffer".** Colour-by-column, hide, and fade live in
+  a small **per-feature colour texture** indexed by feature; a feature-state change
+  re-uploads only that texture, never the (large) geometry textures. This is the reusable
+  primitive for column-driven encodings; picking colours are computed in-shader from the
+  feature index.
+
+Keep hot paths allocation-free across cosmetic renders: the geometry textures and the
+per-feature colour buffer keep stable identities, so deck re-uploads only when the
+geometry or the feature-state runtime actually changes.
+
+Deferred: emitting texture-ready (padded) buffers from the worker to shrink the
+main-thread GPU-upload cost; a WGSL variant (WebGPU can use storage buffers instead of
+texture-packing); a true polygon SDF path. See
+[`docs/plans/shapes-nonblocking-tiled-loading.md`](https://github.com/Taylor-CCB-Group/SpatialData.js/blob/main/docs/plans/shapes-nonblocking-tiled-loading.md).
 
 ## Shape/table annotation controls
 
@@ -302,12 +318,21 @@ When adding these controls, keep the visual encoding layer-specific: choosing
 one fill-colour column for a shapes layer must not affect another layer that
 renders the same shapes element.
 
-Fill-colour encodings should normally control both polygon fill and outline
-colour. Polygon outlines should use the shared shape stroke defaults from
-`@spatialdata/layers`: common-coordinate line width, `lineWidthMinPixels: 0`,
-and a small max-pixel clamp. That keeps outlines from dominating when many
-shapes become tiny on screen, while still allowing headless callers to override
-stroke width, units, and min/max pixel clamps per layer.
+The fill is the *specified* colour (a layer default or a data-column encoding);
+the outline is **derived** from it. `@spatialdata/layers` (`deriveStrokeColor`)
+lightens each feature's resolved fill for its outline, so adjacent shapes read as
+distinct — an outline the same colour as the fill is invisible at the thin default
+width, which is why shapes previously did not read as shapes. A genuine per-feature
+stroke override still wins over the derivation, so the vis layer must **not**
+pre-mirror the fill into `strokeColorByFeatureId` (that would look like an explicit
+override and defeat the derivation). On the object (`ScatterplotLayer`) path this is a
+per-feature accessor. On the polygon (`FlatPolygonLayer`) path the outline is **imputed
+in the fragment shader** from the per-vertex boundary edge-distance — no separate
+outline layer — and lightens the fill there; its width is capped to a fraction of each
+shape's on-screen size and fades out entirely for sub-pixel shapes, so it stays clear
+when zoomed in but never dominates (or aliases into moiré) when zoomed out. A genuine
+per-feature stroke override on the polygon path is a follow-up; today it always
+lightens the fill.
 
 ## See also