Skip to content

Fix phantom selections at window boundaries in interleaved solve#273

Open
JoOkuma wants to merge 4 commits into
mainfrom
fix/windowed-solve-boundary-phantoms
Open

Fix phantom selections at window boundaries in interleaved solve#273
JoOkuma wants to merge 4 commits into
mainfrom
fix/windowed-solve-boundary-phantoms

Conversation

@JoOkuma
Copy link
Copy Markdown
Member

@JoOkuma JoOkuma commented May 13, 2026

When the CLI solves the ILP in windows (interleaved evens then odds), the boundary slices between batches ended up with phantom selected=True rows and broken parent_id references. Result: overlap-constraint violations and dangling parents in the final DB.

Refactored SQLTracking with the two-pass design discussed on the PR:

  • Each batch reads the DB to detect whether its neighbours have already committed -- that determines whether each side is anchored.
  • A non-anchored side keeps the configured overlap on the solver window and commits one slice past the inner range so the next-solving neighbour finds its anchor.
  • An anchored side shrinks the solver window to share exactly the anchored slice with the neighbour and commits only the inner range, leaving the neighbour's prior write intact.
  • The artefact-prone outer overlap is never committed -- only the next slice in, where the optimisation is reliable.

To keep the second pass from spawning or terminating tracks at the now-interior seam, MIPSolver.add_nodes gets free_appear / free_disappear flags; SQLTracking sets them False on the anchored side(s) so the solver pays the regular appearance/disappearance penalty there.

add_solution writes the commit range with full parent_id except at the leftmost slice when that slice is also the solver's leftmost (the solver has no incoming edge there and would write NO_PARENT, clobbering the previous batch's lineage).

Synthetic test runs cleanly for window_size ∈ {2,3,4,5,10,100}, overlap_size ∈ {1,2,3}, 10-30 frames, under both interleaved and sequential execution. _validate_tracking_solution now fails on dangling parents as a regression guard.

Replace the previous heavy-handed unselect/rewrite approach with the
two-pass design discussed in the PR:

- Each batch reads the DB to see whether its neighbours have already
  committed; that determines whether each side is *anchored*.
- A non-anchored side keeps the configured overlap on the solver window
  and commits one slice past the inner range so the next-solving batch
  finds its anchor.
- An anchored side shrinks the solver window to share exactly the
  anchored slice with the neighbour, and the commit window stops at the
  inner range so the neighbour's previous write is left intact.

To prevent the second-pass batch from spawning or terminating tracks at
the now-interior boundary, ``MIPSolver.add_nodes`` gets two new flags
-- ``free_appear`` and ``free_disappear`` -- that disable the
``is_first_t`` / ``is_last_t`` free-slack behaviour on the side(s)
anchored to a neighbour.

``add_solution`` writes the commit range with full ``parent_id`` except
at the leftmost slice when that slice is also the solver's leftmost --
there the solver has no incoming edge and would write ``NO_PARENT``,
clobbering the previous batch's lineage.

The synthetic test that previously reproduced the boundary overlap
violations and dangling parents now passes cleanly for every combination
of ``window_size`` in {2,3,4,5,10,100}, ``overlap_size`` in {1,2,3},
and timelapse length, under both interleaved (CLI default) and sequential
(``batch_index=...``) execution.

Strengthen ``_validate_tracking_solution`` to fail when any selected
node's ``parent_id`` references a non-selected node.
@JoOkuma JoOkuma force-pushed the fix/windowed-solve-boundary-phantoms branch from d855f01 to b99816e Compare May 14, 2026 15:02
JoOkuma added 3 commits May 14, 2026 08:51
Covers each anchoring combination of _compute_layout (first pass, both
sides anchored, mixed) and the out-of-range guard of _is_committed_at,
so the new helpers have direct unit-test coverage without depending on
the full segment->link->solve pipeline.
SQLite stores Boolean as 0/1, so reading the column back via pandas
gives an int64 Series. Pandas refuses to use that as a boolean row
mask (`df[int_series]` raises KeyError because it tries to interpret
the values as column labels), breaking the dangling-parent check on
CI. Cast to bool explicitly.
Add the right-anchored-only case alongside the existing left-anchored-
only case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant