One project file: the .licht format - #1525
Draft
MrNeRF wants to merge 69 commits into
Draft
Conversation
A 0-3 byte .resume or .rad file passed an unchecked file.read() and compared indeterminate bytes as the format magic. Probes now require an exact four-byte read: checkpoint canLoad() returns false, RAD validate-only returns INVALID_HEADER with expected/actual byte counts. QW-1 of the error-architecture hardening batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…paths Request IDs are now a three-state RequestId type (absent/null/value) parsed and serialized in one place: parse errors respond with id null instead of a fabricated 0, valid ids are echoed on every error path, and notifications omit the id field. Missing or non-string tool names map to INVALID_PARAMS instead of surfacing as json exceptions. HTTP request and listener entries funnel exceptions through one logging guard; wire messages are sanitized constants, exception text stays in the server log. QW-4 of the error-architecture hardening batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nd_exit An exception escaping mode dispatch reached std::terminate; nine bare std::_Exit sites could drop buffered diagnostics. main() now runs modes through run_with_exception_firewall (one failure report, exit code 70 / EX_SOFTWARE), every hard exit goes through flush_and_exit, and the terminate handler shares one report_current_exception helper with the firewall. Signal handlers keep their async-signal-safe raw path. QW-3 of the error-architecture hardening batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ding Reader-done event waits, the viewer-release external-semaphore wait, and sidecar ready-event waits discarded their status: training could proceed and reuse GPU memory whose ordering edge was never installed. Each wait now reports through the CUDA failure path and aborts the step; reader ring bits clear only after their own successful wait, and an event whose wait was rejected is reaped in destroySyncPrimitives() after stream synchronization instead of being destroyed mid-flight. QW-2 of the error-architecture hardening batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Default runs previously had no durable log: the file sink existed only with --log-file and truncated the prior session. Every run now writes a rotating lichtfeld.log (10 MiB, active + 4 backups) under the per-user directory; --log-file adds a second rotating sink with the same cap and same-path dedupe. File sinks flush on Error and every two seconds. Failure to set up the default sink prints one stderr warning and leaves console/memory logging intact instead of failing startup. QW-5 of the error-architecture hardening batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…_DEBUG mode list Seven diagnostic gate families collapse to one: LFS_CUDA_SYNC_DEBUG now takes a mode list (cuda-sync, device-trap, vk-fatal), parsed once into a cached bitmask. Legacy truthy values mean cuda-sync for one release and LFS_VK_VALIDATION_FATAL is a deprecated alias for vk-fatal. The dead TENSOR_VALIDATION_ENABLED, OffsetAllocator DEBUG, RMLUI_DEBUG coupling, and always-false FastGS/edge config::debug gates fold into DEBUG_BUILD or the runtime mode; sync-debug now actually covers the FastGS optimizer and edge rasterizer phases, which the dead compile constant had disabled in every build. Phase 0a of the error-architecture rollout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…teardown Port of the allocator-shutdown fix already on LichtFeld-Studio lanes: CudaMemoryPool's destructor shuts down subordinate allocators, so its dependencies (VramProfiler, CudaEventPool, GPUSlabAllocator, SizeBucketedPool) are constructed first to survive reverse static destruction; without this, process exit could release GPU buffers after the CUDA context was gone, SIGABRT mid-teardown, and leave nvidia_uvm holding orphaned state that wedges cuInit machine-wide. CPU-only commands no longer initialize CUDA just to tear it down. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lexical census over src/ with nine stable rules (string-valued expected, discarded CUDA statuses, raw VK check macros, empty catches, local check macros, fatal-invariant sites, infinite Vulkan waits, unchecked kernel launches, host-only result types in CUDA units), a checked-in baseline, and a CI job that fails only when a per-rule, per-file count increases — line drift of pre-existing debt cannot re-flag it. .clang-tidy starts as local/IDE reporting until compile_commands lands in CI. Phase 0b of the error-architecture rollout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gate's design always included an escape for deliberately empty catch-alls and a sanctioned-exception list; the script now implements both: a raw-comment annotation LFS-CENSUS-OK(empty-catch) exempts a reviewed handler, and ALLOWLIST entries (each with owner, reason, expiry) exempt sanctioned declarations such as the legacy string bridge. Surfaced by Phase 1's own code failing the gate it must live under. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One-pointer immutable COW error handle (sizeof(Error) == sizeof(void*)) carrying the stable code/domain taxonomy, severity, retryability, operation id, native status, bounded context frames and suppressed errors; [[nodiscard]] Result<T> wraps std::expected<T, Error> with a pointer-only Result<void>, no implicit string/expected conversions, and explicit adapters (from_legacy_expected, io::Error, std::error_code, AllocationFailure). make_error is noexcept with an immortal OOM seed; error.hpp is host-C++23-only while error_codes.hpp stays CUDA-safe. Success paths allocate nothing (measured ~2-3 ns per Result<void> return); 41 tests including COW isolation, bounds, UTF-8-safe truncation, and TSan/ASan-verified concurrent copy/destroy stress. Phase 1 of the error-architecture rollout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rror report(error, channel) renders the context chain once, applies the per-code stack-capture policy, and dedupes through FailureReport's existing engine via a fingerprint of code + native status + detection site + top operation. Logger handlers are contained: a throwing handler is disabled without affecting the caller or later handlers. Re-entrant reporting and pre-init reporting degrade to a fixed-buffer stderr fallback; the level-suppressed fast path does no formatting (~32 ns). ProcessBoundary reports additionally guarantee one stderr line. Phase 2 of the error-architecture rollout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A catch body that converts the exception into a typed error return (make_error / std::unexpected) is the campaign's mandated boundary pattern, not a swallowed exception. Count it as reviewed instead of requiring an LFS-CENSUS-OK annotation at every migrated boundary. Masked-literal test guards against string-content false acceptance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
load_ply now returns lfs::Result<LoadOutcome<SplatData>>: every detection site throws a typed lfs::Exception with stable ErrorCode, structured SmallFields (byte counts, row counts, property names) and CUDA NativeError payloads; a single outer catch chain attaches `path` once and never logs. Partial-invalid-row loss is a structured Diagnostic carried in LoadOutcome::warnings instead of a LOG_WARN. PLYLoader::load is the one owner-log site (ErrorReporter::report), bridging back to the legacy io::Error surface via new from_lfs_error/from_lfs_error_code adapters and a new io::ErrorCode::RESOURCE_EXHAUSTED bucket. 20 propagation-path LOG_ERROR/LOG_WARN calls deleted from ply.cpp (24 -> 4; a corrupt PLY used to log 2-3 times before leaving the file, now exactly once at the loader boundary). load_ply_point_cloud keeps its legacy exported signature (4 external callers; Phase 11 manifest). Tests: new PlyErrorTaxonomyTest (8 fixtures: NotFound+path field, InvalidArgument, ResourceExhausted overflow, DataLoss truncation with exact byte fields, all-invalid failure, partial-invalid structured warning, Cancelled, exactly-one-owner-report); 6 legacy test files migrated mechanically to the new return shape. Census baseline regenerated: expected-string 761 -> 759, empty-catch 215 -> 127 (includes the typed-error reclassification from the previous commit). Verified: build -j4 green; 191/201 targeted tests pass (10 optional- fixture skips); 2 RotatedShCorrectnessTest reds proven pre-existing by stash A/B against the pre-change tree; load-path perf parity on a 188 MB real PLY (A/B medians 71ms -> 67ms); shuffle x3 stable; sanitizer-free CPU path (no CUDA changes). Independent verify agent unavailable (3x API 529) — verification executed by orchestrator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Convert the 23 file-controlled assertion/throw sites left by 3A to typed throw_ply_error failures: 12 in parse_header (header structure, schema suffixes, format variants), 7 in validate_ply_layout_for_import (including two former bare runtime_error throws for partial scale/rotation schemas that surfaced as Internal), 4 in load_ply_point_cloud (direct std::unexpected returns; exported signature byte-identical). The 15 true load-path invariants and all 19 save-path assertions stay untouched. Fixes the "tensor contract violation" mislabel: LFS_ASSERT_MSG (active in every build type) emitted a wrong-family FailureReport and a bare runtime_error that landed as Internal; these sites now carry the correct taxonomy (InvalidArgument/Unsupported) with structured fields and zero spurious FailureReports. HostBuffer's two propagation-path LOG_ERRORs are gone (identity + requested count now ride the ResourceExhausted throws via describe_staging_failure), and PlyHostStaging::valid() checks a new alloc_failed flag uniformly — a lone shN_swizzled host-OOM previously masqueraded as a legitimately SH-less file. Tests: 8 new PlyErrorTaxonomyTest fixtures (structural-malformation table, unsupported-variant table, 4 point-cloud rejections via legacy CORRUPTED_DATA mapping, no-tensor-contract-violation regression, seeded 200-iteration header-truncation fuzz over a closed error-code set). LFS_*ASSERT_MSG in ply.cpp 55 -> 34; bare throws 2 -> 0. Verified: build -j4 green; 16/16 taxonomy + 62/63 taxonomy+PythonIO (1 optional-fixture skip); shuffle x3; census gate green (baseline unchanged — counts stable, rules do not census raw asserts); perf parity by stash A/B under identical load (188ms vs 192ms medians, noise); independent fable verification SHIP (8 NOTE findings: polarity audit clean on all 12 conversions, valid() equivalence proven for all reachable states, moved-from flags cannot poison reuse). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oundaries Apply the frozen fatal-vs-skip matrix to the COLMAP readers: structural corruption (truncation, impossible counts, unterminated records, unknown camera models, duplicate kept IDs, trailing bytes, zero usable cameras/images) is one typed fatal failure; invalid per-record data (camera dimensions/params, image pose, point xyz/error, track tokens, point2D observations) is skipped and tallied with at most eight sampled record IDs, never one log per row. Binary skips consume the full record before the verdict so the cursor never desyncs; unknown binary model IDs stay fatal for exactly that reason (record length unknowable). 53 LFS_ASSERT_MSG conversions (27 fatal, 25 skip, 1 mixed) + 15 plain throws removed + 15 propagation LOG_ERRORs deleted in colmap.cpp; five Result boundaries get the three-clause catch chain with single path attachment; ColmapLoader::load gains its first structured owner report (lfs::Exception clause only — the pinned partial design). Text-mode zero-points now legal, matching binary and the loader's random-init fallback. Skip-recovery consistency fixes: scene-center tensors shape from the kept count, and camera uids assign from the kept index — in-assemble image skips previously left uid holes that crashed BilateralGrid's dense indexing at training time (found by verification, pinned by SkippedImageLeavesNoCameraUidHoles). Tests: new ColmapBinaryErrorTaxonomyTest suite (10 fixtures: per-type skip/fatal matrices, sample-cap, retained-record byte-equivalence, seeded short-read sweep over every byte length) plus migrations across 4 existing test files for the Result<LoadOutcome> signatures. Verified: build -j4 green; 144/144 targeted battery (25 suites incl. PLY/error regressions); shuffle x3; census green (counts stable); real bicycle load: 194 images/54275 points, zero skip diagnostics on clean data, 50-iter train + checkpoint; fable verification NO-SHIP -> fixed the one MAJOR (uid holes) exactly as prescribed, all other passes clean (cursor safety proven site-by-site, scene-center shape, TBB per-chunk tallies, boundary integrity). Non-gating follow-ups in campaign ledger: text pose-skip double-count, record-bearing truncation fixture, full-consumption fixtures for skipped variable-length tails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New lfs::core::run_guarded(TaskContext, TaskBody<T>, TaskCompletion<T>): normalizes lfs::Exception/std::exception/unknown throws into a typed Result<T> exactly once and invokes completion exactly once, guarded by a three-state atomic TaskSettlement (Pending/Completing/Settled). A completion sink that violates its documented no-throw contract degrades to ErrorReporter's ProcessBoundary fallback and the task still settles — per the owner-approved 7.3 amendment (the doc's noexcept-qualified completion alias contradicted its own settle-even-if-throwing clause and injection test; std::terminate would fire before any catch). lfs::vis::post_guarded_and_wait packages posted work: WorkItem run() and cancel() share one TaskSettlement through TaskContext, making exactly-once settlement a type-level property instead of a queue convention; queue rejection and shutdown produce a typed Cancelled result instead of an abandoned future. Five sites converted: the Python clear-scene bridge (a clearScene() throw previously abandoned the waiting future forever), the visualizer posted-work loop (thread-affinity asserted, structured reports, first-failure still cancels the remainder), the training thread (one guarded path always reaches handleTrainingComplete exactly once), the MCP training worker (training_active_ always clears), and the TCP responder thread entry (a non-standard exception no longer terminates the process). No public cross-TU signature changes; async_task_manager, crash_handler, py_rendering, mcp_app_utils byte-identical. Tests: 13 new fault-injection tests (throw before/inside body, partial work visibility, lfs::Exception classification passthrough, cancellation-as-settled-result, queue rejection, shutdown cancellation with bounded waits, throwing-completion as a real non-death test, alloc-failure normalization, thread-affinity death test) plus a legacy display-message bridge test. Verified: build -j4; 124/124 targeted battery; shuffle x3; census green (annotated boundary catches); clang-format; 3x 7k bicycle smokes tight (30.6-31.1s — converted sites are O(1) per run, per-iteration paths untouched); 45s live GUI with 64k-splat PLY through the converted work queue, zero error noise; fable verification SHIP with structural proof that promise abandonment is unreachable (single global set_value per item via shared settlement CAS; nothrow-move Results). Known deviation, verified forced: WorkItem's copyable std::function requires owning the move-only body via shared_ptr. Ledger: latent fast-path assert for pre-claimed settlements; TrainerManager injection tests infeasible without a seam (structurally covered). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every accepted image request now settles exactly once. The bounded
output queue carries only sequence tokens (its VRAM-throttling
backpressure is unchanged); the typed payload lives solely in a
sequence-keyed ledger behind get_completion()/try_get_completion()
returning LoaderCompletion{sequence, Result<ReadyImage>, SidecarTally}
per the frozen 7.3 protocol. Four silent-loss modes are closed: a
failure completion dropped when the output queue rejected a push at
shutdown, a successfully-decoded ReadyImage destroyed in the same race
with no record, prefetch() inflating in_flight_ after shutdown, and
pending pairs erased at shutdown with no terminal outcome — shutdown
now rejects new work, settles or cancels every accepted sequence,
reconciles (accepted == succeeded + failed + cancelled, debug-asserted
and tally-logged), and only then destroys queues, still ahead of CUDA
stream teardown. Sidecar failures tally requested/delivered/failed per
kind via failure flags recorded before expectation flags clear;
Required policy fails the camera's completion, default WarnAndContinue
delivers degraded with one aggregate ErrorReporter diagnostic at the
loader's own boundary. Loader generations gate late worker settlements
across clear()/reset() sequence-ID reuse. Legacy get()/try_get()
adapters preserve pinned behavior; PipelinedDataLoader::next() and
trainer policy untouched (Phase 5).
Seven frozen-spec-vs-live-code discrepancies resolved and disclosed
(registration moved to the acceptance path, decrement-at-consumption,
count-once reconciliation, live strategy pointers, void prefetch,
generation mechanism, pre-existing mask-contract test failures — the
latter proven pre-existing by stash A/B).
Tests: 7 new ledger tests (500-request exactly-once uniqueness, exact
{200,100,100} per-kind tallies, capacity-1 fill-then-fail, shutdown
race matrix with post-shutdown CUDA probe, generation reuse, Required
vs WarnAndContinue) behind a FailureReport-dedup-resetting fixture
(order dependence under shuffle found by gate, fixed).
Verified: build -j4; battery green (2 pre-existing mask failures);
shuffle 12/12 x4 seeds; census; clang-format; 7k bicycle smoke stash
A/B pre 31.09s vs post 31.20s medians (+0.35%, inside noise); live 7k
reconciliation accepted=7007=7005+2+0 — nothing dropped; fable
verification SHIP (lock-ordering, generation double-destroy, unlocked
publish window, in_flight_ accounting all proven; ledger bounded at
queue+workers entries — identical VRAM envelope). Follow-up LOWs in
campaign ledger: stale-token Internal→Cancelled reclassification, dead
ledger_cv_, generation-test exact-count relaxation option.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Trainer::train_step returns Result<StepDisposition> and train() returns Status; StepResult/RetryAfterOom are gone. The old outer whole-step retry — which replayed iteration callbacks, on_iteration_start_, the Python IterationStart hook, pending parameter installs, and the FastGS pre-forward topology/crop/ADMM block on every OOM — is deleted. The only retryable scope is the fast_rasterize_forward call itself: one typed ResourceExhausted retry after checked recovery (recover_forward_oom: sync result verified before clearing sticky state, completed loss readbacks harvested, arena full_reset + pool trim, final CUDA check; a non-OOM fault during recovery becomes the terminal primary with the initiating OOM as its single suppressed entry; second-attempt OOM is terminal). Classification is typed end to end: ForwardContext carries a plain bool resource_exhausted set only by the three real OOM-class causes (host capacity ceiling, cudaMemGetInfo preflight, cudaMalloc failure) — never invalid dimensions, never the sorted-indices invariant guard — and the fast_rasterizer seam builds ResourceExhausted/Internal from it; "OUT_OF_MEMORY" substring matching is gone from trainer and notification bridge (additive TrainingCompleted::resource_exhausted drives the GUI OOM messaging). memory_arena restores the documented nullable-allocator contract (the previous throw shielded gsplat's unchecked blob carve — both gsplat sites now null-check and throw typed ResourceExhausted; verification catch). Every terminal error leaving train_step/train carries a four-field mutation stamp (iteration, mutation_epoch, step_phase, persistent_commit) with mutation_epoch_ incremented at exactly the eight frozen persistent-commit boundaries. Manager and MCP completion adapters collapse to direct Status returns (run_guarded from 4A absorbs them); last_error_/Python trainer_error string surfaces unchanged. GlobalArenaManager::reconfigure_for_testing provides the deterministic capacity-injection seam (always-compiled, test-named per convention). Owner amendments recorded: resource_exhausted cause classification, single suppression attachment point, SmallFields stamp mechanism — each resolving a frozen-spec self-contradiction surfaced by the implementer refusing to choose silently. Tests: 6 new TrainerRetrySemantics (typed bounded retry, invalid-dims-not-retryable, complete mutation stamp, arena capacity injection, first-attempt-only retry, exactly-one-suppressed-OOM); checkpoint tests migrated. Verified: build -j4; 130/130 battery; shuffle x3; census; clang-format; perf A/B 8 samples/side on the 7k bicycle smoke — medians 30.91 pre vs 30.89 post (-0.05%), minima identical 30.70s; fable verification NO-SHIP->SHIP after fixing its HIGH (gsplat null-check) and LOW (invariant-guard classification) exactly as prescribed; retry-scope proof, mutation-map audit, and recovery-order checks all confirmed. Ledger: train()-frame phase stamping reports TerminalCleanup for in-loop failures (train_step's frame carries the accurate phase); pre-existing init-path string scraping deferred to the phase typing Trainer::initialize; follow-up side-effect-counter regression test recommended. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e never drops a status
LFS_CUDA_TRY / LFS_CUDA_LOG_TEARDOWN (host-only cuda_error_typed.{hpp,cpp};
.cu keeps the C++20 cuda_error.hpp surface): TRY samples pre-call sticky
state, evaluates once, throws Exception(Error{CUDA domain}) with the
predecessor attached as suppressed ("NOT the origin"); LOG_TEARDOWN is
no-throw and reports once via ErrorReporter — discarding a teardown status
is now a named decision, never an accident.
Converted per the frozen manifest: 8 require_cuda_success sites (helper
deleted) + rollback/destroySyncPrimitives/shutdown/cleanup/heatmap teardown
in trainer.cpp (34 sites), camera.cpp stream lifecycle + 10 upload-sync
TRYs (12 sites), gsplat tile frees, startup ordering sync, and mechanical
retypes of the QW ownership waits incl. the sidecar wait's hand-rolled
orphan-then-throw sequence. CudaMemoryPool::release_stream no longer throws
from its 13 pre-cudaStreamDestroy callers (several noexcept destructors —
a live std::terminate hazard), via the file-local ensure_cuda_success
LogOnly idiom to stay single-definition under nvcc. CUDA FailureReport
section provider re-keyed to to_string(ErrorDomain::CUDA) so legacy and
typed paths share one registration.
Named ledger deliverable: lfs::core::teardown_gpu_before_exit() — explicit,
idempotent, ordered pool+pinned teardown before flush_and_exit at all seven
normal exit paths (the headless-success exit previously had none), so
static destructors find nothing to do (wedge scenario 6f3b938). The two
runner.cpp bounded panic exits deliberately skip it (unbounded device-sync
hang risk) and now say so. Deleted the unused, unsafe-by-construction
cuda_alloc_tracker.
Gates: build -j4; 32/32 targeted gtests shuffle x3 (new taxonomy/sticky/
teardown/rollback/multi-stream/sync-debug/poisoned-context-subprocess
matrix; deterministic second-create rollback injection blocked — no seam
without new production surface, constituent guarantees covered); memcheck
0 errors; census discarded-status-macro-free-cuda 199→161 (core 38→26,
training 78→52; 4 Group B hot sites deferred to 6B by sign-off), baseline
regenerated; 7k bicycle A/B interleaved snapshots at parity (B median
34.6s vs A 37.6s under load); GUI validation clean (PLY+COLMAP load,
viewport resizes, ordered exit). Fable verify: SHIP (3 LOW folded in).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012S4jNQwL5VNi8stswXggoa
… the launch site CudaFailureSeed + CudaAwaitTicket + LFS_CUDA_LAUNCH_CHECK (one peek, no sync) + LFS_CUDA_AWAIT wrapping existing waits; breadcrumb sequence ranges with a thread-local watermark and a most-recent-failure latch; two throwing host-only bridges (slow-path handlers deliberately not noexcept). 28 launch checks + 1 AWAIT across fastgs/edge_compute/gsplat incl. both per-CDIM RasterizeToPixels hot launches (shared per-direction tags); stale Intersect.cpp post-hoc checks deleted. Group B trainer retypes: beginModelRead/endModelRead/background_for_step via LFS_CUDA_TRY (endModelRead's silent pending-bit drop now throws), submitLossReadback drain stays best-effort via LFS_CUDA_LOG_TEARDOWN; ~ViewerBorrowPublisher noexcept-dtor try/catch companion; computeCameraMetrics read-window primitives joined its degradation contract (verifier HIGH: throw escaping the metrics jthread). Census marker list gained LFS_CUDA_LAUNCH_CHECK; rasterization unchecked-kernel-launch 22->0, baseline regenerated in-commit. Gates: build -j4; 34/34 targeted gtests x3 shuffle seeds; memcheck clean; census green; GUI validation (plain x2 + bg-modulation) zero error noise; 7k mcmc+mrnf interleaved A/B parity under load. Fable verify: SHIP after the HIGH fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…zero All 191 census unchecked-kernel-launch hits under src/core converted to LFS_CUDA_LAUNCH_CHECK (kdtree_kmeans 21 dead-code sites included for census-zero; template/dispatch-multiplied families share one tag per kernel family incl. the 85x broadcast_binary path). One AWAIT conversion: build_grid's existing AABB readback sync now consumes a recorded range ticket. Bonus fixes per freeze sign-off: selection_ops' six release-noop assert checks (assert compiled out, sticky error consumed and discarded) replaced with real checks; both AABB memcpys wrapped with LFS_CUDA_CHECK_MSG (host-only LFS_CUDA_TRY is illegal in .cu — freeze Amendment 1). Lanczos uint8 branch got its in-window check (asymmetric-window lexical accident). New coverage: SelectionOpsCudaTest success-path regression; TensorLaunchCheckDeathTest proves the shared-tag check fires from a real broadcast_binary instantiation (threadsafe death-test style — the child needs a working CUDA context, which fork() cannot guarantee); standalone bench_tensor_launch_overhead tool (committed with its CMake target — amends the spec's untracked-bench pin, which would have left a dangling add_executable reference). Implemented via grok-4.5 (two packets, user-directed trial replacing codex); orchestrator-verified per-row, one death-test fork flake found and fixed. Gates: build -j4; 366/366 targeted gtests x3 shuffle; memcheck 111 tests 0 errors; census core unchecked 191->0, baseline regen in-commit; tensor bench ABBA parity (sort-middle-axis/broadcast/masking); 7k mcmc ABBA parity (mins within 0.1%). Fable verify: SHIP. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…daries at every export API All 75 census unchecked-kernel-launch hits under src/io and src/rendering converted (image_format 16, kmeans 39, rad 1, color_convert 5, selection 14); 8 AWAIT conversions wrap pre-existing waits (7 kmeans device syncs + rad's stream sync), net-new sync count zero. selection's local checkCudaLaunch helper (bare runtime_error, peek-only, no logging) deleted — 14 sites map 1:1 onto the typed macro; all SelectionService catches verified std::exception-wide. Typed-API companion boundaries so no expected/Result surface leaks exceptions: save_sog catches at its Result<void> boundary (context + owner-log + from_lfs_error); quantize_batch keeps its bool/CPU-fallback contract (catch -> warn once -> false); writeFrameGpu converts to its expected error shape for both encoder backends — without this the new checks would have turned an absorbed sticky error on the unfirewalled GUI export jthread into std::terminate. Raw export-jthread firewall gap ledgered for run_guarded consolidation. lfs_io_rad_quant_cuda gained the 2-line include/link it needed to compile the macros. Implemented via grok-4.5 (5 packets); spec freeze-audited by fable (caller enumeration completed pre-freeze), diff fable-verified SHIP: 75/8/8/14 counts reconciled, boundary walk clean, zero orphans. Gates: build -j4; 84/84 combined gtests x3 shuffle; memcheck 28 tests 0 errors; census io+rendering 0, gate green, baseline regen in-commit; 90s GUI validation zero error noise; 7k mcmc ABBA A/B parity (B faster on all statistics). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ow ZERO repo-wide
All 106 remaining census unchecked-kernel-launch hits converted (ssim/losses/
regularizers 49, mcmc/mrnf/densify/pruning/image 24, grad_alpha/bilateral/
ppisp/heatmap/sparsity 33) + 3 census-blind bonus sites in
ppisp_controller_pool. Depth-anchor readbacks: 2 AWAIT conversions inside the
function's soft-fail boundary (catch -> log -> free -> return {}), one ticket
per awaited range. Campaign arc for this rule: 394 -> 0.
Census scanner bug fixed at the root: mask_source treated C++14 digit
separators (100'000) as char-literal openers and silently blanked the rest of
the file for EVERY rule; guard now skips a quote flanked by hex digits (also
covers 0xAB'CD), with a self-test. Un-blanking reveals pre-existing debt,
disclosed in full and absorbed by the baseline regen in this commit:
expected-string +15 (checkpoint_format.hpp x4, parameters.hpp x2, sogs.cpp x7,
vulkan_context.cpp x2), vk-infinite-wait +4 (vulkan_context.cpp:931/971/1008/
1631 — Phase 7's wait scope is 16, not 12), discarded-status-macro-free-cuda
+3 (ppisp_controller_pool bare memcpys). Diff-introduced: +2 discarded-CUDA in
depth_loss.cu (best-effort cudaFreeAsync in the new soft-fail catch paths,
consistent with the file's idiom).
Implemented via grok-4.5 (3 packets; two runs reaper-killed post-edit,
verification tails run by orchestrator); spec fable-audited clean pre-freeze;
diff fable-verified SHIP (109/2/2 reconciled, boundary walk clean, u8-literal
edge documented as inert). Gates: build -j4; 36/36 x3 shuffle; memcheck 0;
census self-tests 33 OK; src/training unchecked 0 with fixed tool; 90s GUI
zero noise; dual-strategy 7k ABBA parity (mcmc AND mrnf minima favor the
checked build under bimodal machine load).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nally lands
Phase 7A per the doc's pinned contract, machine only (no call-site
conversions — those are 7B/7C): VulkanWaitPolicy (100ms slice / 2s LOG_WARN
stall heartbeat / 10s quarantine), WaitOutcome{Ready,Cancelled,Shutdown,
Quarantined}, SubmissionState with the pinned transition table (no
pre-submit-cancel replacement event; submit_accepted ordered before
publication), DeviceLost as a terminal typed Result error, never
host-signalled. VulkanDispatch PFN seam covering the full begin->submit path
+ injectable clock; gs_pipeline wired: T3 on rejected submit before
cancel+throw, T4 before the publication map writes, T5 after host
publication. New contract-violation guards throw typed lfs::Exception
(ContractViolation/Vulkan) — not the release-no-op _THROW_ERROR (56
pre-existing sites untouched, 7C's ledger).
QW-6 (deferred since the quick-wins batch): failed-then-successful
vkQueueSubmit proves no false timeline readiness and exactly one publication,
driven through the seam with fake handles, GPU-free. Wait matrix + full
transition-table unit tests, all fake-clock/fake-dispatch.
Verifier NO-SHIP->CONFIRMED-SHIP cycle: routing the two legacy infinite
waits through the dispatch made them census-blind (vk-infinite-wait 16->14
passed the ratchet silently — decreases are unchecked; Phase 11 ledger item);
reverted to literal calls, census restored to 16. Vestigial dispatch guards
removed. Orchestrator fixes folded: VK_TIMEOUT enumerator (VK_ERROR_TIMEOUT
does not exist), QW-6 post-retry assertion corrected to monotonic timeline
semantics.
Gates: build -j4; 43/43 unit+regression x3 shuffle seeds; census gate green
with all counts unchanged (16 waits still visible); 60s GUI zero noise; 7k
smoke clean. Implemented via grok-4.5 (foreground-leg dispatch), fable
freeze-audit restored the doc's 7A/7B/7C split pre-dispatch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ults get names Phase 6C per the pinned contract: interface + ONE reference kernel, the contract surface for the tensor-lib hardening campaign. Frozen five-field DeviceFaultRecord ABI (static_assert-armored 32-byte layout), device-side first-fault-wins CAS on failure branches only, per-stream registry with dedicated cudaMalloc slots (never pool memory; teardown drains BEFORE Tensor::shutdown_memory_pool), graph capture rejected as Unsupported at both host entries, DeviceTrap diagnostic mode gains its first consumer (host publishes one bool per launch; device never parses modes — §9 Ruling 1 formally amends the doc's diagnostic_mode.hpp packaging to the shipped Phase-0 reality). Reference kernel: index_select. Assert mode's O(n) pre-flight index D2H scan + host loop is REPLACED by a 32-byte fault-record readback that drains inline — preserving Assert's throw guarantee (the spec's deferred-drain option would have dropped unconsumed faults at the next slot reset) while Clamp/Wrap stay entirely sync-free and byte-identical. OOB now throws typed BoundsViolation carrying op_id/value/bound/thread_id. gather/scatter keep the full scan (untouched baselines). Implemented via grok-4.5 (3 edits-only packets under the post-crash swap-guarded -j2 builds); orchestrator fixes: vestigial template param (undeducible T), the inline Assert drain, fixture ratchet token. Fable freeze-audit clean (zero doc-answered ambiguities mislisted); verify SHIP. Gates: guarded build; 114/114 combined suites x3 shuffle; memcheck 0 (capture test excluded under sanitizer — instrumentation is illegal inside stream capture); racecheck 0 hazards on the CAS protocol; census totals byte-identical (increases AND decreases checked); bench ABBA parity +-0.3% all cases; 7k mcmc ABBA minima parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…to the 7A machine Phase 7B per the pinned contract: frame-fence, acquire, image-fence, current-slot, and both validation waits in VulkanContext converted to the bounded-wait machine (100ms slices, 2s LOG_WARN heartbeat, 10s quarantine, DeviceLost typed+terminal). kWaitForeverNs deleted with all its uses; the UI/frame thread now has no unbounded runtime Vulkan wait. Census vk-infinite-wait 16->10 (vulkan_context.cpp -> 0; gs_pipeline's two literal waits stay for 7C by design). WSI recovery byte-preserved: OUT_OF_DATE keyed on the native code (distinct from quarantine-Unavailable, proven by tests), soft-false sites clear last_error_ so the frame loop skips silently on Cancelled/Shutdown while Quarantined/DeviceLost surface. Along the way the out_suboptimal out-param (freeze ruling AMB-B1) fixes a pre-existing bug: the acquire SUBOPTIMAL bit was overwritten by the image-fence wait result whenever the image was in-flight, so the present-side recreate heuristic never saw it. New context_shutdown_started_ latch makes the Shutdown outcome live during teardown (no recursion into shutdown()). Implemented via grok-4.5 (one edits-only packet); spec fable-audited freeze-ready with zero blockers; diff fable-verified SHIP after a clang-format sweep. Gates: guarded -j2 build; 0 failures x3 shuffle; census movement exact with every other rule byte-identical both directions; 75s GUI validation zero noise (541 frame-timer samples); interleaved frame-time A/B dead parity (0.19ms medians both sides, 8 runs); fake-clock heartbeat/quarantine/outcome matrix is the hard gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…it and raw-vk-check both ZERO Phase 7C closes the Vulkan wait/error tier. All 23 remaining infinite waits (10 census + 13 census-blind numeric_limits::max()) across passes, ui_texture, rmlui, point-cloud, VkSplat, gs_pipeline, and mesh2splat are converted onto the 7A bounded-wait machine; gs_pipeline's two waits kept literal in 7A for census visibility now route through the dispatch. All 156 raw-vk-check sites (104 _THROW_ERROR + 52 LFS_VK_CHECK_MSG) become typed lfs::Exception / vk_try_bool via the rendering + visualizer typed helpers, and BOTH release-noop macros are deleted (LFS_VK_CONTEXT_CHECK_MSG and the CONTEXT helpers survive per freeze Ruling 2). Point-cloud's reset-before-submit lifecycle is wired onto SubmissionState with ResetPreWaitReplacement (the only site allowed to replace a fence — the REFUTED rule is enforced: gs_pipeline/VkSplat stay NoResetNoReplacement, a rejected submit never publishes or replaces). mesh2splat retains fence+CB on quarantine, destroying only after Ready or DeviceWaitIdle. Census exit: vk-infinite-wait 16->0, raw-vk-check 156->0. Disclosed decreases (baseline regenerated in-commit; this also absorbs 7B's missed regen): vk-infinite-wait 16->0, raw-vk-check 156->0, local-check-macro 13->12 (the LFS_VK_CHECK_MSG deletion — its name matched the CHECK rule). Every other rule byte-identical. Freeze Ruling 1 note: the point-cloud vkResetFences-FAILURE branches preserve the existing replaceFenceSignaled helper verbatim (destroy-and-recreate outside SubmissionState, lifecycle restarts at T0). It logs at LOG_ERROR, not the ruling's LOG_WARN — kept intentionally: a failed reset leaves the fence undefined (genuinely error-worthy), and forking shared point-cloud code to downgrade one log line would be a gratuitous edit in a Preserve phase. Implemented in five edits-only packets; spec fable-audited freeze-ready (two rulings); diff fable-verified PASS (no blockers). Gates: guarded -j2 builds after every packet; 0 failures x3 shuffle (QW-6 now asserts the typed lfs::Exception it receives); census double-zero with the decreases table; GUI validation zero noise; interleaved frame-time A/B parity (0.17ms medians both sides). Ledger: point-cloud destroy-drain frees cache buffers a retained fence may reference on the 10s-hung-device path (AMB-4 pin met; consistency fix deferred). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…Python Phase 8 P1 closes the exit criterion: errors surface natively even when the embedded Python is absent or failed. A process-singleton ErrorBus (in liblfs_core, one instance across the exe/Python-module boundary) with a noexcept non-blocking publish(); a GuiErrorConsumer that enqueues native RmlModalOverlay requests (enqueue-only on the publishing thread, RmlUi touched only on the UI frame); and an events->bus bridge that translates the ten failure events (training/dataset/config/export/video/mesh2splat/file-drop/ cuda-unavailable/cuda-version/disk-space) into typed ErrorNotifications and publishes them, registered in setupEventHandlers independent of Python. The Python notification bridge's failure handlers are subsumed (no double modal when Python is up); its success-training modal + Switch-to-Edit action and the native modal-render callback stay wired. Dedup by fingerprint^op-id over a 5s window (fresh op-id per event so distinct failures don't collapse, a repeated same-op fault does); fallback-only ErrorReporter logging (durable report only when no consumer delivered — no double-log). user_stopped Stop is suppressed (not a failure). StatusOnly is a silent no-op and Toast falls back to Modal in P1 (Toast/StatusOnly/Panel are P2). Public error_fingerprint() exposes the reporter's existing algorithm. 14 modal strings localized across all 10 locales; technical params stay English. Implemented by an opus executor from the frozen spec; fable-verified SHIP (M1 folded: should_suppress dropped noexcept so publish's catch-all catches a dedup-map bad_alloc instead of terminating). Gates: guarded -j2 build; 38/38 ErrorBus/bridge/fingerprint/reporter tests x3 shuffle; census ALL RULES byte-identical both directions; GUI validation END-TO-END with Python module hidden + CUDA forced unavailable — bridge->publish->native modal fired with zero Python involvement. Ledger for P2: thread the action op-id into on_invoke (M2); decide whether dedup counts surface / re-notify after sustained faults (M3); finer dataset-failure ErrorCode mapping is Phase 11. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… with counts Phase 8 P2 completes the native error surfaces. A new RmlToastOverlay (mirroring RmlModalOverlay's threading exactly: mutex enqueue from workers, all RmlUi work on the UI frame, non-interactive, 4-deep stack, 6s auto-dismiss, same-fingerprint collapse to "xN"); a thread-safe additive RmlStatusBar postStatusMessage + StatusMessageState for one-line status/cancel notices; and a details modal (monospace, scrollable, escaped format_for_developer in English) reachable from a "Details" button on every error modal. Dedup moves into ErrorDedup with a FIXED-from-last-delivery 5s window (kills P1's slide-forever bug) + a 60s idle sweep, and the suppressed-repeat count now crosses to the consumer via a new ErrorDeliveryInfo param on on_error — rendered as a dim "Repeated xN" line — while ErrorNotification stays byte-frozen. ErrorAction::on_invoke now takes the fresh OperationId (M2), and OpenLog is wired on the eight operation-failure modals (reveal the log file). Fixes a pinned-Cancelled violation the planner caught: cancelling a COLMAP or splat export emitted ExportFailed and popped an "Export Failed" MODAL; it now carries a cancelled flag → a quiet StatusOnly "Export cancelled" line. The ExportFailed field is wire-safe (TCP/MCP serialize explicit fields). Seven new strings localized across all 10 locales; technical params stay English. fable-planned + independently fable-audited (two risk items cleared: the signature change keeps ErrorNotification frozen; the shared-event field is wire-safe) + fable-verified SHIP. Gates: guarded -j1 build (one <cassert> fix); 33/33 headless suites x3 shuffle (ErrorBus/ErrorDedup/ToastStack/ StatusMessageState/GuiErrorConsumer/ErrorEventBridge); census ALL RULES byte-identical both directions. Windowed VISUAL check of the surfaces DEFERRED to a device-present session (display-risk) — markup/threading verified by reading. P3 ledger: force a toast rebuild after reloadResources (LOW-1); refresh the now-stale P1-era ErrorSurface comment; reserve(2) in the details-reenqueue test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…get typed and surfaced Phase 8 P3 completes the error architecture's Phase 8. A headless FrameStateMachine (13-row transition table, main-thread-only, no bus/clock/lock deps) drives handleFrameException: OOM reclaims within an 8-fault budget with a pressure Toast, then suspends scene rendering (holding the last-known-good frame so the GUI + modal stay interactive) with a Retry/OpenLog modal; consecutive internal faults suspend after 3 with a Retry/StopRenderer/OpenLog modal carrying the ORIGINAL structured error; DeviceLost is terminal. A typed public accessor VulkanContext::rendererTerminalState() over a new gpu_device_lost_ atomic (set at the 3 committed DeviceLost sites; the stall path structurally cannot set it) is the ONLY DeviceLost detection path — quarantine never throws, so onFrameCompleted polls it. Because a dead Vulkan context can never present another RmlUi frame, a DeviceLost modal is physically unrenderable in-app; the pinned "restart guidance" is realized faithfully via a blocking OS-native SDL_ShowSimpleMessageBox (return-checked, Wayland-safe) with the correlated LOG_ERROR as the durable record regardless. StopRenderer stops the viewport for the session (GUI/training/save keep running); Retry resumes + re-arms once. titleKeyFor gains Vulkan and op-gated render-frame cases. 8 new strings localized x10. P2 ledger folded: toast rebuild after reloadResources; refreshed ErrorSurface comment; test-vector reserve(2). fable-planned + fable-audited (keystone confirmed: no frame-path code throws DeviceLost today, poll is the sole detector) + fable-verified SHIP with three fixes folded (missing OOM-terminal LOG_ERROR; the internal modal now carries the structured error not a synthetic one; the new titleKeyFor Vulkan/Rendering mapping now has headless tests). Gates: guarded build; FrameStateMachine/ RendererTerminalState/GuiErrorConsumer suites x3 shuffle; census green (the frame-fault std::exception boundary annotated LFS-CENSUS-OK — routes to the state machine + bus, not a swallow). Windowed frame-time A/B, GUI suspension-corruption watch, and device-loss injection DEFERRED to a careful device-present session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…en grammar Replaces the refuted v1 prototype (footer authority, prev_index_offset, packed structs-on-disk all deleted). Superblock + A/B head slots + commit records; append writer with durability total order (payload flush -> head -> flush) and crash boundaries; generation-pinned strict reader, never-scan, bounds-before- allocation on all file-supplied sizes; per-block CRC validation on every read path incl. positional reads; epoch-bound clean-proof reuse; held-fd locking; compaction with verify-before-replace and opaque carry-forward; capability/ min_safe_writer gating; post-publish verification with published-tuple DataLoss contract on append. Spec amendment (pre-release, 1.0): 16-byte head-slot preview locator (spec S4.1) + THMB chunk so foreign tools thumbnail a .licht from fixed offsets -- proven by tools/licht_inspect/foreign_preview.py (159 lines, index-free). Gates: 16 golden fixtures byte-for-byte (15 prior unchanged); full conformance battery 142,544 cases; 10,023-case three-reader parity; real SIGKILL crash matrix at all publication boundaries; tmpfs ENOSPC incl. mid-chunk; census ratchet clean. Independently reviewed (adversarial spec-compliance pass, all majors fixed). Windows runtime column still owed on a Windows box (P0d).
…ayouts, fingerprint precedence - PROJ gains crs/world_origin(f64[3])/world_unit_scale/provenance; SCNG gains optional f64 georef_pose (point-cloud study R2: centralize offset is computed then discarded today; f32 ULP at geo magnitudes is 3cm-0.5m) - scene-centre exclusion row corrected: the removed world origin is not derivable from shifted payloads - new docs/licht_geometry_payloads.md: PCLD/MESH v1 self-describing property tables (RAD shape, no quantization, no deprecated padding) - REFS fingerprint component precedence: mtime fast-path only, xxh3 decisive - spec S14 numbering fix: preview locator is item 18, pre-release amendment
…MS over the container Chapter layer: typed retained-DOM accessors for PROJ/REFS/SCNG/PRMS with unknown-field survival inside UUID-addressed arrays; binary SELM (groups + UUID-keyed splat and point-cloud mask slices, delta+bitpack encoding chosen by measured bake-off, raw encoding still decodable); SPLT embeds LFSP verbatim (imported PLY/SPZ/SOG always embedded, live-RAD external and read-only); PCLD/MESH v1 self-describing property tables with hostile-descriptor preflight and unknown-property carry-forward. ProjectDocument: one-generation append with dirty-set clean-proof span reuse, opaque carry-forward, and strictly two-phase hydration -- Phase A stages and validates everything without touching the live scene, Phase B is a noexcept move-only commit; failure injection at every chapter proves the live session survives any hostile load byte-for-byte. Georeference block wired end-to-end: centralization offset and USD metersPerUnit are recorded in f64 and flow loader -> LoadResult -> PROJ -> reload. Embedded-payload provenance triples (locator + import fingerprint + xxh3-128 content hash, informational). REFS fingerprint precedence: mtime is a fast path, size/hash decisive. Reverse-owner reference index for relink UX. U3 camera in-memory masks refuse to save loudly rather than dropping data. Matrix-row round-trip proof: parses the ownership matrix at test time, 43 P3 rows proven incl. the full 90-field PRMS pending set, drift fails the build; P4/P5 rows explicitly deferred by phase tag. Battery extended with a geometry category (142,567 cases green). Undo history recorded as an explicit non-goal in plan §6 (owner decision): reopen restores exact state with a fresh stack.
Production snapshot capture: bounded safe-point pause (one clock incl. stream syncs), four 128 MiB pinned bands with a single non-temporal drain thread into owned pageable staging, per-process bandwidth calibration with a rig-scaled pause gate (bytes / measured_D2H x 1.12), snapshot-UUID consistency stamping, topology-change replan under the same UUID, background container append only after the optimizer resumes. CKPT embeds the LFKP stream byte-verbatim through bounded 8 MiB windows (no monolithic buffer); training nodes emit no SPLT row; PPIS covers non-checkpoint sessions. Lazy resume: display SplatData hydrates first, training blocked until moments/ADMM/PPISP/bilateral-grid/frozen-ranges adopt; untouched trainer payloads stay clean file-backed references on the next save. CLI: --save-project-at-iter/--save-project-path, --resume accepts .licht (headless). Matrix rows CKPT-126..149 + PPIS-157..160 proven. Measured (RTX 4090, 7k-iter smoke, mid-training save at 6501 + resume to 7001): pause p95 26.967 ms vs 27.185 ms rig gate, D2H 90% of calibrated raw, pinned peak exactly 512 MiB, memcheck/racecheck clean, battery 142,567 green. Known review debt carried to the next phase round: CPU chapters not yet re-captured inside the safe point; first-save prepare stalls the training thread ~100 ms outside the pause clock; regression-window fairness; GUI-mode --resume .licht silently ignored; CKPT-129 proof depth.
…t-consistency hardening Session chapters: GUIL as an abstract layout/area/space tree (today's shell is one versioned space; unknown space types retained opaquely; user-global theme/language/scale/HUD stripped on load and refused on save), VIEW (RenderSettings superset, two direct-matrix panel cameras, split state via SplitViewService::toggleMode, bookmarks, tool prefs, raster_backend canonical), EDTR (full multi-buffer editor session incl. flagged unsaved buffers and the embedded-secrets surface), SEQR (timeline inline, clips + REFS bindings, canonical controller FPS/speed), METR (bounds-checked binary loss/PSNR history; resume repopulates graphs). Restore is staged in Phase A and applied event-driven after the first GUI frame + panels-ready, with panels-ready now terminal in every config including plugin autoload disabled. Snapshot hardening: CPU chapters (SCNG/SELM/PRMS) captured inside the safe-point window under the snapshot UUID (0.14 ms in-window); service init moved off the first-save path (prepare stall 0.2-0.34 ms, cold path meets the rig gate); regression metric uses disclosed steady-state windows; non-headless --resume of a .licht is a typed error until the lifecycle phase; deep optimizer-moment fidelity in the matrix proof. All 40 P5 matrix rows proven (83 total with P3/P4); battery 142,567 green; GUI validation: clean launch + 5M-splat PLY load with zero error noise. Deferred to lifecycle phase: File Save/Open wiring, share-UI secrets warning.
…ites
File menu Save/Save As/Open/New + global Ctrl+S, .licht drag-drop, MRU as
{project_uuid, last_known_path}, restore-last-session and --project/--resume
routed through one project flow. Saves append dirty chapters only; explicit
GUI saves render a <=256 px THMB (autosave/close carries it forward,
hash-verified); share surfaces read the embedded-secrets flag. Open is
shell-first (<100 ms gate: 57 ms p50 on a 2.5 GB-checkpoint project, ~12 ms
live) with background hydration; the partial-open unit is chapter/node,
shown as unloaded, never written back empty (clean-span carry-forward while
hydrating). Save-on-close runs in allowclose() before shutdown, and File ->
Exit enters the same close-save machine — only an explicit Discard bypasses
it; close prompt offers Save/Save As/Discard/Cancel; a failed close-save
re-arms. Dirty-project preflight is centralized under every open path incl.
drag-drop and MCP. Open-over-open and New-over-dirty are transactional: the
live project survives any failed open (corrupt file rejected in 10 ms).
Opening clears undo history (plan §6) with fail-closed replay tests.
One-format switchover: headless training now emits project.licht; the
checkpoint.resume, .ppisp and layout.json writers are deleted, their readers
remain as importers. MCP gains project_save/save_as/open/get_info with
bounded structured errors.
GUI-validated end to end (screenshots + MCP transcript): load 5M-splat PLY,
edit layout/camera/selection, save, close, relaunch -> identical session,
zero error noise. Known open diagnostic: SCNG JSON build (~194 cameras)
inside the snapshot pause costs ~420 ms at scale — fix lands next phase.
…cycle, JobRegistry, sub-10ms safe-point capture
… error during cleanup
…lease corpus, published grammar register
…ssion capture, trim geometry generality
…tests without the full suite
…rget wiring edge cases
Logger uses its own export macro; without it the target's TUs see a dllimport Logger and the header-inline should_emit becomes an unresolved import in the Debug link (LNK2019 via parameters.cpp log_internal).
…; platform-split format-test composition MSVC resolves every object reference before /OPT:REF while GNU ld gc-sections discards unreferenced functions first, so the extended chapter coverage cannot link without the CUDA core on Windows. Windows keeps the container/chapter/DOM surface; Linux keeps the full extended suite plus the authority gates.
…C cell; self-containment uses the active interpreter Containers without user namespaces cannot mount the isolated tmpfs; the cell stays required wherever the unshare probe succeeds, and any other skip or failure remains fatal. The nested compatibility check no longer assumes uv exists on PATH.
compact() kept a second shared_ptr owner of the temp NativeFile alive across ReplaceFileW, which opens the replacement with no sharing mode — deterministic ERROR_SHARING_VIOLATION. Transfer ownership instead so commit()'s reset closes the only handle.
Replace dev-release/measure-release with two presets whose binary dirs are build/ and debug/, matching the directories the project already uses. The tracked presets stay machine-neutral: no CUDA or vcpkg paths, and no forced BUILD_TESTS or ENABLE_COMPILER_CACHE, so existing caches keep their values and fresh clones get the CMakeLists defaults. Add a testPresets entry for release so ctest can be driven by preset. Update the build docs accordingly. The measurement workflow no longer has a dedicated preset; it configures a throwaway build-measure tree with ENABLE_COMPILER_CACHE=OFF instead.
Summarise lfs::core::Tensor as dtype, shape, device, element and byte counts plus the data pointer, and expand strides/dtype/device as readable children so output does not depend on the libstdc++ printers being auto-loaded. Add an lfs-tensor command to dump element values. Device memory is read through cuda-gdb's @global address-space qualifier: an unqualified read of a device pointer appears to succeed and returns zeros in both gdb and cuda-gdb, so under plain gdb the command refuses instead of reporting wrong values. verify_pretty_printers.sh builds a -g fixture against the configured build tree and asserts the printer output under both debuggers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Today a piece of work in LichtFeld Studio is scattered across a directory: a checkpoint here, a
.ppispsidecar there, a layout file somewhere else, and a pile of exports namedsplat_30000 (1).ply. Close the app mid-session and you get most of it back, not all of it.This branch introduces
.licht— one project file that holds the whole session. Scene graph,selection, parameters, training checkpoint, GUI layout, viewport and camera state, editor
buffers, sequencer timeline, metrics history. Open it tomorrow on another machine and you are
where you left off, including a training run you can resume. It is the same idea as a
.blendor a
.psd: the project is a thing you own, not a folder you hope stays intact..lichtis now the only format the app writes for projects. Legacy inputs(
checkpoint.resume,.ppisp,layout.json) can still be read, so existing work openswithout a manual conversion step — but that import path is transitional, not permanent. It will
be dropped in a future release, and that will be an announced breaking change. The forever
promise applies to
.lichtitself: a.lichtwritten today stays readable by later versions.Interop exports (PLY / SPZ / SOG /
.rad/ USD / HTML) are unchanged one-way bakes.The shape of the file
Moving a camera and pressing save writes a few kilobytes. The 9 GB checkpoint is not copied
again — generation 2's table of contents simply points back at the copy generation 1 already
wrote.
What a save actually does
Pull the plug at any point and you get one of exactly two outcomes:
There is no in-between state, because the only thing that switches a file from old to new is a
single 4 KB block that is either checksum-valid or not.
Where the other files fit
How the format is built
The figures above cover the two load-bearing ideas — append-only writes and the two
checksum-guarded head slots, which together are why an interrupted save cannot corrupt a project,
and why publication does not depend on the filesystem writing 4 KB atomically (it does not).
The rest:
Content is chapters. Each kind of state is its own chunk with its own version:
PROJ,REFS,SCNG(scene),SELM(selection),SPLT/PCLD/MESH(geometry),PRMS(parameters),CKPT(training checkpoint), and the session chapters
GUIL/VIEW/EDTR/SEQR/METR. Saving onlyrewrites chapters that actually changed; a 10 GB checkpoint is not touched when you move a camera.
Big things stay outside; identity is content, not path. Datasets and live
.radstreams arereferenced, never embedded — with a fingerprint (size plus hashes of the first and last 64 KB) so a
moved or edited file is detected honestly instead of silently mis-binding. Imported splats
(PLY/SPZ/SOG) are embedded, so a project you hand to someone else is self-contained.
Saving during training pauses the optimizer for about a millisecond. The safe point copies plain
values and issues the GPU→host transfer, nothing else — JSON and DOM building happen after training
resumes. On a 200-camera, 1.3M-splat scene the in-window capture measures ~1.5 ms, against a budget
of 10 ms. Everything in that snapshot carries one snapshot ID, so the checkpoint and the scene it
describes can never be from different moments.
Autosave is a separate disposable file.
project.licht.autosaveis bound to the exact commit ofits master file, so it can never be applied to the wrong project or the wrong generation. If it does
not match, it is ignored and deleted. If it does match, you are offered recovery on open (or
--recoverwhen headless). Recovered work merges into the master on your next real save. The masteris never modified to make autosave work.
Compaction reclaims dead space. Because saving appends, files grow. Compaction builds a complete
new file, verifies it, and only then swaps it in — chunks it does not understand are carried across
byte-for-byte, so a file written by a newer build survives compaction by an older one.
Forward and backward compatibility are declared, not hoped for. Every file states the minimum
reader version and the capability bits it needs. An older app opens a newer file read-only instead of
writing something lossy. Unknown chunks and unknown JSON fields are preserved through saves.
A thumbnail lives at a fixed offset. A PNG preview sits at a known place in the header, so a file
manager or an asset browser can show a project preview with ~160 lines of code and no knowledge of the
container. That contract is documented and covered by a test written against the spec alone.
What is in the branch
Phases P0–P8: the byte grammar and its invariants, node UUID identity across scene/selection/
sequencer/undo, the container reader and writer, the chapters, the training snapshot service,
the session chapters, project lifecycle (Save/Save As/MRU/drag-drop/save-on-close/restore-last-
session), autosave and recovery and compaction, and finally compatibility hardening.
Also included: a reference parser and an independent second parser in
tools/licht_inspect/, afrozen copy of the 1.0 parser pinned by content hash (so "an old reader can still open this" is a
test, not a promise), and a release corpus of real files emitted by the shipping writer with locked
checksums.
Testing
randomized inputs, append lifecycles, 64-bit size edges, cross-parser agreement).
SIGKILLat every publication boundary: save, autosave, compaction —every survivor is either the old file intact or the complete new one, never a mix.
steady disk use stays at master + one autosave, peak stays inside master + two + 2 MB.
never a second writer.
file byte-identical.
and File→Exit exercised live.
gates plus the full conformance battery inside the build container; the Windows lanes build
and run the CPU-only container and chapter suites on MSVC, Debug and Release.
Known gaps
build and run on Windows CI in both configurations. Their first run immediately caught a real
bug: compaction's atomic replacement failed with a sharing violation because a second handle to
the compacted temporary survived into
ReplaceFileW— fixed in this branch. Still owed onnative Windows: the killpoint and disk-quota cells listed in
docs/compatibility.md, whichneed dedicated prototypes rather than the CI build.
release tag naming the manifest root and parser hash) are written down in
docs/compatibility.md.empty undo stack.
Reading order for reviewers
PROJECT_FORMAT_PLAN.md— decisions and why, including the ones that were rejected.docs/licht_format_spec.md— the byte grammar and state machines.docs/licht_ownership_matrix.md— which chapter owns which field, one authority per field.src/io/project/— container, writer, chapters, recovery.