development plans

[alexlib]

Description

To maximize the computational efficiency and robustness of OpenPTV2, we propose a development plan targeting key bottlenecks in our calibration, segmentation, and tracking pipelines.

By explicitly bypassing high-overhead, voxel-based Iterative Particle Reconstruction (IPR), we focus our efforts on high-utility, lightweight improvements inspired by the MyPTV framework.

Below is the complete proposed development plan. We welcome feedback from the maintainers and the community on these potential integration paths.

---

OpenPTV2 Development Plan: Speed & Robustness Modernization

1. Objectives & Utility Metrics

• Computational Speedup (FPS): Minimize CPU cycles and memory usage during 3D reconstruction.
• Trajectory Length & Yield: Maximize uninterrupted, reconstructed particle tracks.
• Data Fidelity: Minimize "ghost" tracks without sacrificing sparse particle detection.

---

2. Speed Optimization: Analytical & Numerical Inversion

2.1. Analytical Inversion of the Soloff Calibration Model

• The Problem: Polynomial camera models (such as Soloff) are analytically non-invertible, requiring expensive iterative numerical triangulation during 3D coordinate estimation.
• Proposed Implementation: Integrate the analytical inversion of the Soloff polynomial (recently validated by MyPTV/OpenPTV collaborators). This performs 2D-to-3D projection in a single algebraic pass.
• Benefit: Estimated $3\times$ to $5\times$ speedup during the 3D projection phase.

2.2. Extended Soloff Calibration (No Initial Guess)

• The Problem: Traditional calibration (Tsai model) requires manual Euler angle and physical camera coordinate initial guesses. Incorrect parameters lead to setup divergence.
• Proposed Implementation: Implement a direct linear algebraic initializer for the Soloff calibration coordinates from calibration target files.
• Benefit: Reduces human configuration errors and decreases setup time from hours to minutes.

---

3. Robustness Optimization: Track-Guided Matching

3.1. "Particle Marching" Time-Aware Stereo Matching

• The Problem: Traditional frame-by-frame stereo matching checks all light-ray intersections at each frame, leading to an $O(N^2)$ combinatorial expansion and creating "ghost particles."
• Proposed Implementation: Adopt MyPTV's Time-Aware Stereo Matching (Particle Marching). Use active trajectories from frame $t$ to guide coordinate correspondences in frame $t+1$. Establish matches for ongoing tracks first and extract them from the candidate pool before general matching.
• Benefit: Reduces candidate matching combinations by up to $80%$ and significantly reduces ghost track generation.

3.2. Automated Dynamic Background Subtraction

• The Problem: Glare, optical reflections, and static noise are misidentified as particles during 2D segmentation.
• Proposed Implementation: Integrate temporal sliding-minimum subtraction inside the pre-processing loop to clean raw frames before centroid extraction.
• Benefit: Improves centroid location accuracy below $0.1\text{ px}$ and prevents noise-induced ghost tracks.

---

4. Trajectory Post-Processing & Data Conditioning

4.1. Track Repair & Stitching

• The Problem: Temporary optical occlusion breaks continuous trajectories into short, fragmented tracklets.
• Proposed Implementation: Implement a post-processing module that matches exit velocity vectors of terminated tracks with entry vectors of newly initiated tracks within a spatio-temporal radius.
• Benefit: Yields longer, unbroken trajectory datasets for Lagrangian statistics.

4.2. Backtracking Extension

• The Problem: Forward tracking often misses early entry frames of particles due to initial frame constraints.
• Proposed Implementation: Run a backward tracking sweep starting from stable, established tracks to recover earlier entry points.
• Benefit: Recovers up to $15%$ more trajectory frames.

---

5. Phased Implementation Roadmap