Skip to content

feat(sim): swarm-test per-seed generator configurations - #1307

Merged
FlorentinDUBOIS merged 3 commits into
mainfrom
feat/sim-swarm
Aug 10, 2026
Merged

feat(sim): swarm-test per-seed generator configurations#1307
FlorentinDUBOIS merged 3 commits into
mainfrom
feat/sim-swarm

Conversation

@FlorentinDUBOIS

Copy link
Copy Markdown
Collaborator

Implements swarm testing (Groce et al., ISSTA 2012) in both deterministic simulations of the sozu-sim crate: instead of every seed running the single all-features workload grammar, each seed now draws a random subset of the generator features from its own seeded RNG before the first operation.

What changes

  • sim/tests/udp_simulation.rs — the weighted Action grammar becomes ACTION_TABLE (exact historical dispatch order, so an all-features configuration is draw-identical to the pre-swarm harness). Per-seed SwarmConfig: ClientDatagram is MANDATORY (sole flow creator), Drain/CloseAll/AbortFlow/SetMaxFlows are SUPPRESSORS (each repairs or prevents the very full-table state a capacity bug needs), the rest OPTIONAL at 50% inclusion with weights renormalized. Each buggify arm is gated by its sibling feature (skipped, never redrawn). The single-seed replay path and pinned tests now set_iterations(1) — previously moonpool's UntilCoverageStable default kept drawing up to ~1000 fresh random seeds after the requested one, so SOZU_UDP_SIM_SEED did not actually replay "that ONE seed".
  • sim/tests/tcp_preread_sim.rs — the 25 scenario generators become GENERATOR_TABLE, all OPTIONAL (a fresh SniPrereadCore per connection means nothing can suppress across connections; the swarm benefit is the paper's passive competition), plus the one genuinely orthogonal fragmentation delivery axis (off → one-shot delivery only, forced-drip generator excluded). The CoverageTally::assert_full_coverage gate stays asserted on the MERGED campaign tally, unchanged and unweakened.
  • Campaign composition — one seed in four keeps the inclusive all-features configuration (the paper is explicit swarm complements, never replaces it: a bug needing k features together appears in a coin-toss subset with probability 1/2^k); an all-off draw collapses back to it. SOZU_SIM_SWARM=0 pins every seed to the historical grammar with zero extra RNG consumption, so swarm and inclusive campaigns of identical seed count compare directly.
  • Replay contract — the drawn configuration is a pure function of the seed and prints as one canonical swarm-config line per seed before the workload runs; new *_swarm_config_is_stable_across_draws tests assert two draws of the same seed are identical. Replaying SOZU_UDP_SIM_SEED=0xdeadbeef twice yields a byte-identical configuration line (verified by diff).
  • Docsdoc/testing.md gains the swarm doctrine section (feature definition, MANDATORY/OPTIONAL/SUPPRESSOR classification, campaign composition, replay contract, gate placement); doc/udp_simulation.md documents the knob and buggify gating.
  • Preliminary commitchore(clippy): mechanical, semantics-preserving fixes for pre-existing lints that fail cargo clippy --all-targets --locked -- -D warnings on the pinned 1.93.1 toolchain on main (manual_checked_ops, for_kv_map, sort-by-Reverse, Option::filter, a collapsible match guard, a needless borrow), plus three unnecessary_cast (u64 as u64) in sozu-top that the 1.97 clippy flags under --all-features. No behavior change; there is no CI clippy gate, which is how the drift landed.

Protocol / security impact

None at runtime. The feature commit touches only the test-only sozu-sim harness and docs. The lint commit touches lib/command/bin with behavior-preserving rewrites only (checked_div keeps the identical divisor-zero skip; map .values() iterates identically; the QueryCertificatesFromWorkers match guard keeps the no-fingerprint fall-through to notify_proxys).

Verification (all exit 0, pinned 1.93.1 toolchain)

cargo +nightly fmt --all -- --check
cargo clippy --all-targets --locked -- -D warnings
cargo test --workspace --locked          # sozu-sim builds as an empty 0-test binary — tokio_unstable scoping intact
RUSTFLAGS="--cfg tokio_unstable" cargo test -p sozu-sim
RUSTFLAGS="--cfg tokio_unstable" SOZU_UDP_SIM_SEEDS=256 cargo test -p sozu-sim --test udp_simulation
RUSTFLAGS="--cfg tokio_unstable" SOZU_TCP_PREREAD_SIM_SEEDS=256 cargo test -p sozu-sim --test tcp_preread_sim
RUSTFLAGS="--cfg tokio_unstable" cargo clippy -p sozu-sim --all-targets --locked -- -D warnings

The 256-seed swarm campaign passes with the campaign-level coverage gate asserting every class:

tally=CoverageTally { accepted: 6383, not_tls: 1009, malformed_record: 566, malformed_handshake: 374,
fragmented: 592, too_large: 342, no_sni: 525, ech_outer_absent: 490, sni_unmatched: 812,
alpn_unmatched: 576, proxy_header_invalid: 301, front_closed: 318, fragmented_delivery: 3997,
multi_record: 811, complete_over_cap: 408 }

No invariant was tripped by any swarm subset across the sweeps run.

`cargo clippy --all-targets --locked -- -D warnings` on the pinned 1.93.1
toolchain fails on main with mechanical, pre-existing lints
(manual_checked_ops, for_kv_map, sort-by-Reverse, Option::filter, a
collapsible match guard, a needless borrow), and the 1.97 clippy adds
three unnecessary_cast (u64 -> u64) in sozu-top under --all-features —
there is no CI clippy gate to catch the drift. Every rewrite is
semantics-preserving: checked_div replaces the divisor>0 guards with
identical skip behavior, the map .values()/.values_mut() forms iterate
identically, and the QueryCertificatesFromWorkers match guard keeps the
no-fingerprint fall-through to notify_proxys.

Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
Implement swarm testing (Groce et al., ISSTA 2012) in both deterministic
simulations: each seed draws a SwarmConfig from its seeded RNG BEFORE the
first operation — a random subset of the OPTIONAL/SUPPRESSOR grammar
features at 50% inclusion each, weights renormalized over the survivors,
MANDATORY features always retained. One seed in four keeps the inclusive
all-features configuration (the paper's C_D, which swarm complements but
never replaces); an all-off draw collapses back to it.

udp_simulation.rs: the weighted Action grammar becomes ACTION_TABLE in
the exact historical dispatch order, so an all-features configuration is
draw-identical to the pre-swarm harness. ClientDatagram is MANDATORY
(sole flow creator); Drain, CloseAll, AbortFlow, and SetMaxFlows are
SUPPRESSORS (each repairs or prevents the full-table state a capacity
bug needs); the rest are OPTIONAL. Each buggify arm is gated by its
sibling feature and skipped — never redrawn — when that feature is off.
The single-seed replay path and pinned tests now set_iterations(1),
matching the TCP sim's replay contract: without it moonpool's
UntilCoverageStable default kept drawing up to ~1000 fresh random seeds
after the requested one.

tcp_preread_sim.rs: the 25 scenario generators become GENERATOR_TABLE
(all OPTIONAL — SniPrereadCore is fresh per connection, so nothing can
suppress across connections; the swarm benefit is passive competition),
plus the one genuinely orthogonal fragmentation delivery axis (off →
one-shot delivery only, drip generator excluded). The per-class coverage
gate stays asserted on the MERGED campaign tally, unchanged.

The drawn configuration is a pure function of the seed, printed as one
canonical swarm-config line per seed (byte-identical across replays —
the new *_swarm_config_is_stable_across_draws tests assert it), and
SOZU_SIM_SWARM=0 pins every seed to the historical all-features grammar
with zero extra RNG consumption, so a swarm campaign and an inclusive
campaign of identical seed count can be compared directly. The pinned
*_replays_known_seed smoke tests run with swarm off for the same reason.

doc/testing.md gains the swarm doctrine section (feature definition,
classification, draw, campaign composition, replay contract, gate
placement); doc/udp_simulation.md documents the knob and the buggify
gating.

Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
@FlorentinDUBOIS
FlorentinDUBOIS merged commit cca1710 into main Aug 10, 2026
23 checks passed
@FlorentinDUBOIS
FlorentinDUBOIS deleted the feat/sim-swarm branch August 10, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant