feat(sim): swarm-test per-seed generator configurations - #1307
Merged
Conversation
`cargo clippy --all-targets --locked -- -D warnings` on the pinned 1.93.1 toolchain fails on main with mechanical, pre-existing lints (manual_checked_ops, for_kv_map, sort-by-Reverse, Option::filter, a collapsible match guard, a needless borrow), and the 1.97 clippy adds three unnecessary_cast (u64 -> u64) in sozu-top under --all-features — there is no CI clippy gate to catch the drift. Every rewrite is semantics-preserving: checked_div replaces the divisor>0 guards with identical skip behavior, the map .values()/.values_mut() forms iterate identically, and the QueryCertificatesFromWorkers match guard keeps the no-fingerprint fall-through to notify_proxys. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
Implement swarm testing (Groce et al., ISSTA 2012) in both deterministic simulations: each seed draws a SwarmConfig from its seeded RNG BEFORE the first operation — a random subset of the OPTIONAL/SUPPRESSOR grammar features at 50% inclusion each, weights renormalized over the survivors, MANDATORY features always retained. One seed in four keeps the inclusive all-features configuration (the paper's C_D, which swarm complements but never replaces); an all-off draw collapses back to it. udp_simulation.rs: the weighted Action grammar becomes ACTION_TABLE in the exact historical dispatch order, so an all-features configuration is draw-identical to the pre-swarm harness. ClientDatagram is MANDATORY (sole flow creator); Drain, CloseAll, AbortFlow, and SetMaxFlows are SUPPRESSORS (each repairs or prevents the full-table state a capacity bug needs); the rest are OPTIONAL. Each buggify arm is gated by its sibling feature and skipped — never redrawn — when that feature is off. The single-seed replay path and pinned tests now set_iterations(1), matching the TCP sim's replay contract: without it moonpool's UntilCoverageStable default kept drawing up to ~1000 fresh random seeds after the requested one. tcp_preread_sim.rs: the 25 scenario generators become GENERATOR_TABLE (all OPTIONAL — SniPrereadCore is fresh per connection, so nothing can suppress across connections; the swarm benefit is passive competition), plus the one genuinely orthogonal fragmentation delivery axis (off → one-shot delivery only, drip generator excluded). The per-class coverage gate stays asserted on the MERGED campaign tally, unchanged. The drawn configuration is a pure function of the seed, printed as one canonical swarm-config line per seed (byte-identical across replays — the new *_swarm_config_is_stable_across_draws tests assert it), and SOZU_SIM_SWARM=0 pins every seed to the historical all-features grammar with zero extra RNG consumption, so a swarm campaign and an inclusive campaign of identical seed count can be compared directly. The pinned *_replays_known_seed smoke tests run with swarm off for the same reason. doc/testing.md gains the swarm doctrine section (feature definition, classification, draw, campaign composition, replay contract, gate placement); doc/udp_simulation.md documents the knob and the buggify gating. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements swarm testing (Groce et al., ISSTA 2012) in both deterministic simulations of the
sozu-simcrate: instead of every seed running the single all-features workload grammar, each seed now draws a random subset of the generator features from its own seeded RNG before the first operation.What changes
sim/tests/udp_simulation.rs— the weightedActiongrammar becomesACTION_TABLE(exact historical dispatch order, so an all-features configuration is draw-identical to the pre-swarm harness). Per-seedSwarmConfig:ClientDatagramis MANDATORY (sole flow creator),Drain/CloseAll/AbortFlow/SetMaxFlowsare SUPPRESSORS (each repairs or prevents the very full-table state a capacity bug needs), the rest OPTIONAL at 50% inclusion with weights renormalized. Each buggify arm is gated by its sibling feature (skipped, never redrawn). The single-seed replay path and pinned tests nowset_iterations(1)— previously moonpool'sUntilCoverageStabledefault kept drawing up to ~1000 fresh random seeds after the requested one, soSOZU_UDP_SIM_SEEDdid not actually replay "that ONE seed".sim/tests/tcp_preread_sim.rs— the 25 scenario generators becomeGENERATOR_TABLE, all OPTIONAL (a freshSniPrereadCoreper connection means nothing can suppress across connections; the swarm benefit is the paper's passive competition), plus the one genuinely orthogonalfragmentationdelivery axis (off → one-shot delivery only, forced-drip generator excluded). TheCoverageTally::assert_full_coveragegate stays asserted on the MERGED campaign tally, unchanged and unweakened.kfeatures together appears in a coin-toss subset with probability1/2^k); an all-off draw collapses back to it.SOZU_SIM_SWARM=0pins every seed to the historical grammar with zero extra RNG consumption, so swarm and inclusive campaigns of identical seed count compare directly.swarm-configline per seed before the workload runs; new*_swarm_config_is_stable_across_drawstests assert two draws of the same seed are identical. ReplayingSOZU_UDP_SIM_SEED=0xdeadbeeftwice yields a byte-identical configuration line (verified bydiff).doc/testing.mdgains the swarm doctrine section (feature definition, MANDATORY/OPTIONAL/SUPPRESSOR classification, campaign composition, replay contract, gate placement);doc/udp_simulation.mddocuments the knob and buggify gating.chore(clippy): mechanical, semantics-preserving fixes for pre-existing lints that failcargo clippy --all-targets --locked -- -D warningson the pinned 1.93.1 toolchain onmain(manual_checked_ops,for_kv_map, sort-by-Reverse,Option::filter, a collapsible match guard, a needless borrow), plus threeunnecessary_cast(u64 as u64) insozu-topthat the 1.97 clippy flags under--all-features. No behavior change; there is no CI clippy gate, which is how the drift landed.Protocol / security impact
None at runtime. The feature commit touches only the test-only
sozu-simharness and docs. The lint commit toucheslib/command/binwith behavior-preserving rewrites only (checked_divkeeps the identical divisor-zero skip; map.values()iterates identically; theQueryCertificatesFromWorkersmatch guard keeps the no-fingerprint fall-through tonotify_proxys).Verification (all exit 0, pinned 1.93.1 toolchain)
The 256-seed swarm campaign passes with the campaign-level coverage gate asserting every class:
No invariant was tripped by any swarm subset across the sweeps run.