Skip to content

Layer 1: scenarios-as-modules (Cl2Benchmark archetype + Kata demo + cycle gate)#1230

Draft
Bickor wants to merge 3 commits into
Azure:v2from
Bickor:v2-modular-scenarios
Draft

Layer 1: scenarios-as-modules (Cl2Benchmark archetype + Kata demo + cycle gate)#1230
Bickor wants to merge 3 commits into
Azure:v2from
Bickor:v2-modular-scenarios

Conversation

@Bickor

@Bickor Bickor commented Jun 19, 2026

Copy link
Copy Markdown

Summary

Draft for review. Introduces Layer 1 of the modularization effort: authoring a whole Telescope test (a scenario) as a reusable KCL module, so scenario owners can stand up or tweak a benchmark with a couple of changes instead of hand-writing the full pipeline.

Three self-contained commits:

  1. Cl2Benchmark scenario archetype (2100b44)

    • Adds kcl/lib/scenario/cl2_benchmark.k: a schema + builder that captures the shared CL2 pipeline spine (resource group → cluster → node pool → CreateKwokNodes → RunClusterLoader2 → collect/upload → cleanup).
    • Proven non-regressive: kcl/example_pipeline/pipeline.k rebuilt on the archetype regenerates byte-identical pipeline.yaml vs. the prior hand-written output.
  2. Kata benchmark scenario (cc9b50a)

    • Demonstrates the archetype's reuse: a new scenario enabled via two touch-points only —
      • nodePool.runtime = "kata"--workload-runtime KataMshvAccVmIsolation + nested-virt SKU, and
      • CL2_RUNTIME_CLASS_NAME in the cl2-override ConfigMap → measured pods get runtimeClassName.
    • Adds kcl/lib/scenario/cl2_manifest.k (typed Cl2Override/Cl2Manifest + buildCl2Manifest) so a scenario can generate its own cl2.yaml from KCL (kcl run -S manifests). RunClusterLoader2 is unchanged; cl2Manifest stays a path.
  3. Cross-scenario reference cycle validation gate (0ab8c87)

    • scripts/validate_scenarios.py: a thin /generate_yaml gate catching path-string cross-scenario reference cycles (one scenario's cl2Manifest/kwokNodeManifest pointing into another scenario's directory) — the one blind spot KCL's compiler cannot see.
    • KCL already rejects circular imports (E1001 RecursiveLoad); this gate deliberately does not re-check those. format_kcl_cycle_error() just reframes that native diagnostic in scenario terms.
    • scripts/test_validate_scenarios.py: 21 unittest cases (GREEN); documented in the generate-yaml skill.

What to review

  • Archetype boundary: is Cl2Benchmark's field set the right seam, and is the path-string cl2Manifest contract acceptable vs. inlining?
  • Kata enablement ergonomics (the two touch-points).
  • Scope of the validation gate (path-string cycles only; leaning on KCL for import cycles).

Not in this PR (deliberately deferred)

  • Layer 2 (named, tool-agnostic metric catalog + the missing CollectClusterLoader2Result step).
  • Open decisions: escape-hatch boundary for Karpenter/private/cilium, catalog mechanism, Kata SKU/runtime defaults, emit naming alignment.

Validation

$ python3 scripts/test_validate_scenarios.py   # 21 tests, OK
$ python3 scripts/validate_scenarios.py         # OK (no cross-scenario reference cycles)
$ kcl run kcl/example_pipeline/pipeline.k -S output -o /tmp/out.yaml  # byte-identical to committed golden

Bickor added 3 commits June 15, 2026 14:50
Introduce lib/scenario/cl2_benchmark.k: a Cl2Benchmark schema plus build
lambda that composes existing step modules into a full ap.Pipeline from a
compact, author-facing config. Re-express example_pipeline via the archetype
(167 -> 76 lines) with byte-identical generated YAML (verified against a
regenerated golden oracle, 10413 bytes).
Demonstrates whole-test modularization: enabling Kata containers requires
only two touch-points on top of the shared scenario spine.

- NodePool.runtime adds --workload-runtime KataMshvAccVmIsolation flag
- lib/scenario/cl2_manifest.k builds the CL2 driver manifest from KCL,
  emitting CL2_RUNTIME_CLASS_NAME so measured pods get runtimeClassName
- kata_benchmark scenario reuses Cl2Benchmark, differing only in Kata
  fields (nested-virt SKU + runtime class)

Verified: example_pipeline regen stays byte-identical to golden; kata
pipeline diff is Kata-only.
Add a thin /generate_yaml gate that catches path-string cross-scenario
reference cycles (cl2Manifest / kwokNodeManifest pointing into another
scenario's directory) - the blind spot KCL's own import-cycle check
cannot see. KCL natively rejects circular imports (E1001 RecursiveLoad),
so this gate does not re-check those; format_kcl_cycle_error() instead
reframes that diagnostic in scenario terms.

- scripts/validate_scenarios.py: pure detection fns + main() (argparse,
  --kcl-root) + format_kcl_cycle_error()
- scripts/test_validate_scenarios.py: 21 unittest cases (GREEN)
- generate-yaml skill: document the validation gate step
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant