PerProgramAutoTune

Standalone per-program compiler auto-tuning baselines for comparing against RFunipassLab.

This project intentionally does not modify any file in /root/exp/RFunipassLab. It reads the same external LLVM IR files, the same pass-pair CSV, and uses the same LLVM21 toolchain by default.

Default Controlled Setup

• LLVM tools: /root/llvm/llvm-project-21/build/bin
• Autophase: /root/exp/RFunipass/lib/libAutophase_21_1_8.so
• IR root: /root/exp/RFunipassLab/external_datasets/ir
• External manifest: /root/exp/RFunipassLab/external_datasets/manifests/external_ir_manifest.json
• Pass-pair CSV: /root/exp/RFunipassLab/data/Step3_EnumeratedPairs.csv
• Objective: instruction count after applying the candidate pass sequence
• Normalization baseline: -Oz
• Loop policy: wrap, matching the current recommended RFunipassLab setting
• Autophase mode: external /usr/bin/python3 worker by default, to avoid conda libstdc++ conflicts while still using the same Autophase .so

Implemented Per-Program Methods

Each method tunes every selected benchmark independently and returns one best pass sequence per benchmark.

• rio_random: RIO-style random search over pass enable bit-vectors.
• ga_bitvector: GA over pass enable bit-vectors.
• boca_flat: BOCA-style RF + expected improvement over pass enable bit-vectors.
• cfsat_random: CFSAT Phase4/search-only random walk over a synergy graph.
• cfsat_greedy: CFSAT Phase4/search-only greedy extension over a synergy graph.
• cfsat_ga: CFSAT Phase4/search-only GA over graph paths.
• cfsat21_label_random: LLVM21 CFSAT artifact mode; use precomputed graph cluster labels and cluster-specific random walk.
• cfsat21_label_greedy: LLVM21 CFSAT artifact mode; use precomputed graph cluster labels and cluster-specific greedy extension.
• cfsat21_label_ga: LLVM21 CFSAT artifact mode; use precomputed graph cluster labels and cluster-specific seeded GA.

Important: the current cfsat_* implementations are not full CFSAT. Full CFSAT also includes Phase1-Phase3 synergy discovery, clustering, ProGraML/PyG dataset generation, GNN training, and GNN-based test-program cluster-label selection. The available /root/DJY/Comparasion/cfsat artifacts currently miss Phase3_graph_trained_model.pth and the environment lacks torch_geometric / torch_scatter, so this project labels these runs as CFSAT Phase4/search-only baselines.

The cfsat21_label_* strategies are separate from those search-only baselines. They do not train CFSAT inside PerProgramAutoTune. Instead they consume LLVM21-tagged CFSAT artifacts, then evaluate the selected per-program pass sequences through the same LLVM21 evaluator used by all other strategies. By default they expect:

/root/lw/lw2/cgo/home/cfsat/output/Phase4_graph_test_Sel_Label_llvm21.csv
/root/lw/lw2/cgo/home/cfsat/output/Phase2_Cluster_Synerpairs_llvm21.csv
/root/lw/lw2/cgo/home/cfsat/output/Phase2_Cluspairs_Passseq_llvm21.csv

Pass compatibility defaults to strict, meaning CFSAT artifacts must already use the active LLVM21 NewPM pass vocabulary such as function(newgvn) and module(globalopt). Use --cfsat21-pass-compat filter or legacy-map only for explicit compatibility/debugging experiments, not for formal main-table claims.

Quick Smoke Test

cd /root/exp/PerProgramAutoTune
python run_per_program_autotune.py \
  --suite polybench_full \
  --limit 1 \
  --strategy rio_random \
  --budget 3 \
  --seed 456 \
  --tag smoke

Validate the CFSAT21 graph converter without running training:

cd /root/exp/PerProgramAutoTune
python -m perprog_tune.cfsat21.smoke \
  --ir /root/exp/RFunipassLab/external_datasets/build/binarysize/none/cbench/automotive_bitcount/optimized.ll \
  --llvm2graph-bin /root/projects/ProGraML/bazel-bin/programl/bin/llvm2graph-21

Run On RFunipassLab tuning_results.csv

cd /root/exp/PerProgramAutoTune
python run_per_program_autotune.py \
  --program-csv /root/exp/RFunipassLab/data/tuning_results.csv \
  --program-csv-column filename \
  --program-csv-suite poj_tuning_results \
  --strategy ga_bitvector \
  --budget 60 \
  --jobs 8 \
  --seed 456 \
  --tag poj_tuning_results_ga_bitvector_budget60

Recommended Main Experiments

cd /root/exp/PerProgramAutoTune

python run_per_program_autotune.py \
  --suite polybench_full \
  --strategy all \
  --budget 60 \
  --seed 456 \
  --tag polybench_full_all_budget60

python run_per_program_autotune.py \
  --suite cbench \
  --suite csmith \
  --strategy all \
  --budget 60 \
  --seed 456 \
  --tag cbench_csmith_all_budget60

python run_per_program_autotune.py \
  --suite polybench_full \
  --strategy boca_flat \
  --strategy ga_bitvector \
  --strategy cfsat_ga \
  --budget 200 \
  --seed 456 \
  --tag polybench_full_strong_budget200

Run CFSAT21 artifact-mode tuning after LLVM21-tagged artifacts exist:

python run_per_program_autotune.py \
  --suite polybench_full \
  --strategy cfsat21_label_random \
  --strategy cfsat21_label_greedy \
  --strategy cfsat21_label_ga \
  --budget 60 \
  --max-seq-len 20 \
  --seed 456 \
  --baseline oz \
  --loop-policy wrap \
  --llvm-tools /root/llvm/llvm-project-21/build/bin \
  --cfsat21-root /root/lw/lw2/cgo/home/cfsat \
  --cfsat21-output-tag llvm21 \
  --llvm2graph-bin /root/projects/ProGraML/bazel-bin/programl/bin/llvm2graph-21 \
  --cfsat21-strict-artifacts \
  --tag cfsat21_label_polybench_full_budget60

Outputs

Every run creates a new directory under results/<run_id>/:

• manifest.json: controlled variables, toolchain, pass-space signature, selected programs, output paths.
• per_program.csv: one row per (program, strategy) with the best per-program sequence.
• per_program_details.json: full trace and best result details.
• summary.csv: method-level aggregate metrics.
• timing_events.csv: fine-grained setup/search/evaluation/LLVM timing events.
• timing_summary.csv: phase-level timing aggregation by strategy.
• REPORT.md: human-readable report and command flow.

per_program.csv also includes timing columns:

• baseline_time_s: time to compute the selected baseline for this program/strategy context.
• search_time_s: inclusive per-program strategy search time, excluding baseline setup.
• eval_time_s: total true candidate evaluation time inside the search budget.
• llvm_opt_time_s: total LLVM opt time observed for baseline and candidates.
• autophase_time_s: total Autophase TotalInsts time observed for baseline and candidates.
• strategy_overhead_s: search_time_s - eval_time_s, covering model fitting, candidate generation, selection, and Python overhead.

Timing phases use an explicit schema so later strategies can add their own phases without changing existing outputs. Current high-value phases include setup.load_pass_space, setup.load_programs, program.baseline, strategy.search_total, strategy.model_fit, strategy.acquisition, strategy.evolution, eval.total, llvm.opt, and llvm.autophase.

Interpretation

These baselines are online per-program auto-tuners. They use feedback from each target benchmark during search. Therefore they should not be interpreted as zero-shot generalization baselines. In a paper, compare them against RFunipass as a cost-quality frontier:

• RFunipass universal sequence: zero target-program tuning evaluations.
• Per-program baselines: budget target-program evaluations per benchmark.