bpf_xdp_egress_dev cost bench

A microbench of what one bpf_xdp_egress_dev() call adds to an XDP program. The kfunc resolves a VLAN ifindex to its physical parent; the cost is one dev_get_by_index_rcu(). Shared as part of our research toward a new VLAN XDP kfunc, not a benchmark of anything shipped. Tested on a single isolated Zen 4 core.

Cost is mode1 - mode0 per cell: one BPF object carries both modes behind a mutable global, so the two JIT images are identical and the paired delta isolates the call. Two methods that do not share failure modes measure it: a PROG_TEST_RUN self-PMU loop (micro, 12 paired runs/cell, ABBA order) and a real attached XDP program read with bpftool prog profile (literal, counts aligned to run_cnt, threaded NAPI pinned to the isolated core). An A/A null cell measures the floor; each run writes its own audit (CHECKS.txt). Cycles are the primary metric, ns from the measured clock. Kernel 7.1.0-14078-g39f1d383ccb2 (bpf-next plus the proposed kfunc), AMD Ryzen 9 7900. Method in the run_kfunc.sh header.

Result (mean of paired deltas, 95% bootstrap CI)

• 49 +/- 2 cyc (~13 ns) warm, flat to ~512 VLANs; 130 cyc (~35 ns) at the 4094 ceiling (2.7x). The growth is instruction count (43 -> 146 added instructions at marginal CPI ~1, the chain walk lengthening), not memory stalls.
• ~76 cyc (~20 ns) while the working set is L2-resident, rising to ~108 (~29 ns) at full cache pressure, at constant chain depth throughout: the midpoint-strided working set holds mode0's instruction count flat (373 inst/call at every size), and the analyzer gates the headline on that. The ws=1 point sits below the band (62 cyc, one L1-resident mid-depth target) and matches the chain sweep's same-depth cell (61 cyc at pop 2048) within 2 cyc.
• Measurement floor (A/A null cells, both halves run mode 0): micro -0.40 +/- 0.52 cyc, literal -3.39 +/- 8.72 cyc.
• Real frames through the attached prog agree with the test-run numbers within ~10% warm (44 vs 49 cyc), and the deep-chain penalty largely vanishes in the real datapath (~40-60 cyc even at the ceiling), so the numbers above are upper bounds.
• DRAM fills/call <= 3e-4 everywhere (near+far), including cells with ~18 L2 misses/call: L3-resident, never a DRAM fill.
• vs bpf_fib_lookup warm (~330 cyc): ~15% warm, ~31% at the ceiling. An upper bound, since the baseline fib here is an empty-table miss; a populated route costs more and shrinks the fraction.
• Throughput cost ~1.0-1.2%, a per-call CPU-cost proxy on one veth sender core, not line rate.
• net_rx_action ~31-33% of host CPU under load.

All eight figures, the raw CSVs, and the run's machine-evaluated audit (CHECKS.txt, 10/10 PASS) are in results/kfunc-bench-20260703-113158/ (midpoint-strided working set). What changed since the first upload is in ERRATA.md.

Notes

• mode1 - mode0 is the kfunc's marginal cost in the fib->kfunc call pattern: bpf_fib_lookup fetches the device by ifindex first, so the kfunc re-walks a cache-warm chain. An upper bound on the split-over-fold delta (no fold variant measured); a standalone cold call would cost more and is not measured here.
• Literal absolute columns include profiling-trampoline overhead and do not decompose; only within-cell mode deltas are meaningful (their agreement with micro is the validation). The in-situ ceiling staying at ~40-60 cyc is consistent with the dependent chain re-walk overlapping the ~1100 cycles of surrounding independent per-packet work; fib itself still pays the depth once (literal mode0 rises ~190 cyc at the 4094 ceiling).
• The measured kfunc is the proposed version; a merged one adds net_eq/ndo_xdp_xmit gates this may lack, a few cycles.
• Clock is cycles/run time (~3.77 GHz, boost off); sysfs scaling_cur_freq's nominal ~3.0 GHz is not used.
• Isolation: isolcpus/nohz_full/rcu_nocbs, performance governor, boost and prefetcher off, perf_event_paranoid<=0, bpf_stats on, single netns. Boot cmdline, SMT-sibling state, and CPU mitigations are recorded in meta.txt; knobs verified before measuring.

Run

make                          # builds vmlinux.h from your kernel, then all
sudo ./run_kfunc.sh /out      # fresh timestamped subdir; writes CHECKS.txt
                              # (machine-evaluated audit) alongside the CSVs
python3 analyze_kfunc.py /out/kfunc-bench-<ts>

Needs clang, gcc, bpftool, libbpf, perf, iproute2; numpy/pandas/matplotlib to plot. Kernel must export bpf_xdp_egress_dev in BTF. Figures land in results/<run>/: cost, sits, ab, footprint, dist, context, pps, controls.