benchmark

Benchmark Suite — Rust vs Python

Reproducible head-to-head benchmark comparing mediaflow-proxy-light (Rust) against the reference mediaflow-proxy (Python).

The published measurements and architectural analysis live in docs/benchmark.md. This README is the operational guide.

What it measures

• Latency — avg / p50 / p90 / max per request through each proxy
• Throughput — effective MB/s (requests × file-size ÷ wall-time)
• CPU usage — summed across all worker processes during the run
• Memory (RSS) — summed across all worker processes during the run
• AES-CTR decryption — raw DRM cipher throughput (Python-only script here; the Rust ceiling is verified by openssl speed -evp aes-128-ctr)

Files

File	Purpose
bench.go	Go-based benchmark client (compiled, no GIL)
upstream.go	Optional simple Go upstream (less realistic than nginx)
nginx-bench.conf	Production-style nginx config used for published numbers
bench.toml	Rust proxy config used for benchmarking (300 s connect_timeout, all tunables shown)
bench_decrypt.py	PyCryptodome AES-CTR microbenchmark

Prerequisites

• Go 1.20+ — benchmark client compiler
• nginx native build — C/C++ upstream used in the published numbers (Docker nginx also works but adds VPN-layer overhead on macOS)
• Python 3.10+ with pycryptodome — AES microbenchmark only
• Both proxies built and runnable side-by-side

# macOS
brew install go nginx
pip install pycryptodome

# Debian / Ubuntu
apt-get install golang nginx python3-pip
pip install pycryptodome

Quick start

Five steps, one terminal each for the three servers plus a fourth for the run.

1. Build the benchmark client

cd tools/benchmark
go build -o bench bench.go

2. Start the nginx upstream (terminal 1)

# one-time — generate the test file if it doesn't exist
dd if=/dev/urandom of=/tmp/test.bin bs=1M count=7

# run nginx in foreground on :9997
nginx -c $(pwd)/nginx-bench.conf

3. Start the Rust proxy (terminal 2)

cd ../..   # back to project root
CONFIG_PATH=tools/benchmark/bench.toml \
    ./target/release/mediaflow-proxy-light

4. Start the Python proxy (terminal 3, in a sibling checkout)

cd ../mediaflow-proxy
API_PASSWORD=dedsec .venv/bin/uvicorn mediaflow_proxy.main:app \
    --host 0.0.0.0 --port 8889 --workers 8 --no-access-log

5. Run the benchmark (terminal 4)

cd ../mediaflow-proxy-light/tools/benchmark
BENCH_UPSTREAM="http://127.0.0.1:9997/test.bin" ./bench

You should see Rust's numbers appear alongside Python's for concurrency levels 10 / 20 / 30 / 50 / 100, with a summary table at the end.

Configuration (environment variables)

All knobs can be overridden:

Variable	Default
BENCH_UPSTREAM	http://127.0.0.1:9998/test.bin
BENCH_RUST_URL	http://127.0.0.1:8888
BENCH_PYTHON_URL	http://127.0.0.1:8889
BENCH_API_PASSWORD	dedsec
BENCH_FILE_MB	7
BENCH_WARMUP	5 (global warmup rounds)
BENCH_ROUNDS	5 (timed rounds per concurrency level)
BENCH_CONCURRENCY	10,20,30,50,100

Example — test higher concurrency with a larger file:

BENCH_FILE_MB=25 BENCH_SIZEMB=25 \
BENCH_CONCURRENCY=25,50,75,100,150 \
BENCH_ROUNDS=10 \
./bench

AES / CENC decryption microbenchmark

Raw AES-128-CTR throughput — the cipher used by CENC (Common Encryption) for Widevine, ClearKey, and PlayReady DRM.

Python (PyCryptodome)

python bench_decrypt.py

Rust ceiling

The Rust side uses the aes + ctr crates from the proxy. Their throughput matches OpenSSL's hardware path (AES-NI on x86, ARMv8 crypto extensions on Apple Silicon). Verify with:

openssl speed -evp aes-128-ctr

On Apple Silicon both reach ~19 GB/s for blocks ≥ 1 KB — roughly 41× faster than PyCryptodome's ~450 MB/s.

Methodology notes

• Global warmup: hits upstream + both proxies 5× at c=50 before any timed round so cold-start bias doesn't favour whichever proxy is tested last.
• Per-level warmup: 3 additional rounds before timed samples at each concurrency level so connection pools are warm.
• Median of 5 timed rounds: picks the median by wall time — avoids single-outlier distortions from GC pauses or OS scheduler jitter.
• Go HTTP client (compiled, goroutine per request): avoids the Python GIL artifacts that a Python urllib client would introduce.
• CPU / memory sampled via ps -o pcpu,rss across all processes matching the proxy pattern — covers all uvicorn workers and the single-process Rust binary.

Interpreting results

Average latency (avg) is what each user perceives. Throughput (MB/s) is wall-time-based and heavily penalises single outliers — useful for burst-scenario sizing, less meaningful for steady-state streaming where per-user latency matters more.

The DIRECT row is the upstream measured without any proxy — any latency difference between RUST / PYTHON and DIRECT is proxy overhead.