|
| 1 | +# Environment Variables |
| 2 | + |
| 3 | +All engine configuration is read from the environment via `env::getOr(name, fallback)` (`source/shared/utilities/env.cpp`) — every variable is **optional with a default** unless noted. An unset *or empty* variable falls back to its default. |
| 4 | + |
| 5 | +The `run` and `analysis` subcommands print a startup diagnostic dump of the whole environment to stderr, masking any variable whose name contains `PASSWORD`, `PASSWD`, `SECRET`, `TOKEN`, `CREDENTIAL`, `KEY`, or `AUTH`. |
| 6 | + |
| 7 | +## Quick reference |
| 8 | + |
| 9 | +| Variable | Default | `load` | `experiments` | `run` | `analysis` | `ingest` | `live` | `tracking` | `positions` | Purpose | |
| 10 | +| --- | --- | :-: | :-: | :-: | :-: | :-: | :-: | :-: | :-: | --- | |
| 11 | +| `REDIS_HOST` | `127.0.0.1` | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | Redis host (port is always `6379`) | |
| 12 | +| `BACKTEST_BATCH` | *(current UTC ISO week)* | ✓ | ✓ | | | | | | | Weekly batch label (`YYYY-WW`, e.g. `2026-28`) naming the Elasticsearch outcome indices (`backtesting-results-2026-28`, ...; `experiments`: `backtesting-experiments-2026-28`). Frozen into every queued run descriptor and carried through all rolling-window rungs; `scripts/load.sh` exports it once so its per-strategy invocations share one label. Set explicitly to re-run into a past week's indices | |
| 13 | +| `ENTRY_SLIPPAGE_TENTH_PIPS` | `0` | ✓ | | | | | | | | Slippage stress toggle: every backtest entry fills this many **tenths of a pip** against the trade (`3` = 0.3 pip; SL/TP anchors stay on the raw tick). Read once at `load` and frozen into every queued run descriptor, carried through all rolling-window rungs, and recorded in each run's results doc. **Validated** — a non-integer or negative value fails the load | |
| 14 | +| `QUESTDB_HOST` | `127.0.0.1` | | | ✓ | | ✓ | ✓ | | | QuestDB host (`run`: the required `<questdb-host>` argument covers tick reads only — the OHLC/range-bar warm-up still connects to `$QUESTDB_HOST`, so `run <remote-host>` reads ticks from the argv host but seeds bar histories from here; `live`: OHLC pre-population reads) | |
| 15 | +| `QUESTDB_PORT` | `8812` | | | ✓ | ✓ | | ✓ | | | QuestDB pgwire port for tick reads (`live`: OHLC pre-population reads). **Validated** — a non-numeric value is a hard failure | |
| 16 | +| `QUESTDB_ILP_PORT` | `9000` | | | | | ✓ | | | | QuestDB ILP-over-HTTP port for tick writes | |
| 17 | +| `TICK_CACHE_TTL_MINUTES` | `60` | | | ✓ | | | | | | Lifetime of a cached tick superset before it is reloaded from QuestDB (queue mode only — direct mode and `analysis` bypass the cache). **Validated** — a non-numeric or negative value throws on the cache's first use (once the first run is claimed), aborting the worker; `0` passes | |
| 18 | +| `TICK_CACHE_MAX_SUPERSETS` | `1` | | | ✓ | | | | | | How many tick supersets (one per symbol set) a worker keeps resident before evicting the oldest. **Validated** — same rule as the TTL | |
| 19 | +| `ELASTIC_ENABLED` | `1` | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | Master toggle for Elasticsearch reporting; set `0` to disable (`load`/`experiments`: also skips the weekly index/alias admin; `live`: also silences the trade audit, the trace documents, and the `live-logs` log feed; `tracking`: also silences the per-deal `live-trades` documents; `positions`: also silences the per-cycle `live-function-logs` reports) | |
| 20 | +| `ELASTIC_HOST` | `http://localhost:9200` | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | Elasticsearch base URL (results reporting; weekly index/alias admin at `load`/`experiments`; `analysis` experiment documents; live winner source + trade audit; `tracking` deal documents; `positions` cycle reports) | |
| 21 | +| `ELASTIC_USER` | *(empty)* | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | Basic-auth username; empty means no auth header | |
| 22 | +| `ELASTIC_USER_PASSWORD` | *(empty)* | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | Basic-auth password | |
| 23 | +| `ELASTIC_TRADES_ENABLED` | `0` | | | ✓ | | | | | | Set `1` to bulk-index every closed trade into `backtesting-trades` | |
| 24 | +| `ELASTIC_DEADLETTER_PATH` | `elastic_deadletter.ndjson` | | | ✓ | ✓ | | ✓ | ✓ | ✓ | File that failed Elasticsearch documents are appended to after retries are exhausted | |
| 25 | +| `ELASTIC_FLUSH_SECONDS` | `30` | | | ✓ | ✓ | | ✓ | ✓ | ✓ | Cadence of the background `_bulk` flusher that delivers queued reporting documents (run outcomes, experiment documents, engine exceptions, live trade audits, tracking deal documents); intervals with an empty buffer send nothing. A non-numeric or `< 1` value falls back to `30` with a logged warning | |
| 26 | +| `OHLC_PREPOPULATE` | `1` | | | ✓ | | | ✓ | | | Seed OHLC **and range** bar histories from QuestDB at each symbol's first tick (`run`: replay start; `live`: launch — big-bar strategies trade immediately instead of warming up for days). One switch covers both bar types; set `0` to disable all warm-up queries | |
| 27 | +| `INGEST_BIND_ADDR` | `127.0.0.1` | | | | | ✓ | | | | UDP bind address for the tick receiver | |
| 28 | +| `INGEST_UDP_PORT` | `11111` | | | | | ✓ | | | | UDP bind port (overridable by the command's port argument) | |
| 29 | +| `TRACKING_BIND_ADDR` | `127.0.0.1` | | | | | | | ✓ | | UDP bind address for the deal/trade-update receiver | |
| 30 | +| `TRACKING_UDP_PORT` | `11112` | | | | | | | ✓ | | UDP bind port (overridable by the command's port argument) | |
| 31 | +| `LIVE_BIND_ADDR` | `127.0.0.1` | | | | | | ✓ | | | UDP bind address for the live tick receiver | |
| 32 | +| `LIVE_UDP_PORT` | `11110` | | | | | | ✓ | | | UDP bind port (overridable by the command's port argument) | |
| 33 | +| `LIVE_MIN_SCORE` | `20` | | | | | | ✓ | | | Winner floor on `results.performanceScore` when selecting strategies from Elasticsearch (top 3 per (symbol, strategy) pair is fixed in code, not env-configurable) | |
| 34 | +| `LIVE_MAX_DRAWDOWN_PERCENT` | `10` | | | | | | ✓ | | | Hard ceiling on `results.maxDrawdownPercent` (peak-to-trough giveback) when selecting winners — the Calmar half of `performanceScore` only blends drawdown in, so this gate is what actually excludes spiky runs | |
| 35 | +| `LIVE_MIN_CALMAR_SCORE` | `30` | | | | | | ✓ | | | Floor on `results.calmarScore` when selecting winners (Calmar ratio ~2 on the score scale where ratio 3 = 50): growth must be ~2x the worst giveback. Complements the drawdown ceiling — the floor rejects smooth-but-stagnant runs, the ceiling rejects fast growers with deep absolute givebacks | |
| 36 | +| `LIVE_TRADE_LOCK_SECONDS` | `30` | | | | | | ✓ | | | TTL of the per-(strategy, direction) Redis trade lock | |
| 37 | +| `LIVE_TRACE_ENABLED` | `1` | | | | | | ✓ | | | Emit live-mode trace documents (order/close lifecycle, book sync, IG guard refusals, startup/shutdown, minutely stats) to the Elasticsearch index `live-traces` through the async batch publisher; set `0` to disable. Every document carries an `env` field (`TRADING_ENVIRONMENT`). Delivery still requires `ELASTIC_ENABLED=1` | |
| 38 | +| `LIVE_LOG_SHIP_ENABLED` | `1` | | | | | | ✓ | | | Ship every engine log line (`logLine`/`error`) as a document to the Elasticsearch index `live-logs`; set `0` to disable. An independent kill switch from `LIVE_TRACE_ENABLED` — the narrative log and the structured trace events are separate feeds. Delivery still requires `ELASTIC_ENABLED=1` | |
| 39 | +| `TRADING_ENVIRONMENT` | `demo` | | | | | | ✓ | ✓ | ✓ | IG environment; lowercased into the DynamoDB credentials key `Auth#<env>` (`demo`/`live`); `tracking` stamps it into each `live-trades` document's `env` field | |
| 40 | + |
| 41 | +## AWS credentials (`live` and `positions`) |
| 42 | + |
| 43 | +The IG session credentials are pulled from the DynamoDB table `MarketDataLive` (`source/shared/aws/dynamoAuth`). The engine reads no AWS variables itself — the AWS SDK's default credential/region chain applies: |
| 44 | + |
| 45 | +- `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION` — or a configured profile / instance role. |
| 46 | + |
| 47 | +Effectively **required** for `live` order placement and the `positions` sync (without a session, `positions` logs a warning and every cycle skips safely); there is no in-code default. |
| 48 | + |
| 49 | +## Build and script variables |
| 50 | + |
| 51 | +| Variable | Default | Read by | Purpose | |
| 52 | +| --- | --- | --- | --- | |
| 53 | +| `CC` / `CXX` | `clang` / `clang++` | `scripts/build.sh` (non-Homebrew path) | Pin a specific compiler, e.g. `CC=clang-20 CXX=clang++-20` | |
| 54 | +| `ENABLE_COVERAGE` | `OFF` | `scripts/build.sh` | Instrument with Clang source-based coverage (CI turns it on for the SonarCloud report) | |
| 55 | +| `CLEAN` | `0` | `scripts/test.sh` | `CLEAN=1` forces a clean reconfigure before the test build | |
| 56 | + |
| 57 | +`scripts/run.sh` additionally **requires** `ELASTIC_HOST`, `ELASTIC_USER`, `ELASTIC_USER_PASSWORD`, and `REDIS_HOST` to be set and non-empty — it aborts up front if any are missing (the engine itself would fall back to the defaults above). It also pins `ELASTIC_TRADES_ENABLED=0` for the run it launches (per-trade indexing off). |
| 58 | + |
| 59 | +## Secrets management |
| 60 | + |
| 61 | +I manage secrets with [Infisical](https://infisical.com/), which injects them into the process environment at runtime: |
| 62 | + |
| 63 | +``` |
| 64 | +infisical run -- bash ./scripts/run.sh |
| 65 | +``` |
| 66 | + |
| 67 | +If you're not using Infisical, export the variables yourself (shell profile or a sourced `.env`) before invoking the scripts. |
0 commit comments