feat(consumption): --compare mode — period-over-period spend delta by workflow ("why did spend change?")

[rshade]

Overview

Add a read-only --compare mode to gh-aw-fleet consumption that runs the
existing per---by rollup over two windows — the current period and the
immediately-preceding period of equal length — joins them on the group key,
and renders per-key spend deltas ranked by magnitude of change. It is the
"why did spend change?" derivative lens: consumption today answers what
(rollup) and where (--by workflow attribution), but has no why-did-it-move
axis. Motivated by external feedback on the fleet-observability announcement.

User Story

As a fleet operator under usage-based Copilot billing,
I want to compare this period's per-workflow spend against the prior period and
see which workflows drove the change,
So that when the bill moves I can name the workflow responsible instead of
eyeballing two separate rollups and diffing them in my head.

Problem Statement

The fleet has three FinOps lenses today and none of them answers "why did spend
change":

• Rollup / attribution (consumption, Add gh-aw-fleet consumption subcommand for cross-fleet billing rollups #57/feat(consumption): source AIC from gh aw logs --json, not aw_info.json cost #103 — shipped): one window, no
  comparison. Answers what and where.
• Absolute threshold (consumption --budget, feat(consumption): read-only over-budget highlighting in the rollup (--budget) #129 — shipped): flags rows
  over a fixed AIC ceiling. A static line, not movement.
• Forecast (feat: gh-aw-fleet forecast — fleet-wide pre-spend cost projection #102 — open): projects future spend. Forward-looking, not
  "what already changed."
• Overview snapshot (feat(cmd): gh-aw-fleet overview — unified cross-repo health + cost + drift view #153 — shipped): a single point-in-time drift×health×cost
  view. No temporal diff.

An operator whose bill jumped 30% this week has to run consumption twice
(over two hand-picked windows), export or eyeball both tables, and mentally
subtract per workflow. The signal the reviewer named — the derivative of
spend, attributed to the finest grain the fleet has — is exactly the thing the
tool makes you compute by hand.

Proposed Solution

Technical Approach

--compare is a pure post-aggregation join over the substrate that already
exists — no new gh aw logs seam, no new data path. It mirrors how
ApplyBudget (internal/fleet/consumption_budget.go) is pure post-processing
over result.Groups.

• Flag surface: one boolean cmd.Flags().BoolVar(&flags.compare, "compare", …).
  Auto-derives the prior window from the current window's length. Requires
  --trailing Nd or --since D; errors under --latest (a single most-recent
  report has no equal-length predecessor). Message:
  --compare requires --trailing or --since; --latest has no comparable prior period.
• Window derivation:
  • --trailing Nd --compare → current [now-Nd, now], prior [now-2Nd, now-Nd].
  • --since D --compare → current [D, now], prior [D-(now-D), D] (prior is
    the equal-length window ending at D).
• Two aggregations, one join: call the existing
  AggregateConsumption(ctx, cfg, mode, by, source) (via the cmd-level
  aggregateConsumption seam) twice — once for the current FetchMode, once
  for the prior — then join the two []ConsumptionGroup on .Key. For each key
  compute ΔAIC, ΔUSD (Cost), and Δ%; sort by |ΔAIC| descending so the
  biggest movers surface first.
• Nil↔value (reporting-state) transitions — AIC/Cost are *float64;
  nil means "no successful runs / all-failed / no data," which the all-or-nothing
  group-merge (nilAICDiag) can produce. A nil↔value transition is a reporting
  change, not real movement, and must not render as a numeric ±:
  • prior nil, current value → NEW (workflow first reported spend this period).
  • prior value, current nil → GONE (no data / all runs failed this period).
  • both nil → omit the row.
  • both present → real ΔAIC / Δ%.
    Reporting-state rows sort after all real movers, so the ranking stays an
    honest "how much did real spend move."
• Δ% guard: undefined when prior is 0 or nil → render —, never a
  divide-by-zero or spurious +∞%.
• Source restriction: --compare is meaningful only under --source logs
  (the artifacts source has structurally-nil AIC under Copilot AI-Credits, so
  every row would be a nil-transition). Reject --compare + --source artifacts
  with a clear error, or document the degradation — implementer's call, but do
  not silently emit an all-NEW/GONE table.
• Output:
  • Text: a delta table <KEY> PRIOR CURRENT ΔAIC Δ% PRIOR_COST CURRENT_COST ΔCOST,
    biggest |ΔAIC| first, reporting-state rows (NEW/GONE) last. When
    --compare is absent, output is byte-identical to today (no new columns).
  • JSON: an additive comparison payload on ConsumptionResult
    (e.g. {current_window, prior_window, deltas: [{key, prior_aic, current_aic, delta_aic, delta_pct, state}]}). Additive — no cmd.SchemaVersion bump.

Files Likely Affected

• cmd/consumption.go — add compare to consumptionFlags; validate the
  --latest/--source artifacts conflicts; new renderComparisonText branch
  parallel to renderConsumptionText.
• internal/fleet/consumption.go — the join/delta builder (new
  ConsumptionComparison + WorkflowDelta types on ConsumptionResult); the
  prior-window FetchMode derivation.
• Likely net-new plumbing: today's FetchMode kinds are all anchored to
  now (FetchLatest, FetchTrailing.Days, FetchSince.Since→now). The prior
  window needs a bounded [start, end] range, which no current FetchKind
  expresses. Expect a new internal FetchBetween{Start,End} (or a client-side
  end-bound re-filter reusing the existing created_at re-filter). This is the
  main non-trivial change; flag it in the plan.
• internal/fleet/testdata/{consumption,logs}/ — two-window fixtures exercising:
  real movers, NEW, GONE, both-nil (omitted), and Δ% divide-by-zero guard.
• skills/fleet-budget-review/SKILL.md — document the compare lens alongside
  the rollup and --budget.

Acceptance Criteria

• consumption --by workflow --trailing 7d --compare prints a per-workflow
  delta table ranked by |ΔAIC|, with prior = the preceding 7d.
• --compare under --latest errors with the "no comparable prior period"
  message; --compare works under both --trailing Nd and --since D.
• --compare works for every --by axis (repo/profile/cost-center/workflow),
  workflow being the headline grain.
• nil↔value transitions render as NEW/GONE (never a numeric ±); both-nil
  rows are omitted; reporting-state rows sort after real movers.
• Δ% renders — when prior is 0 or nil — no divide-by-zero, no +∞%.
• --compare + --source artifacts is rejected (or documented-degraded),
  not silently all-NEW/GONE.
• Without --compare, text and JSON output are byte-identical to today.
• --output json carries the additive comparison payload; no
  cmd.SchemaVersion bump.
• Table-driven tests over the two-window fixtures run offline via the
  existing seams; make ci green.
• fleet-budget-review SKILL.md documents the compare lens.
• No hand edits to CHANGELOG.md (release-please owns it).

Out of Scope

• Enforcement, alerting, capping, or deploy-gating on a delta — read-only,
  exit-code unaffected (matches the --budget highlight-not-enforce boundary).
• Forecasting / pre-spend projection (feat: gh-aw-fleet forecast — fleet-wide pre-spend cost projection #102) and runtime cap-hit hints (feat(diagnostics): cap-hit hints (max-ai-credits / max-turns exceeded) #106).
• Arbitrary two-window comparison (explicit --baseline-since/--baseline-until)
  — the auto-derived equal-length prior period is the shipped model; an explicit
  baseline window can be a follow-up if operators ask for it.
• Anomaly detection / statistical significance on the delta — this is a raw
  diff, not a "is this change surprising" model.
• --source artifacts support (structurally-nil AIC makes the diff meaningless).

Technical Notes

• The delta builder is stateless post-aggregation logic over two
  []ConsumptionGroup slices — an excellent offline unit-test target (no
  network), same testing posture as ApplyBudget.
• Keep every gh access behind the existing aggregateConsumption /
  ghLogsAPI / ghWorkflowsAPI seams so tests stay offline over
  testdata/{consumption,logs}/.
• Running two aggregations doubles the gh aw logs fan-out per invocation.
  Consider noting this in --compare's help text; it composes with the bounded-
  concurrency work in consumption --source logs: bounded concurrency + no-download fast path for the per-workflow fan-out #113.
• Decide the Δ% sign/format convention up front (e.g. +120% / -9% /
  —) and pin it in a test — presentation drift here is easy to regress.

Related

• Sibling FinOps lenses: feat: gh-aw-fleet forecast — fleet-wide pre-spend cost projection #102 (forecast — future spend), feat(consumption): read-only over-budget highlighting in the rollup (--budget) #129 (--budget
  threshold — shipped), feat: cost-oriented trigger-risk lint over the resolved fleet #104 (trigger-risk lint), feat(diagnostics): cap-hit hints (max-ai-credits / max-turns exceeded) #106 (cap-hit hints),
  feat(deploy): tier-driven GH_AW_DEFAULT_* guardrail injection at compile #107 (tier guardrails).
• Attribution grain: Tracking: surface Copilot credit attribution once aw_info.json cost field stabilizes upstream #59 (per-workflow Copilot-credit attribution tracking).
• Context: docs: document the two-layer FinOps cost model (aggregate rollup vs. per-run attribution) #127 (two-layer FinOps model — this adds the third, derivative layer).
• Perf composition: consumption --source logs: bounded concurrency + no-download fast path for the per-workflow fan-out #113 (bounded concurrency for the logs fan-out).
• Spec: specs/009-consumption-subcommand/.
• Origin: external feedback on the fleet-observability announcement post
  ("a next useful axis might be 'why did spend change?' by workflow delta").