perf(daemon): preserve telemetry source collection state

[tttboy123]

Closes #268.

What changed

• Keep a long-lived telemetry source registry on the daemon service instead of recreating provider instances every collection cycle.
• Build collectors from those stable source instances while continuing to reload account/config options each cycle.
• Make unchanged Codex and Claude Code JSONL files emit zero events.
• Preserve Claude Code append-only suffix parsing, but emit only newly parsed events.
• Replace cached historical event slices with compact file signature/offset state.
• Update the collection-loop comment to reflect first-import/changed-file behavior.

Root cause

collectAndFlush rebuilt collectors through providers.AllProviders() every cycle. Since AllProviders() constructs new provider instances, the per-provider telemetry caches were discarded every 20–30 seconds. Codex therefore reparsed historical session JSONL files repeatedly.

Even when a provider cache survived within one instance, cache hits returned all cached events again. That kept the cycle non-empty, repeated mapping/SQLite dedup work, retained large event slices, and prevented adaptive backoff.

User impact

Large local Codex or Claude Code histories no longer cause unchanged session files to be reparsed and re-ingested every collection cycle. After the initial import, idle cycles return no local-file events and can back off normally.

Codex still reparses a file when that file changes; implementing stateful suffix parsing for Codex can be a separate follow-up because token deltas and tool-result correlation cross line boundaries.

Tests

New regression coverage verifies:

• collector rebuilds reuse the same telemetry source instances
• a second collection over unchanged Codex files emits zero events
• changed Codex files are still collected
• a second collection over unchanged Claude Code files emits zero events
• appended Claude Code files emit only the new suffix event

Validation run:

• make fmt
• make vet
• make build
• make test
• go test -race -count=1 ./internal/providers/codex ./internal/providers/claude_code
• go test -race -count=1 ./internal/daemon -skip TestChangeDetectorReturnsFalse_WhenNoFiles

One existing environment-dependent daemon test, TestChangeDetectorReturnsFalse_WhenNoFiles, fails on this machine because a real Kiro CLI database is present. The same failure reproduces unchanged on upstream main.

[tttboy123]

Update: incremental telemetry read model

I pushed a structural follow-up in 0e05513.

Root cause

The existing aggregate cache avoids work while the database is idle, but every new event invalidates it. Under an active Codex session, OpenUsage repeatedly:

1. scans the complete usage history;
2. runs the logical-event window deduplication;
3. materializes a temporary table;
4. scans it again across multiple aggregate queries.

Rolling windows also used the exact nanosecond Since value in the cache key, so consecutive 30-day views could create different cache entries.

Implementation

The new implementation adds:

• a durable usage_event_changes sequence log maintained by SQLite triggers;
• tracking for inserts, updates, deletes, and raw-payload cleanup;
• an in-process candidate/winner projection;
• incremental winner replacement and candidate promotion after deletion;
• delta-based Go aggregation instead of repeatedly running the SQL aggregate suite;
• stable cache identity for rolling time windows;
• data-version-based daemon refresh invalidation;
• bounded change-log retention with safe cold rebuild when a sequence gap is detected.

This is intentionally a hybrid read model: the change log is durable, while candidates, winners, and aggregate state are process-local. A daemon restart performs one cold build; subsequent refreshes are incremental.

Measurements

Test database:

• 174 MB SQLite database
• approximately 73k Codex logical events
• active Codex JSONL writes during monitoring

Results:

Path	Before	After
Full/cold read-model build	~3.0s	0.47–0.69s
One new logical event	full rebuild	~1ms
New event plus rolling-window advance	full rebuild	~13ms
240-sample CPU average	—	0.58%
240-sample CPU maximum	34.6% on the previous candidate	7.6%
Samples at or above 20% CPU	observed previously	0

A later 60-sample check after more than one hour of daemon uptime, while 18 additional events were ingested, measured:

• average CPU: 0.44%
• maximum CPU: 2.1%
• samples at or above 5%: 0
• Codex 30-day cached read-model response: below 10ms
• steady RSS: approximately 205–218 MB

Correctness coverage

Tests cover:

• inserts of new logical events;
• higher-priority winner replacement;
• winner deletion and candidate promotion;
• rolling-window advancement without a cold rebuild;
• equivalence with the legacy SQL aggregates;
• change-log creation and pruning;
• daemon data-version refresh behavior.

Validation completed with:

• go test ./internal/telemetry
• go test -race ./internal/telemetry
• targeted daemon tests
• go vet ./internal/telemetry ./internal/daemon