Summary
Route AI API calls through a centralized service that calculates and tracks token costs per request. Instead of each component calling AI providers directly and discarding token counts, funnel calls through a query service that:
- Calls the underlying provider (OpenRouter, Gemini, etc.)
- Looks up the model in a cost table
- Calculates cost from input/output token counts and model pricing
- Returns the response with
cost_in_dollars attached
Motivation
Currently AI API calls happen directly with no cost visibility. This makes it impossible to understand per-video generation costs or identify expensive operations.
Approach
- Add a general-purpose
query method to a central AI service that wraps any provider
- Track
input_tokens, output_tokens, cost_in_dollars, and model_slug on the relevant record
- Accumulate costs across multiple AI calls within a single operation
- Persist totals after successful completion
Reference
Implemented in Knowatoa's AiQueryService.query + TaskExecutable concern as a working pattern.
Summary
Route AI API calls through a centralized service that calculates and tracks token costs per request. Instead of each component calling AI providers directly and discarding token counts, funnel calls through a query service that:
cost_in_dollarsattachedMotivation
Currently AI API calls happen directly with no cost visibility. This makes it impossible to understand per-video generation costs or identify expensive operations.
Approach
querymethod to a central AI service that wraps any providerinput_tokens,output_tokens,cost_in_dollars, andmodel_slugon the relevant recordReference
Implemented in Knowatoa's
AiQueryService.query+TaskExecutableconcern as a working pattern.