feat(load): WHOOP API v2 loader with OAuth2 - #31
Conversation
…reeze) Adds load/whoop_api.py: OAuth2 authorization-code flow (rotating refresh tokens persisted immediately), incremental on-disk record cache, and DataFrame mappers matching the CSV-loader schemas. whoop.py public loaders dispatch to the API when authorized, so all_df/sleep pick up live data with no config change. File-export loaders remain as fallback; journal stays file-based (not exposed by the API). Context: Whoop columns in the QS export were frozen at the date of the last manual CSV download (see ErikBjare/alice#65).
|
@greptileai review |
Greptile SummaryThis PR adds live WHOOP API loading alongside the existing export loaders. The main changes are:
Confidence Score: 4/5The WHOOP API mapper needs fixes before merging live data from it.
src/quantifiedme/load/whoop_api.py Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[whoop.load_sleep_df / load_cycles_df / load_workouts_df] --> B{OAuth token file exists?}
B -- yes --> C[WHOOP API loader]
B -- no --> D[CSV/GDPR export loader]
C --> E[Fetch paginated API collections]
E --> F[Update JSON cache]
F --> G[Map records to existing DataFrame schemas]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[whoop.load_sleep_df / load_cycles_df / load_workouts_df] --> B{OAuth token file exists?}
B -- yes --> C[WHOOP API loader]
B -- no --> D[CSV/GDPR export loader]
C --> E[Fetch paginated API collections]
E --> F[Update JSON cache]
F --> G[Map records to existing DataFrame schemas]
Reviews (1): Last reviewed commit: "feat(load): add WHOOP API v2 loader with..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| "debt": (score.get("sleep_needed") or {}).get( | ||
| "need_from_sleep_debt_milli", 0 | ||
| ) | ||
| / 60_000, |
There was a problem hiding this comment.
When WHOOP omits sleep_needed or need_from_sleep_debt_milli from a scored sleep, this maps the missing value to 0 minutes. That makes incomplete API data look like a real zero sleep debt, while CSV blanks would stay missing and downstream summaries can underreport debt.
| "debt": (score.get("sleep_needed") or {}).get( | |
| "need_from_sleep_debt_milli", 0 | |
| ) | |
| / 60_000, | |
| "debt": ( | |
| (score.get("sleep_needed") or {}).get("need_from_sleep_debt_milli") | |
| ) | |
| / 60_000 | |
| if (score.get("sleep_needed") or {}).get("need_from_sleep_debt_milli") is not None | |
| else None, |
| "strain": cycle_score.get("strain"), | ||
| "energy_kcal": kilojoule * KCAL_PER_KILOJOULE if kilojoule else None, | ||
| } |
There was a problem hiding this comment.
When WHOOP returns a valid cycle score with kilojoule: 0, this truthiness check exports None instead of 0.0 kcal. Callers can no longer tell a real zero-energy day from a missing energy score.
| "strain": cycle_score.get("strain"), | |
| "energy_kcal": kilojoule * KCAL_PER_KILOJOULE if kilojoule else None, | |
| } | |
| "strain": cycle_score.get("strain"), | |
| "energy_kcal": kilojoule * KCAL_PER_KILOJOULE if kilojoule is not None else None, | |
| } |
| "strain": score.get("strain"), | ||
| "energy_kcal": kilojoule * KCAL_PER_KILOJOULE if kilojoule else None, | ||
| "max_hr": score.get("max_heart_rate"), |
There was a problem hiding this comment.
When WHOOP returns a workout score with kilojoule: 0, this truthiness check maps it to None. The workout DataFrame then reports the energy field as missing instead of preserving the real zero value.
| "strain": score.get("strain"), | |
| "energy_kcal": kilojoule * KCAL_PER_KILOJOULE if kilojoule else None, | |
| "max_hr": score.get("max_heart_rate"), | |
| "strain": score.get("strain"), | |
| "energy_kcal": kilojoule * KCAL_PER_KILOJOULE if kilojoule is not None else None, | |
| "max_hr": score.get("max_heart_rate"), |
| if df.empty: | ||
| return df |
There was a problem hiding this comment.
When all API sleep or recovery records are filtered out, _to_daily_df returns a DataFrame with no columns. Existing CSV loaders still expose the expected columns on empty results, so downstream code can silently lose score, duration, or recovery columns instead of receiving an empty compatible WHOOP frame.
Greptile SummaryThis PR adds live WHOOP API loading alongside the existing export loaders. The main changes are:
Confidence Score: 4/5The WHOOP API mapper needs fixes before merging.
src/quantifiedme/load/whoop_api.py Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[whoop.load_* caller] --> B{API token file exists}
B -- no --> C[CSV/GDPR export loader]
B -- yes --> D[whoop_api loader]
D --> E[OAuth access token]
D --> F[Incremental cache]
F --> G[WHOOP API pagination]
G --> H[Record mappers]
H --> I[Pandas dataframes]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[whoop.load_* caller] --> B{API token file exists}
B -- no --> C[CSV/GDPR export loader]
B -- yes --> D[whoop_api loader]
D --> E[OAuth access token]
D --> F[Incremental cache]
F --> G[WHOOP API pagination]
G --> H[Record mappers]
H --> I[Pandas dataframes]
Reviews (2): Last reviewed commit: "feat(load): add WHOOP API v2 loader with..." | Re-trigger Greptile |
| else pd.Timedelta( | ||
| hours=int(tz_offset[:3]), minutes=int(tz_offset[0] + tz_offset[4:6]) | ||
| ) |
There was a problem hiding this comment.
When WHOOP returns a negative fractional timezone offset such as -03:30, this builds -3h + 30m instead of -3h - 30m. Sleep and recovery rows can then be indexed under the wrong local wake date.
| else pd.Timedelta( | |
| hours=int(tz_offset[:3]), minutes=int(tz_offset[0] + tz_offset[4:6]) | |
| ) | |
| else pd.Timedelta( | |
| minutes=(1 if tz_offset[0] == "+" else -1) | |
| * (int(tz_offset[1:3]) * 60 + int(tz_offset[4:6])) | |
| ) |
| score = r["score"] | ||
|
|
||
| sleep = sleep_by_id.get(r.get("sleep_id")) | ||
| if sleep is not None: |
There was a problem hiding this comment.
When a scored cycle has kilojoule: 0, this falsy check exports None instead of 0.0 kcal. Downstream summaries then see a missing value rather than a real zero-energy day.
| "start": start, | ||
| "end": end, | ||
| "duration": end - start, | ||
| "activity": r.get("sport_name"), |
There was a problem hiding this comment.
When a workout score contains kilojoule: 0, this writes None instead of 0.0 kcal. Callers lose the difference between an actual zero-energy workout and an absent energy score.
| return _sleeps_to_df(fetch_collection("sleeps")) | ||
|
|
There was a problem hiding this comment.
When every fetched sleep or recovery record is filtered out as a nap or unscored, _to_daily_df([]) returns a dataframe with no columns. That breaks the promised CSV-compatible schema, so callers expecting fields like score, duration, or recovery can fail on an otherwise valid empty API result. Should the daily mappers pass their expected column lists into _to_daily_df, or should each mapper return its own empty dataframe before calling this helper?
…hema Address Greptile review: kilojoule=0 no longer maps to None (real zero-energy days), missing sleep_needed no longer fabricates zero debt, and empty daily frames keep the CSV-loader column schema.
PR #31 was merged while the typecheck job was still running (the CI watch raced and bound to the previous commit's checks) and broke typecheck on master: dict spreads over unannotated module-level fixtures infer object, and Index.tz needs a DatetimeIndex narrow.
1 participant
Why
Whoop physiology in the QS export has been frozen at 2026-05-13 (ErikBjare/alice#65). Root cause:
config.tomlpoints at a static CSV export dir downloaded on 05-13 — there was never any OAuth to expire. This adds a live WHOOP API v2 integration so the data keeps advancing without manual re-downloads.What
load/whoop_api.py— new module:python -m quantifiedme.load.whoop_api auth): one-shot localhost redirect server, browser consent, token persisted to platformdirs with 0600 perms.nextToken, limit 25, 429 backoff) for cycles / recoveries / sleeps / workouts._load_sleep_standard/_load_cycles_standard/_load_workouts_standard), incl. wake-date indexing convention and kJ→kcal conversion.auth/status/fetch.load/whoop.py— public loaders dispatch to the API when a token exists on the machine; file exports remain the fallback. API errors are deliberately not swallowed into a file fallback — silent staleness is exactly the failure mode this fixes. Journal loader stays file-based (API doesn't expose journal entries).WHOOP_CLIENT_ID/WHOOP_CLIENT_SECRETenv vars or.env.whoopat repo root (now gitignored via.env.*).requestspromoted to a direct dependency.Tests
25 new tests (fixtures match the official v2 response schemas): mapping incl. tz-offset wake-date edge cases, nap/unscored exclusion, refresh-token rotation persistence, pagination param casing (
next_tokenin response vsnextTokenin query), incremental cache upsert, and dispatch. Existing whoop CSV tests pinned to file dispatch so a real token on a dev machine doesn't flip them. All 50 whoop tests pass.