Skip to content

fix(cron): classify agent job errors into actionable user messages (#2279)#2340

Merged
senamakel merged 1 commit into
tinyhumansai:mainfrom
sanil-23:fix/2279-cron-error-classifier
May 21, 2026
Merged

fix(cron): classify agent job errors into actionable user messages (#2279)#2340
senamakel merged 1 commit into
tinyhumansai:mainfrom
sanil-23:fix/2279-cron-error-classifier

Conversation

@sanil-23
Copy link
Copy Markdown
Contributor

@sanil-23 sanil-23 commented May 20, 2026

Summary

  • Replace the generic AGENT_JOB_USER_FAILURE_MESSAGE boilerplate on cron job failures with actionable, canned, user-facing messages keyed off AgentError variants.
  • Classifier is canned-string-only — never interpolates any error content — proven by a leak-canary fuzz test that asserts no marker substring reaches the output.
  • Falls back to the existing generic message for unmapped variants (ToolExecutionError, Other), so behaviour is bug-compatible with main in the residual.

Problem

Bug B of #2279. When the morning-briefing (or any agent) cron job fails, cron/scheduler.rs:333 returned only the generic "Something went wrong. Please try again. This error has been reported…" boilerplate, discarding the structured last_agent_error. Users saw repeated identical generic-failure cards in the notifications panel with no indication of cause (budget, rate-limit, misconfiguration, permission).

(Bug A — markup leak in the renderer — is a separate PR on fix/2279-notification-link-markup.)

Solution

Two private fns in cron/scheduler.rs:

  1. fn agent_error_to_user_message(err: &AgentError) -> &'static str — matches the 8 AgentError variants:

    Variant Message hint
    ProviderError {retryable: true} retry-automatically
    ProviderError {retryable: false} "Settings → AI → LLM" (creds)
    ContextLimitExceeded start a new session / larger context window
    CostBudgetExceeded "Settings → Billing"
    MaxIterationsExceeded "Settings → AI → LLM" (iteration cap)
    CompactionFailed fresh-context next run
    PermissionDenied adjust permissions in Settings
    ToolExecutionError, Other fallback to AGENT_JOB_USER_FAILURE_MESSAGE

    All canned strings ≤120 chars; Settings sub-paths verified against useSettingsNavigation.ts:163, 187, 195 breadcrumbs.

  2. fn classify_agent_anyhow_for_user(err: &anyhow::Error) -> &'static str — downcasts to AgentError (mirroring runtime.rs::sanitize_event_error_message) and routes through (1); falls back to the generic message when the downcast fails.

Wired at scheduler.rs:397 — the live anyhow::Error returned by agent.run_single is classified before any .to_string(). The raw error continues into last_agent_error for the observability pipeline (report_error), where stack traces and provider URLs are appropriate.

AGENT_JOB_USER_FAILURE_MESSAGE is preserved as the residual fallback — not removed, not renamed.

Submission Checklist

  • Tests added or updated — 14 new tests in scheduler_tests.rs: 7 per-variant mapping, 2 residual fallback, 2 anyhow wrapper, 1 length cap (≤120 chars), 2 leak-canary fuzz.
  • Diff coverage ≥ 80%cargo test --lib cron::scheduler → 47 passed, 0 failed. Every classifier arm is exercised.
  • Coverage matrix updated — N/A: behaviour-only fix to existing cron error surface.
  • All affected feature IDs listed under Related — N/A (no matrix row).
  • No new external network dependencies introduced.
  • Manual smoke checklist updated — N/A: not a release-cut surface; covered by Rust unit tests.
  • Linked issue closed via Closes #NNN — see Related (this PR closes Bug B only; Morning briefing notifications show generic failure errors #2279 stays open until Bug A's PR also merges).

Airtight guarantee: classifier never leaks error content

The classifier output is a &'static str. The leak-canary test (classifier_does_not_leak_error_content + classify_agent_anyhow_does_not_leak_when_downcast_succeeds) injects 10 distinct LEAK_CANARY_<n>_<hex> markers across every String/wrapped-source field of every variant (including Other(anyhow::anyhow!(canary).context(canary))), runs each through the classifier, and asserts no marker appears in the output and that the residual constant itself is canary-free. This guards against future refactors accidentally introducing format!("{}", err) or err.to_string() interpolation.

Impact

  • Runtime: Rust core only; no schema / RPC / frontend change.
  • UX: cron-job failure notifications now carry one of 7 contextual canned messages or the original generic fallback. No notification body now contains stack-trace fragments, provider URLs, or other error-derived content.
  • Notification dedup (notifications/store.rs::insert_if_not_recent) is intentionally not touched. Today two failures with the same body collapse within 60s; after this change, two failures with different classified reasons no longer collapse. Acceptable — arguably informative — and easily revisited if noisy in practice.
  • Compatibility: unmapped variants and non-typed errors still produce today's generic message. Existing log greppers / Sentry rules keyed off AGENT_JOB_USER_FAILURE_MESSAGE are unaffected.

Related


AI Authored PR Metadata

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: fix/2279-cron-error-classifier
  • Commit SHA: 69b4c86e

Validation Run

  • pnpm --filter openhuman-app format:checkN/A: no frontend changes.
  • pnpm typecheckN/A: no frontend changes.
  • Focused tests: cargo test --manifest-path Cargo.toml --lib cron::scheduler → 47 passed, 0 failed.
  • Rust fmt/check (if changed): cargo fmt --check clean; cargo check --lib clean (only pre-existing unrelated warnings).
  • Tauri fmt/check (if changed): N/A: shell not touched.

Validation Blocked

  • command: N/A
  • error: N/A
  • impact: N/A

Behavior Changes

  • Intended behavior change: agent cron-job failures with a known AgentError variant now surface a specific user-facing message instead of the generic "Something went wrong" boilerplate.
  • User-visible effect: morning-briefing and other cron failures now point users at the right Settings page (AI → LLM, Billing) or explain the failure mode (context limit, iteration cap, compaction).

Parity Contract

  • Legacy behavior preserved: unmapped variants and non-typed errors return AGENT_JOB_USER_FAILURE_MESSAGE byte-identical to today. The constant itself is unchanged. Dedup behaviour is untouched.
  • Guard/fallback/dispatch parity checks: leak-canary fuzz proves no error-derived substring leaks into user-facing output; per-variant tests prove each mapping; residual tests prove the fallback fires for the unmapped tail.

🤖 Generated with Claude Code

…inyhumansai#2279)

Cron job failures previously surfaced a generic
`AGENT_JOB_USER_FAILURE_MESSAGE` in user notifications, discarding the
structured `last_agent_error`. Add a classifier that maps known
`AgentError` variants to canned, user-actionable strings (budget,
rate-limit, timeout, misconfigured-model, …) while falling back to
the existing generic message for unknown variants.

Classifier output is canned-string-only — never interpolates error
contents — guarded by a leak-canary test that proves no error-derived
substring reaches the user-facing message.

Bug B of tinyhumansai#2279 (generic error). Bug A (markup leak) is a separate PR.

Refs tinyhumansai#2279.

Co-Authored-By: Claude <noreply@anthropic.com>
@sanil-23 sanil-23 requested a review from a team May 20, 2026 11:17
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 20, 2026

📝 Walkthrough

Walkthrough

The PR introduces error-to-user-message classification for agent cron-job failures. The scheduler now maps typed AgentError variants into fixed, data-free user notifications while preserving raw error strings for observability, validated by comprehensive test coverage including leak-canary fuzzing.

Changes

Agent Error Classification for Cron Notifications

Layer / File(s) Summary
Error classifier implementation and wiring
src/openhuman/cron/scheduler.rs
Imports AgentError and adds private helpers agent_error_to_user_message and classify_agent_anyhow_for_user that map known error variants to static user-facing messages. The error handler wires the classifier into the user notification path while keeping raw error strings for observability.
Comprehensive classifier test suite
src/openhuman/cron/scheduler_tests.rs
Tests validate per-variant classification expectations, fallback behavior for un-actionable errors, message length constraints (≤120 chars), and "leak-canary" checks ensuring classifier output never includes error field content across downcast and anyhow-wrapper paths.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

working

Suggested reviewers

  • graycyrus

Poem

🐰 A rabbit hops through error breeds,
Classifying cron job needs,
No secrets in the user's sight—
Just canned messages, safe and tight!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: introducing error-to-user-message classification for agent cron-job failures, making errors more actionable.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added the working A PR that is being worked on by the team. label May 20, 2026
Copy link
Copy Markdown
Contributor

@graycyrus graycyrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work — the &'static str return type is an elegant structural guarantee against data leakage, and the leak-canary fuzz tests are thorough. No critical or major issues; a couple of minor UX observations below.

File +/− Summary
scheduler.rs +58 −5 Two classifier fns (agent_error_to_user_message, classify_agent_anyhow_for_user), wired at the error site
scheduler_tests.rs +290 14 tests: per-variant mapping, residual fallback, anyhow wrapper, length cap, leak-canary fuzz

"The model provider is temporarily unavailable. The next run will retry automatically."
}
AgentError::ProviderError { retryable: false, .. } => {
"The model provider rejected the request. Check your provider credentials in Settings \u{2192} AI \u{2192} LLM."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] "Start a new session" is solid advice in an interactive context, but cron jobs start a fresh session on every run automatically — a user reading this notification can't really act on it. Consider rewording to something like:

"The conversation grew too long for the model's context window. The next run will start fresh. If this recurs, pick a model with a larger context window in Settings → AI → LLM."

This also gives the user a concrete Settings path, matching the other messages.

"The model provider rejected the request. Check your provider credentials in Settings \u{2192} AI \u{2192} LLM."
}
AgentError::ContextLimitExceeded { .. } => {
"The conversation grew too long for the model. Start a new session or pick a model with a larger context window."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] "daily cost budget" assumes the budget window is always daily. The variant fields are spent_microdollars / budget_microdollars with no period attached. If the budget window is ever configurable (or already isn't daily), this message will mislead. A safer phrasing: "You've reached the cost budget for this agent" (drop "daily").

@senamakel senamakel merged commit 48c4da4 into tinyhumansai:main May 21, 2026
31 of 36 checks passed
mtkik pushed a commit to mtkik/openhuman-meet that referenced this pull request May 21, 2026
…inyhumansai#2279) (tinyhumansai#2340)

Co-authored-by: sanil-23 <sanil@alphahuman.xyz>
Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

working A PR that is being worked on by the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants