Skip to content

refactor: migrate remaining flat/forwarding RetryableError impls to derive (tier 1)#3757

Closed
insipx wants to merge 2 commits into
insipx/retryable-migrationfrom
insipx/retryable-tier1
Closed

refactor: migrate remaining flat/forwarding RetryableError impls to derive (tier 1)#3757
insipx wants to merge 2 commits into
insipx/retryable-migrationfrom
insipx/retryable-tier1

Conversation

@insipx

@insipx insipx commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Stacked on #3754 (which stacks on #3753) — merge in order.

Completes the #[derive(Retryable)] migration for every remaining impl that the macro can express, including the heavyweights deliberately deferred from #3754: GroupError (45 variants), SubscribeError, StorageError, ConnectionError, ApiClientError, GroupMessageProcessingError. 17 hand-written impls deleted across 6 crates + xnet.

What each shape became

  • ~80 forwarding arms → #[retry(inherit)]
  • Hardcoded trues (GroupError::LockUnavailable, SubscribeError::StreamStale, EnvelopeError::Decode, ApiClientError::Http/Expired, …) → #[retry(true)]
  • All-true enums (GrpcError, wasm PlatformStorageError) → #[retry(default = true)]; all-false validation enums → bare derive, zero attrs
  • Multi-field forwards (ApiClientError::ClientWithEndpoint, BootstrapSynthesisError::IdentityUpdateLookup, CommitLogError::FailedToSendReadd) → #[retry(when = source.is_retryable())]
  • Collection aggregation (CommitLogError::FailedReadds) → #[retry(when = errors.iter().any(|e| e.is_retryable()))]
  • GroupMessageProcessingError's nested match collapsed to inherit by deriving on the inner ProcessMessageWithAppDataError (the one API-surface change: that generic enum gains a ProcessMessageError<S>: RetryableError<Mls> where-bound, per the derive's documented generics rule; the only workspace instantiation already satisfies it)

Still hand-written, by design

The 22 specialized impl RetryableError<Mls> for openmls::*/diesel orphan-rule impls (mirror-type macro support would cost more code than it removes), GroupAppDataError<SqlKeyStoreError> (instantiation-specific), ReceiveErrors/SyncSummary one-line aggregators, and test mocks. After this PR, hand-written impls are a single explainable category: foreign-type orphan workarounds.

Verification

  • Ground truth captured verbatim before any edit; adversarial audit re-derived all 204 variants against it: zero mismatches
  • 32 new golden tests pin pre-migration booleans (every forwarding direction, every hardcoded-true oddball, the catch-all-false pins); 73 retryable tests green across touched crates
  • cargo check --workspace --all-targets clean; wasm cargo check -p xmtp_db --target wasm32-unknown-unknown clean (wasm PlatformStorageError migrated); fmt + clippy -Dwarnings clean

🤖 Generated with Claude Code

Note

Migrate manual RetryableError implementations to #[derive(Retryable)] across tier-1 crates

Replaces hand-written impl RetryableError blocks across ~18 files with #[derive(Retryable)] and per-variant #[retry(...)] annotations from xmtp_common.

  • Each variant is annotated with #[retry(true)], #[retry(inherit)], or a conditional expression (e.g. #[retry(when = source.is_retryable())]) to preserve existing retryability semantics exactly.
  • Enums where all variants were unconditionally retryable use #[retry(default = true)] at the enum level, and enums where all variants were non-retryable are left unannotated.
  • Golden tests are added alongside most conversions to pin retryability behavior and catch future regressions.
  • Affected crates include xmtp_api_d14n, xmtp_api_grpc, xmtp_db, xmtp_mls, xmtp_proto, and xnet.
  • Behavioral Change: retryability logic is unchanged in intent, but is now enforced at compile time via the derive macro rather than runtime hand-written logic.

Macroscope summarized 2455683.

insipx and others added 2 commits June 10, 2026 13:13
…able)]

Replace ~28 hand-written `impl RetryableError` blocks across 9 crates with the
new `#[derive(Retryable)]`. Behavior is preserved per-variant:
- #[from] variants auto-forward to the inner error's is_retryable()
- foreign-wrapping #[from] variants (prost/http/derive_builder/etc) carry an
  explicit #[retry(true|false)] since the inner type can't impl RetryableError
- inverted 'retryable-unless' enums use #[retry(default = true)] + #[retry(false)]
- ClientError::Generic uses #[retry(when = this.contains("database is locked"))]
- VerifierError uses #[retry(true)] + #[retry(inherit)]

Golden equivalence tests added for every non-trivial migrated enum.

Kept hand-written: specialized RetryableError<Mls> openmls impls, collection
aggregators (ReceiveErrors), and enums matching into foreign inner errors
(StorageError, ConnectionError, ApiClientError, EnvelopeError,
GroupMessageProcessingError, SubscribeError, ...).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…erive (tier 1)

Migrates the 17 remaining non-specialized hand-written impls: GroupError,
SubscribeError, StorageError, ConnectionError, ApiClientError, PlatformStorageError
(native+wasm), GroupMessageProcessingError, CommitLogError, EnvelopeError,
QueryError, GrpcError, BootstrapSynthesisError, stream errors, and four flat
validation enums. Multi-field forwards use #[retry(when = source.is_retryable())];
collection aggregation uses when over .any(); inverted enums use default = true.

Pre-step: ProcessMessageWithAppDataError now derives Retryable itself (with a
ProcessMessageError<S>: RetryableError<Mls> where-bound), letting the outer
nested match collapse to #[retry(inherit)].

Still hand-written (intentionally): the 22 specialized RetryableError<Mls>
impls for foreign openmls/diesel types, GroupAppDataError (instantiation-
specific), ReceiveErrors/SyncSummary aggregator one-liners, and test mocks.

32 golden tests pin pre-migration behavior; adversarial audit re-derived all
204 variants against the saved ground truth: zero mismatches.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@insipx insipx requested a review from a team as a code owner June 10, 2026 20:00
@claude

claude Bot commented Jun 10, 2026

Copy link
Copy Markdown

Claude finished @insipx's task —— View job


Code Review Complete

Review Date: 2026-06-10 20:02 UTC

Overview

This PR successfully completes the migration of the remaining hand-written RetryableError implementations to the #[derive(Retryable)] macro, including major enums like GroupError (45 variants), SubscribeError, StorageError, CommitLogError, and ApiClientError. The migration is comprehensive, well-tested, and maintains backward compatibility.

✅ Strengths

  1. Excellent test coverage - 32 new golden tests pin pre-migration behavior across all touched crates
  2. Comprehensive documentation - Each variant has clear doc comments explaining retryability
  3. Proper attribute usage - Correct application of #[retry(inherit)], #[retry(true)], #[retry(when = ...)], and #[retry(default = true)]
  4. Verification rigor - Author's adversarial audit of all 204 variants with zero mismatches
  5. Clean migration patterns:
    • ~80 forwarding arms → #[retry(inherit)]
    • Hardcoded true#[retry(true)]
    • All-true/all-false enums → #[retry(default = true)] or bare derive
    • Multi-field forwards → #[retry(when = source.is_retryable())]
    • Collection aggregation → #[retry(when = errors.iter().any(|e| e.is_retryable()))]

Findings

No critical issues found. The migration is high-quality with excellent attention to detail.

Minor observations (not blocking):
  1. crates/xmtp_api_d14n/src/queries/mod.rs:78 - QueryError::Envelope doesn't inherit retryability even though the inner EnvelopeError can be retryable (golden test confirms this was intentional pre-migration behavior)

  2. Complex #[retry(when = ...)] expressions - Some are quite sophisticated (e.g., CommitLogError::FailedReadds checking errors.iter().any(|e| e.is_retryable())). These look correct but could be more maintainable with helper methods if this pattern repeats elsewhere.

  3. Nested error forwarding - The needs_db_reconnect() implementations correctly follow nested errors (e.g., CommitLogError::FailedToSendReadd forwards to source, FailedReadds checks all errors). Good defensive depth.

Code Quality Assessment

  • Correctness: Golden tests validate all migrations match original behavior
  • Best practices: Consistent attribute usage, comprehensive documentation
  • Performance: No concerns - macro generates equivalent code
  • Security: No concerns identified
  • Test coverage: Excellent - 32 golden tests + 73 retryable tests passing

Conclusion

This is a well-executed refactoring that:

  • Reduces boilerplate (17 hand-written impls deleted)
  • Improves maintainability through macro consistency
  • Maintains 100% behavioral compatibility
  • Adds comprehensive test coverage

Recommendation: Approve and merge. The PR is ready to land after #3754.


@macroscopeapp

macroscopeapp Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved

Mechanical refactoring that migrates manual RetryableError trait implementations to #[derive(Retryable)] macros. Behavior is preserved and verified by extensive golden tests. All files are owned by the author.

You can customize Macroscope's approvability policy. Learn more.

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.91%. Comparing base (526c596) to head (2455683).

Additional details and impacted files
@@                      Coverage Diff                       @@
##           insipx/retryable-migration    #3757      +/-   ##
==============================================================
+ Coverage                       84.63%   84.91%   +0.27%     
==============================================================
  Files                             407      407              
  Lines                           59818    59871      +53     
==============================================================
+ Hits                            50629    50840     +211     
+ Misses                           9189     9031     -158     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@insipx insipx force-pushed the insipx/retryable-migration branch from 526c596 to 89a43a6 Compare June 11, 2026 16:09
@insipx

insipx commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #3754 per review simplification — one migration PR instead of two, and per-enum golden tests dropped in favor of the macro's own test suite as the behavioral contract (net −456 lines). The tier-1 commit content is now part of #3754.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant