Skip to content

feat(sync): optional end-to-end encryption for cloud sync and backups (#520)#548

Merged
ericgriffin merged 20 commits into
mainfrom
worktree-520-encrypted-sync
Jul 11, 2026
Merged

feat(sync): optional end-to-end encryption for cloud sync and backups (#520)#548
ericgriffin merged 20 commits into
mainfrom
worktree-520-encrypted-sync

Conversation

@ericgriffin

@ericgriffin ericgriffin commented Jul 10, 2026

Copy link
Copy Markdown
Member

Closes #520.

Opt-in client-side encryption for everything Submersion writes to a cloud provider: sync changesets, base snapshots, manifests, epoch/moved markers, and cloud backup artifacts. The provider only ever stores opaque blobs; the sync protocol is unchanged because discovery is filename-based and all merging is client-side.

Design spec: docs/superpowers/specs/2026-07-10-encrypted-cloud-sync-design.md
Implementation plan: docs/superpowers/plans/2026-07-10-encrypted-cloud-sync.md

Design highlights

  • Wrapped master key (restic/LUKS pattern). A random 256-bit master library key is wrapped by Argon2id-derived keys in a small plaintext keyslot file (submersion_keyslots.json) with two slots: the user passphrase and a generated 8-word (EFF short wordlist) recovery code. Passphrase change rewraps one file; KDF parameters are stored per slot and upgradable.
  • SBE1 envelope at a single seam. EncryptingCloudStorageProvider decorates the configured provider, so every uploaded byte is AES-256-GCM ciphertext by construction (gzip-before-encrypt, per-file random nonce, the logical filename bound as AAD so stored files cannot be replayed under a different protocol role). Exactly two filename exemptions: the keyslot file and backup artifacts (which carry their own framed format). Sync internals, checksums, and the streamed 8 MB base parts are untouched.
  • Enable/disable ride the existing epoch-replace machinery. Enabling is a library replace whose files come out encrypted; peers prompt for the passphrase (awaitingPassphrase halt + banner) and re-download via the existing adopt flow. Older app versions fail closed on the unreadable epoch marker (the already-shipped path) - verified and pinned by test, including the 1-hour _recoverUnreadableEpoch wipe hazard that made encrypting the marker mandatory.
  • Self-decrypting cloud backups. Cloud backup uploads become framed .sbe artifacts (8 MiB GCM frames, frame index + final flag in AAD, embedded keyslots) - restorable on a fresh device with just the passphrase, even downloaded manually from the provider's web UI. Local backups (including pre-migration backups) deliberately stay plaintext: disaster recovery must never sit behind a passphrase.
  • Key custody. The unlocked key lives in the platform keystore via the existing FallbackSecureStorage; the enabled flag is device-local (SyncPreferences). No database schema migration.

Testing

  • Known-answer vectors for Argon2id/HKDF/AES-GCM/envelopes generated with python3 (scripts/generate_crypto_test_vectors.py, committed fixture) - the Dart seal is byte-exact against python-built envelopes.
  • No-plaintext-leak invariant: after enable -> sync -> steady-state traffic, every stored byte except the keyslot file carries the SBE1 magic (asserted against the fake provider's raw storage).
  • Encrypted two-device convergence end-to-end: enable-as-replace -> locked-device halt -> unlock -> adopt -> bidirectional changeset sync.
  • Tamper/truncation/frame-reorder detection, wrong-passphrase/recovery-code paths, keyslot self-heal, lifecycle flows (enable/disable/change/regenerate), widget tests for the settings section, dialogs, unlock banner.
  • Full suite: 10798 passed / 12 skipped, flutter analyze clean, dart format clean.

Not in this PR (follow-ups)

  • Device verification on real hardware (two current devices + one legacy-version device against a real backend) before release.
  • cryptography_flutter platform-acceleration registration (dependency added; register during device verification only if profiling shows pure-Dart crypto is a bottleneck - measure first).
  • GitHub wiki user-guide page and release notes.

Copilot AI review requested due to automatic review settings July 10, 2026 13:25
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds opt-in, client-side end-to-end encryption for all cloud sync payloads and cloud backups, implemented as a provider decorator (EncryptingCloudStorageProvider) plus a keyslot-based key management flow with passphrase + recovery code, and corresponding UI/tests.

Changes:

  • Introduces SBE1 envelope encryption for sync files, keyslot file bootstrap, and framed .sbe encrypted cloud backup artifacts.
  • Adds settings UI + flows to enable/disable encryption, unlock a device, rotate passphrase/recovery code, and surface “needs passphrase” sync-paused states.
  • Adds extensive crypto KAT vectors, integration tests, and widget/provider tests; wires new localization strings and crypto dependencies.

Reviewed changes

Copilot reviewed 67 out of 69 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/support/fake_cloud_storage_provider.dart Exposes raw stored bytes for “no-plaintext-leak” assertions.
test/fixtures/crypto/crypto_vectors.json Adds committed crypto known-answer vectors (base64 fixtures).
test/features/settings/presentation/widgets/encryption_settings_section_test.dart Widget tests for encryption section states and enable flow.
test/features/settings/presentation/providers/sync_encryption_providers_test.dart Provider seam tests to ensure wrapping depends on unlocked session + flag.
test/features/settings/presentation/pages/troubleshoot_sync_page_test.dart Updates tests to provide SharedPreferences for new encryption status row.
test/features/settings/presentation/pages/cloud_sync_page_test.dart Adds tests for the “needs passphrase” unlock banner on Cloud Sync page.
test/features/backup/presentation/providers/backup_providers_restore_test.dart Updates restore signatures to accept an encryption secret.
test/features/backup/presentation/pages/backup_settings_page_test.dart Updates restore signatures to accept an encryption secret.
test/features/backup/data/services/backup_service_encryption_test.dart Verifies cloud backups become .sbe when encryption enabled; restore behaviors.
test/features/backup/data/services/backup_crypto_test.dart Tests framed backup encryption (round-trip, tamper detection, key mismatch).
test/core/services/sync/sync_encryption_gate_test.dart Pins fail-closed behavior and awaitingPassphrase gate when seeing encrypted bytes.
test/core/services/sync/encrypted_sync_integration_test.dart End-to-end encrypted sync convergence + leak invariant across two devices.
test/core/services/sync/crypto/sync_envelope_test.dart KAT + integrity/authentication tests for SBE1 envelopes.
test/core/services/sync/crypto/sync_encryption_service_test.dart Tests enable/unlock/rotate/self-heal/disable flows for keyslots.
test/core/services/sync/crypto/recovery_code_test.dart Tests recovery code generation/normalization and wordlist integrity.
test/core/services/sync/crypto/keyslots_test.dart Tests Argon2id/HKDF vectors and keyslot wrapping/unwrapping behavior.
test/core/services/sync/crypto/encryption_key_store_test.dart Tests secure storage key persistence and the new preference flag.
test/core/services/cloud_storage/encrypting_cloud_storage_provider_test.dart Tests provider decorator behavior and exemption rules.
scripts/generate_eff_wordlist.py Adds generator script for EFF short wordlist Dart constant.
scripts/generate_crypto_test_vectors.py Adds generator script for python-produced crypto vectors.
pubspec.yaml Adds cryptography and cryptography_flutter dependencies.
pubspec.lock Locks new crypto dependencies.
lib/l10n/arb/app_zh.arb Adds encryption UI strings (Chinese).
lib/l10n/arb/app_pt.arb Adds encryption UI strings (Portuguese).
lib/l10n/arb/app_nl.arb Adds encryption UI strings (Dutch).
lib/l10n/arb/app_it.arb Adds encryption UI strings (Italian).
lib/l10n/arb/app_hu.arb Adds encryption UI strings (Hungarian).
lib/l10n/arb/app_he.arb Adds encryption UI strings (Hebrew).
lib/l10n/arb/app_fr.arb Adds encryption UI strings (French).
lib/l10n/arb/app_es.arb Adds encryption UI strings (Spanish).
lib/l10n/arb/app_en.arb Adds encryption UI strings (English).
lib/l10n/arb/app_de.arb Adds encryption UI strings (German).
lib/l10n/arb/app_ar.arb Adds encryption UI strings (Arabic).
lib/l10n/arb/app_localizations.dart Adds generated localization getters for encryption UI strings.
lib/l10n/arb/app_localizations_en.dart Adds English implementations for new encryption strings.
lib/l10n/arb/app_localizations_zh.dart Adds Chinese implementations for new encryption strings.
lib/l10n/arb/app_localizations_pt.dart Adds Portuguese implementations for new encryption strings.
lib/l10n/arb/app_localizations_nl.dart Adds Dutch implementations for new encryption strings.
lib/l10n/arb/app_localizations_it.dart Adds Italian implementations for new encryption strings.
lib/l10n/arb/app_localizations_hu.dart Adds Hungarian implementations for new encryption strings.
lib/l10n/arb/app_localizations_he.dart Adds Hebrew implementations for new encryption strings.
lib/l10n/arb/app_localizations_fr.dart Adds French implementations for new encryption strings.
lib/l10n/arb/app_localizations_es.dart Adds Spanish implementations for new encryption strings.
lib/l10n/arb/app_localizations_de.dart Adds German implementations for new encryption strings.
lib/l10n/arb/app_localizations_ar.dart Adds Arabic implementations for new encryption strings.
lib/features/settings/presentation/widgets/encryption_settings_section.dart Adds encryption settings section and shared unlock orchestration.
lib/features/settings/presentation/widgets/encryption_passphrase_dialog.dart Adds passphrase/recovery-code prompt + change-passphrase dialog.
lib/features/settings/presentation/widgets/enable_encryption_dialog.dart Adds enable-encryption flow with warnings + recovery code step.
lib/features/settings/presentation/providers/sync_providers.dart Adds encryption key store/service/session providers; wraps cloud provider; adds needsPassphrase state.
lib/features/settings/presentation/pages/troubleshoot_sync_page.dart Adds encryption status row and unlock entrypoint on troubleshoot page.
lib/features/settings/presentation/pages/cloud_sync_page.dart Adds encryption section and needs-passphrase banner entrypoint.
lib/features/backup/presentation/providers/backup_providers.dart Wires backup service with encryption key store/preferences; adds secret-based restore retry path.
lib/features/backup/presentation/pages/backup_settings_page.dart Prompts for passphrase on encrypted backup restore and retries with secret.
lib/features/backup/domain/exceptions/backup_encrypted_exception.dart Adds exception signaling encrypted backup requiring a secret.
lib/features/backup/data/services/backup_crypto.dart Implements framed .sbe backup encryption/decryption with embedded keyslots.
lib/core/services/sync/sync_service.dart Adds awaitingPassphrase result, encrypted-marker detection, and keyslot self-heal hook.
lib/core/services/sync/sync_preferences.dart Adds persisted sync_encryption_enabled preference flag.
lib/core/services/sync/crypto/sync_envelope.dart Implements SBE1 single-shot envelope (gzip-before-encrypt + filename AAD).
lib/core/services/sync/crypto/sync_encryption_service.dart Implements enable/unlock/rotate/disable/self-heal keyslot lifecycle.
lib/core/services/sync/crypto/recovery_code.dart Implements 8-word recovery code generation + normalization.
lib/core/services/sync/crypto/keyslots.dart Implements keyslot file schema + Argon2id wrapping and HKDF data key derivation.
lib/core/services/sync/crypto/encryption_key_store.dart Secure storage for unlocked MLK + keyslot mirror.
lib/core/services/sync/crypto/crypto_errors.dart Adds typed exceptions for encryption-required and envelope corruption.
lib/core/services/sync/changeset_log/sync_manifest.dart Distinguishes encrypted bytes from corrupt manifests (throws SyncEncryptionRequired).
lib/core/services/cloud_storage/encrypting_cloud_storage_provider.dart Provider decorator that encrypts/decrypts non-exempt uploads/downloads.

Comment thread lib/core/services/cloud_storage/encrypting_cloud_storage_provider.dart Outdated
Comment thread lib/features/settings/presentation/providers/sync_providers.dart
@ericgriffin ericgriffin self-assigned this Jul 10, 2026
@ericgriffin ericgriffin moved this from Backlog to In review in Submersion Release Tracker Jul 10, 2026
Copilot review fixes:
- EncryptingCloudStorageProvider.downloadFile now honors the backup exemption
  by resolved name, symmetric with upload. Without this, restoring an
  encrypted cloud backup (a framed .sbe, which also starts with SBE1) through
  the decorated provider tried to open it as a single-shot envelope and failed.
- isExempt tightened to submersion_backup_* AND the .sbe extension, so a
  plaintext .db backup is never mistaken for a self-framed artifact.
- runEncryptionUnlockFlow bails before showing the dialog when no cloud
  provider is configured, instead of letting onSubmit succeed silently and
  reporting a spurious unlock.
- EncryptionKeyNotifier.clear() resets the memoized load to null (not a
  completed null future) so a later ensureLoaded() re-reads secure storage
  (e.g. a disable followed by a re-enable in the same container).

Tests: new coverage for both passphrase dialogs (0 -> 100%), the settings
section enable/unlock/change/regenerate/disable flows, decorator delegations
and the backup-download exemption, enable-dialog error paths, the notifier
clear/reload regression, the backup-restore encrypted-secret rethrow path,
validateBackupFile .sbe branches, and the crypto exception toStrings.
Patch coverage 97.6% (excl. generated l10n). Whole suite green, analyze clean.
Copilot AI review requested due to automatic review settings July 10, 2026 22:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 70 out of 72 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

lib/features/backup/presentation/providers/backup_providers.dart:245

  • restoreFromBackup swallows WrongPassphraseException (it falls into the generic catch), which means showEncryptionPassphraseDialog will treat the restore as a success and close, even though the restore failed. Catch WrongPassphraseException, reset state back to idle, and rethrow so the dialog can show its inline "Incorrect passphrase" error and stay open.
    } catch (e) {
      state = BackupOperationState(
        status: BackupOperationStatus.error,
        message: 'Restore failed: $e',
      );

lib/features/backup/presentation/providers/backup_providers.dart:371

  • restoreFromFilePath swallows WrongPassphraseException (it falls into the generic catch), so a bad passphrase entered in showEncryptionPassphraseDialog will still close the dialog instead of showing the inline error. Catch WrongPassphraseException, reset state to idle, and rethrow so the dialog can handle it.
    } catch (e) {
      state = BackupOperationState(
        status: BackupOperationStatus.error,
        message: 'Restore failed: $e',
      );

Comment thread lib/features/backup/presentation/providers/backup_providers.dart
- Backup restore notifier now rethrows WrongPassphraseException (not just
  BackupEncryptedException) so a wrong secret during a retry reaches the
  passphrase dialog's inline error and keeps it open, instead of being
  swallowed into an error state that closes the dialog on apparent success.
- Troubleshoot Sync's encryption-status row now uses the localized
  settings_cloudSync_encryption_* strings via context.l10n instead of
  hardcoded English, matching the Cloud Sync page; added a statusOff string
  in all ten locales.

Tests: notifier WrongPassphraseException rethrow (both restore paths),
troubleshoot row Off/On/Locked states and the tap-to-unlock closure.
Patch coverage stays ~97.5%; whole affected suite green, analyze clean.
@ericgriffin ericgriffin merged commit 39ac9b7 into main Jul 11, 2026
19 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in Submersion Release Tracker Jul 11, 2026
@ericgriffin ericgriffin deleted the worktree-520-encrypted-sync branch July 11, 2026 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Feature Request: Optional client-side encryption for cloud sync

2 participants