Skip to content

feat(media-store): large objects, phase 3 (multipart, resume, video)#557

Merged
ericgriffin merged 19 commits into
worktree-media-store-phase2from
worktree-media-store-phase3
Jul 11, 2026
Merged

feat(media-store): large objects, phase 3 (multipart, resume, video)#557
ericgriffin merged 19 commits into
worktree-media-store-phase2from
worktree-media-store-phase3

Conversation

@ericgriffin

@ericgriffin ericgriffin commented Jul 10, 2026

Copy link
Copy Markdown
Member

Summary

Phase 3 (large objects) of the Media Store, per docs/superpowers/specs/2026-07-10-s3-media-storage-design.md sections 8, 9, 17; implementation plan docs/superpowers/plans/2026-07-10-media-store-phase3.md.

STACKED on #556, which stacks on #550 - merge bottom-up (#550, #556, then this); GitHub retargets each PR as its base branch is deleted on merge.

Videos are now first-class: uploads above 8 MiB go through S3 multipart with per-part resume state persisted on the queue row (a transfer killed at part N restarts at part N+1, proven by request counting against a protocol-level fake S3 server), downloads stream Range chunks to disk so memory stays bounded, and a video attached on device A plays from the store on device B through a resolvedFilePathProvider fallback - the single seam the video player already consumes.

Changes

  • MediaObjectStore.putFile/getFile gain optional progress and adapter-opaque resume parameters (the extension Phase 1 reserved)
  • S3ApiClient: createMultipartUpload / uploadPart / completeMultipartUpload / abortMultipartUpload / listParts / getObjectRange, plus unsigned extra-header support for Range requests
  • S3MediaObjectStore: threshold-switched putFile (single-shot <= 8 MiB, multipart above) with resume validation via ListParts (stale/mismatched sessions abort and restart fresh); chunked Range-GET downloads with progress
  • Queue (local cache DB v3): progress_bytes/total_bytes columns, updateResumeState/updateProgress; markDone clears resume/progress while markFailed/retry/defer preserve them
  • Pipeline: video eligibility lifted (the Phase 2 videosOnCellular gate activates with zero changes); resume state and progress thread through the queue row; gallery-video poster thumbs upload via the existing thumb step
  • Video playback on remote devices: resolvedFilePathProvider order is now gallery asset -> existing localPath -> store download into the content-addressed cache -> null; photo_viewer_page untouched (its loading state covers the download)
  • Transfers view renders determinate progress bars for in-flight transfers
  • Test infrastructure: reusable FakeS3Server (multipart sessions, Range, part-failure injection) exercising the real SigV4 signing path; existing S3ApiClient test doubles converted to Fake-based so future client additions cannot break them

Test Plan

Screenshots

Transfers view with a determinate progress bar to follow with the manual pass.

Documented deviation (also in the plan): non-gallery (localFile) video thumbnails are deferred to Phase 5 with placeholder degradation; gallery videos - the dominant path - get posters through photo_manager. No native thumbnailer dependency was added.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements Phase 3 of the Media Store “large objects” work: multipart/resumable S3 uploads and chunked (Range) downloads to keep memory bounded, lifts video eligibility through the upload pipeline, threads resume/progress through the per-device transfer queue, and adds a store-backed fallback for cross-device video playback plus determinate progress UI in Transfers. Test infrastructure is expanded with a reusable in-memory Fake S3 server to exercise the real SigV4 path for multipart and Range behaviors.

Changes:

  • Extend MediaObjectStore with optional progress + adapter-opaque resume hooks; propagate through the upload pipeline and queue persistence.
  • Add S3 multipart + Range support (S3ApiClient, S3MediaObjectStore) including resume validation via ListParts, and chunked downloads.
  • Improve UX/behavior around videos and transfers (video playback fallback; determinate progress bars) with expanded coverage using a shared FakeS3Server.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/helpers/in_memory_media_object_store.dart Adds progress/resume hook support to the in-memory MediaObjectStore test double.
test/helpers/fake_s3_server.dart New reusable in-memory S3 protocol fake (multipart + Range) for tests.
test/features/settings/presentation/s3_config_page_test.dart Updates S3ApiClient test double to Fake for forward-compatible stubbing.
test/features/media_store/transfers_page_test.dart Adds widget test for determinate progress rendering.
test/features/media_store/support/fake_local_file_resolver.dart Allows separate thumbnail data to model gallery video posters in tests.
test/features/media_store/resolved_file_path_store_fallback_test.dart New provider test covering store-backed video playback fallback behavior.
test/features/media_store/media_upload_pipeline_test.dart Expands pipeline tests for video uploads, resume/progress plumbing, and poster thumb handling.
test/features/media_store/media_transfer_queue_repository_test.dart Adds migration and resume/progress persistence tests for queue schema v3.
test/features/media_store/media_store_end_to_end_test.dart Adds cross-device video end-to-end coverage (upload on A, play/resolve on B).
test/core/services/media_store/s3_media_object_store_test.dart Refactors to use FakeS3Server and adds multipart/resume/chunked-download tests.
test/core/services/media_store/media_object_store_contract.dart Extends store contract to require progress callbacks reaching full size.
test/core/services/cloud_storage/s3/s3_multipart_test.dart New unit tests for multipart lifecycle + Range GET in S3ApiClient.
test/core/services/cloud_storage/s3_storage_provider_test.dart Updates S3ApiClient test double to Fake to avoid breakage from new methods.
lib/features/media/presentation/providers/resolved_asset_providers.dart Adds store-backed fallback path for video playback when local file is missing.
lib/features/media_store/presentation/pages/transfers_page.dart Renders determinate/indeterminate progress bars for transferring queue entries.
lib/features/media_store/data/media_upload_pipeline.dart Removes video ineligibility; wires resume state + progress updates to queue.
lib/features/media_store/data/media_transfer_queue_repository.dart Adds progress/resume persistence helpers; clears resume/progress on completion.
lib/core/services/media_store/s3_media_object_store.dart Implements multipart upload with resume validation + chunked Range download path.
lib/core/services/media_store/media_object_store.dart Extends store interface with progress/resume parameters.
lib/core/services/cloud_storage/s3/s3_api_client.dart Adds multipart endpoints, Range GET, and unsigned extra-header support.
lib/core/database/local_cache_database.dart Bumps local cache DB schema to v3 with progress columns + migration.
docs/superpowers/plans/2026-07-10-media-store-phase3.md Adds the Phase 3 implementation plan documentation.

Comment thread lib/core/services/media_store/s3_media_object_store.dart Outdated
Comment thread lib/features/media_store/data/media_transfer_queue_repository.dart
Comment thread lib/core/services/cloud_storage/s3/s3_api_client.dart
feat(media-store): backends, phase 4 (dropbox, google drive, icloud)
@ericgriffin ericgriffin self-assigned this Jul 11, 2026
@ericgriffin ericgriffin moved this from Backlog to In review in Submersion Release Tracker Jul 11, 2026
@ericgriffin ericgriffin merged commit 6de1226 into worktree-media-store-phase2 Jul 11, 2026
@ericgriffin ericgriffin deleted the worktree-media-store-phase3 branch July 11, 2026 04:41
@github-project-automation github-project-automation Bot moved this from In review to Done in Submersion Release Tracker Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants