feat(media-store): large objects, phase 3 (multipart, resume, video)#557
Merged
ericgriffin merged 19 commits intoJul 11, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Implements Phase 3 of the Media Store “large objects” work: multipart/resumable S3 uploads and chunked (Range) downloads to keep memory bounded, lifts video eligibility through the upload pipeline, threads resume/progress through the per-device transfer queue, and adds a store-backed fallback for cross-device video playback plus determinate progress UI in Transfers. Test infrastructure is expanded with a reusable in-memory Fake S3 server to exercise the real SigV4 path for multipart and Range behaviors.
Changes:
- Extend
MediaObjectStorewith optional progress + adapter-opaque resume hooks; propagate through the upload pipeline and queue persistence. - Add S3 multipart + Range support (
S3ApiClient,S3MediaObjectStore) including resume validation viaListParts, and chunked downloads. - Improve UX/behavior around videos and transfers (video playback fallback; determinate progress bars) with expanded coverage using a shared
FakeS3Server.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/helpers/in_memory_media_object_store.dart | Adds progress/resume hook support to the in-memory MediaObjectStore test double. |
| test/helpers/fake_s3_server.dart | New reusable in-memory S3 protocol fake (multipart + Range) for tests. |
| test/features/settings/presentation/s3_config_page_test.dart | Updates S3ApiClient test double to Fake for forward-compatible stubbing. |
| test/features/media_store/transfers_page_test.dart | Adds widget test for determinate progress rendering. |
| test/features/media_store/support/fake_local_file_resolver.dart | Allows separate thumbnail data to model gallery video posters in tests. |
| test/features/media_store/resolved_file_path_store_fallback_test.dart | New provider test covering store-backed video playback fallback behavior. |
| test/features/media_store/media_upload_pipeline_test.dart | Expands pipeline tests for video uploads, resume/progress plumbing, and poster thumb handling. |
| test/features/media_store/media_transfer_queue_repository_test.dart | Adds migration and resume/progress persistence tests for queue schema v3. |
| test/features/media_store/media_store_end_to_end_test.dart | Adds cross-device video end-to-end coverage (upload on A, play/resolve on B). |
| test/core/services/media_store/s3_media_object_store_test.dart | Refactors to use FakeS3Server and adds multipart/resume/chunked-download tests. |
| test/core/services/media_store/media_object_store_contract.dart | Extends store contract to require progress callbacks reaching full size. |
| test/core/services/cloud_storage/s3/s3_multipart_test.dart | New unit tests for multipart lifecycle + Range GET in S3ApiClient. |
| test/core/services/cloud_storage/s3_storage_provider_test.dart | Updates S3ApiClient test double to Fake to avoid breakage from new methods. |
| lib/features/media/presentation/providers/resolved_asset_providers.dart | Adds store-backed fallback path for video playback when local file is missing. |
| lib/features/media_store/presentation/pages/transfers_page.dart | Renders determinate/indeterminate progress bars for transferring queue entries. |
| lib/features/media_store/data/media_upload_pipeline.dart | Removes video ineligibility; wires resume state + progress updates to queue. |
| lib/features/media_store/data/media_transfer_queue_repository.dart | Adds progress/resume persistence helpers; clears resume/progress on completion. |
| lib/core/services/media_store/s3_media_object_store.dart | Implements multipart upload with resume validation + chunked Range download path. |
| lib/core/services/media_store/media_object_store.dart | Extends store interface with progress/resume parameters. |
| lib/core/services/cloud_storage/s3/s3_api_client.dart | Adds multipart endpoints, Range GET, and unsigned extra-header support. |
| lib/core/database/local_cache_database.dart | Bumps local cache DB schema to v3 with progress columns + migration. |
| docs/superpowers/plans/2026-07-10-media-store-phase3.md | Adds the Phase 3 implementation plan documentation. |
feat(media-store): backends, phase 4 (dropbox, google drive, icloud)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3 (large objects) of the Media Store, per
docs/superpowers/specs/2026-07-10-s3-media-storage-design.mdsections 8, 9, 17; implementation plandocs/superpowers/plans/2026-07-10-media-store-phase3.md.STACKED on #556, which stacks on #550 - merge bottom-up (#550, #556, then this); GitHub retargets each PR as its base branch is deleted on merge.
Videos are now first-class: uploads above 8 MiB go through S3 multipart with per-part resume state persisted on the queue row (a transfer killed at part N restarts at part N+1, proven by request counting against a protocol-level fake S3 server), downloads stream Range chunks to disk so memory stays bounded, and a video attached on device A plays from the store on device B through a
resolvedFilePathProviderfallback - the single seam the video player already consumes.Changes
MediaObjectStore.putFile/getFilegain optional progress and adapter-opaque resume parameters (the extension Phase 1 reserved)S3ApiClient:createMultipartUpload/uploadPart/completeMultipartUpload/abortMultipartUpload/listParts/getObjectRange, plus unsigned extra-header support for Range requestsS3MediaObjectStore: threshold-switched putFile (single-shot <= 8 MiB, multipart above) with resume validation via ListParts (stale/mismatched sessions abort and restart fresh); chunked Range-GET downloads with progressprogress_bytes/total_bytescolumns,updateResumeState/updateProgress;markDoneclears resume/progress whilemarkFailed/retry/deferpreserve themvideosOnCellulargate activates with zero changes); resume state and progress thread through the queue row; gallery-video poster thumbs upload via the existing thumb stepresolvedFilePathProviderorder is now gallery asset -> existinglocalPath-> store download into the content-addressed cache -> null;photo_viewer_pageuntouched (its loading state covers the download)FakeS3Server(multipart sessions, Range, part-failure injection) exercising the real SigV4 signing path; existing S3ApiClient test doubles converted toFake-based so future client additions cannot break themTest Plan
flutter testpasses (targeted per repo practice: media_store/core/cloud_storage 284, media 566, sync 422, settings 608; includes multipart round-trip, kill-and-resume with parts 1-2 not re-uploaded, stale-resume restart, chunked-download progress, cross-device video e2e with byte-identical playback and poster thumbs)flutter analyzepasses (no issues)Screenshots
Transfers view with a determinate progress bar to follow with the manual pass.
Documented deviation (also in the plan): non-gallery (localFile) video thumbnails are deferred to Phase 5 with placeholder degradation; gallery videos - the dominant path - get posters through photo_manager. No native thumbnailer dependency was added.