feat(storage): Add full object checksum validation for appendable uploads#16219
Open
v-pratap wants to merge 19 commits into
Open
feat(storage): Add full object checksum validation for appendable uploads#16219v-pratap wants to merge 19 commits into
v-pratap wants to merge 19 commits into
Conversation
product-auto-label
Bot
added
the
api: storage
v-pratap
force-pushed
the
fastbyte-full-object-checksum-final
branch
from
3aad347 to
915cef5
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for CRC32C checksum validation during asynchronous uploads, specifically during finalization, resume, and flush operations. It updates the AsyncWriter interface and its underlying connection implementations to accept and verify expected checksums, and enhances HashFunction to support state retrieval and restoration. The review feedback highlights three critical issues: a compilation error in AsyncWriterConnectionImpl::Finalize caused by inconsistent return types in a lambda, and two data races in AsyncWriterConnectionResumed where impl_ is accessed concurrently without holding the appropriate mutex lock.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #16219 +/- ##
==========================================
- Coverage 92.27% 92.25% -0.02%
==========================================
Files 2214 2214
Lines 206394 206332 -62
==========================================
- Hits 190443 190350 -93
- Misses 15951 15982 +31 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kalragauri
requested changes
Jul 1, 2026
v-pratap
force-pushed
the
fastbyte-full-object-checksum-final
branch
from
006eeda to
9fca72d
Compare
kalragauri
requested changes
Jul 2, 2026
kalragauri
reviewed
Jul 3, 2026
kalragauri
approved these changes
Jul 6, 2026
| future<StatusOr<google::storage::v2::Object>> Finalize(AsyncToken token); | ||
| future<StatusOr<google::storage::v2::Object>> Finalize( | ||
| AsyncToken token, | ||
| absl::optional<Crc32cChecksumValue> const& expected_checksum); |
Contributor
There was a problem hiding this comment.
nit: https://google.github.io/styleguide/cppguide.html#Inputs_and_Outputs recommends avoiding absl::optional<T> const&.
If possible, we should either pass std::optional<T> by value (if it's a by-value input) or pass const T* (if it would otherwise be passed by reference).
… in appendable uploads
…overloads without default arguments
…ck slice calculations
…tate to avoid offset rewinding and MD5 errors
…heckpoints during a stream resume
…ose() phases as per revised design
Based on the revised design, full-object checksum verification is exclusively handled server-side at Finalize(), eliminating the need for continuous client-side incremental hashing across the stream. - Removed dynamic hash computation from CreateHashFunction for all async writers. - Removed dead client-side mismatch checking in AsyncWriterConnectionImpl and AsyncWriterConnectionResumed. - Consolidated appendable and standard hash creation to use PrecomputedHashFunction only. - Cleaned up obsolete tests asserting client-side hash mismatch failures.
v-pratap
force-pushed
the
fastbyte-full-object-checksum-final
branch
5 times, most recently
from
255bfa1 to
299f716
Compare
v-pratap
force-pushed
the
fastbyte-full-object-checksum-final
branch
from
299f716 to
b8eafc4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.