fix: [CI-22408]: allow PLUGIN_ACL=none to disable x-amz-acl on S3 cache uploads#181
Open
hemanthmantri wants to merge 1 commit into
Open
fix: [CI-22408]: allow PLUGIN_ACL=none to disable x-amz-acl on S3 cache uploads#181hemanthmantri wants to merge 1 commit into
hemanthmantri wants to merge 1 commit into
Conversation
…he uploads Customers writing the cache to S3 buckets with Object Ownership = BucketOwnerEnforced (ACLs disabled) hit AccessControlListNotSupported on CreateMultipartUpload because the plugin always sends x-amz-acl when PLUGIN_ACL is non-empty, and the upstream Harness CI manager injects PLUGIN_ACL=private by default (see CI-10547). Removing PLUGIN_ACL from the pipeline YAML alone does not clear the upstream injection. Treat PLUGIN_ACL values "none", "disabled", or "off" (case-insensitive, trimmed) as a sentinel meaning "do not send the x-amz-acl header" so customers can override the upstream default with a step env var. Real canned ACL values continue to flow through unchanged. Fixes CI-22408.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes CI-22408
Proposed Changes
storage/backend/s3/s3.go: addisACLDisabledhelper. InNew, whenPLUGIN_ACLmatches the sentinel valuesnone,disabled, oroff(case-insensitive, trimmed), leavebackend.aclempty so thePutpath does not attachx-amz-aclto the S3 request.main.go,DOCS.md,README.md: document the new sentinel values on the--acl/PLUGIN_ACLflag.storage/backend/s3/s3_unit_test.go: addTestIsACLDisabledtable test covering empty, sentinel, and real canned-ACL values.CHANGELOG.md: add Unreleased entry.Description
When a customer's S3 bucket has Object Ownership set to
BucketOwnerEnforced(the AWS default for new buckets since April 2023), AWS rejects any canned ACL header other thanbucket-owner-full-controlwith HTTP 400AccessControlListNotSupported: The bucket does not allow ACLs. The Harness Cache Intelligence cache step hit this on CreateMultipartUpload because:x-amz-aclwheneverPLUGIN_ACLis non-empty.PLUGIN_ACL=privateby default (originally added in harness-core PR #56087 to address CI-10547 for OVH), so simply removingaclfrom the pipeline YAML does not stop the header from being sent — the upstream default still wins unless the step explicitly overrides it.This change gives customers an in-pipeline opt-out: setting
PLUGIN_ACL=none(ordisabled/off) at the step level overrides the upstreamprivateinjection and tells the plugin not to send the header at all, which is whatBucketOwnerEnforcedrequires.Existing canned-ACL values (
private,public-read,bucket-owner-full-control, ...) continue to be forwarded to S3 unchanged.Note: customers can also unblock themselves today, before this PR ships, by setting
PLUGIN_ACL=bucket-owner-full-controlat the step level — AWS documents that value as accepted onBucketOwnerEnforcedbuckets. The sentinel-based opt-out introduced here is the cleaner long-term option.Existing pattern in this same file (S3 Express directory buckets — CI-13695, CI-21463, CI-22058) already skips the ACL header conditionally; this change extends that pattern with a customer-controllable knob for the regular-bucket case.
I considered two alternative approaches and rejected both:
AccessControlListNotSupported: the upload uses non-seekable pipe readers, so the body cannot be replayed safely.GetBucketOwnershipControlsat startup: requires an additional IAM permission that many cross-account roles don't grant.Checklist
Unreleasedsection of CHANGELOG.