feat(blobmanager): add managed CAS backend via S3 Access Points#3121
Conversation
Introduce a new `AWS-S3-ACCESS-POINT` CAS backend that targets a single shared bucket via per-tenant S3 Access Points. Each upload/download mints scoped temporary credentials via `sts:AssumeRole` with a session policy narrowed to the tenant's AP ARN and key prefix, and a session name derived from the authenticated requesting org carried in `ctx` (`s3accesspoint.WithRequestingOrg`). Both upstream binaries pick up a new optional `blob_backends.s3_access_point` config block (`base_role_arn`, `region`, `session_duration`); when the block is absent the provider stays unregistered and behaviour is identical to before. The pod's ambient AWS identity (IRSA / instance profile / env vars) is used to call STS — no static credentials live in config. Per-tenant data (AP ARN, region override, key prefix) is stored as a JSON blob in the secrets manager and read via `FromCredentials`, so the existing `backend.Provider` interface is unchanged. Add `OrgID` to the CAS robotaccount JWT claims so artifact-cas can enrich its context with the requesting org before invoking the backend; existing providers ignore the key. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
… dev mode
Two related refinements to the AWS-S3-ACCESS-POINT provider.
1. The per-tenant key prefix is now derived at request time from the
authenticated requesting org carried in ctx via WithRequestingOrg,
rather than read from a `KeyPrefix` field in the secrets-manager
blob. The prefix and the AssumeRole `RoleSessionName` now share
their single source of truth, so a tampered Credentials blob can no
longer reroute a tenant's writes into another tenant's namespace.
The Credentials struct shrinks to {AccessPointARN, Region}. The
session policy and the bucket-level key both use `<orgUUID>` as the
prefix; the AP resource policy's Resource ARN must be
`${apARN}/object/<orgUUID>/*` to match.
2. Add a `dev_mode_use_ambient_credentials` Config flag (proto +
wire-plumbed in both binaries) that bypasses `sts:AssumeRole` and
routes S3 calls through whatever ambient AWS identity the SDK's
default credential chain produced. Local dev no longer requires an
IAM role + trust policy setup. The missing-org fail-closed check
still fires in dev mode so callers that forget WithRequestingOrg
surface the same bug locally that they would in production. A loud
warning is logged at startup. DEV ONLY — never enable in
multi-tenant deployments.
Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
…wire output For Managed=true CAS backends, replace Location with "managed by Chainloop" and Provider with "Chainloop" everywhere the controlplane emits a CASBackend outside its trust boundary: * API responses (bizCASBackendToPb), so `chainloop cas-backend ls` no longer prints the AWS account ID, region, or AP name. * Audit-log events on the NATS bus (CASBackendCreated, CASBackendUpdated, CASBackendDeleted, CASBackendPermanentDeleted, CASBackendStatusChanged), so downstream consumers can't surface the same details to tenants either. The DB and biz layer continue to carry the real ARN and provider ID unchanged, so PerformValidation, the platform reconciler, and any forensic join by CASBackendID still work. Two helpers (displayLocation, displayProvider) keep the sanitization rule in one place. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
AI Session Analysis
|
| Status | Policy | Material | Messages |
|---|---|---|---|
| ✅ Passed | ai-config-ai-agents-allowed |
ai-coding-session-234a03 |
- |
| ✅ Passed | ai-config-no-dangerous-commands |
ai-coding-session-234a03 |
- |
ai-config-no-secrets |
ai-coding-session-234a03 |
|
|
| ✅ Passed | ai-config-mcp-servers-allowed |
ai-coding-session-234a03 |
- |
Powered by Chainloop and Chainloop Trace
Kusari Analysis Results:
No pinned version dependency changes, code issues or exposed secrets detected! Note View full detailed analysis result for more information on the output and the checks that were run.
Found this helpful? Give it a 👍 or 👎 reaction! |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 34 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="app/controlplane/internal/conf/controlplane/config/v1/conf.proto">
<violation number="1" location="app/controlplane/internal/conf/controlplane/config/v1/conf.proto:152">
P2: Enforce `base_role_arn` when dev mode is disabled; the current schema allows invalid production config that will fail only at runtime.</violation>
</file>
<file name="app/controlplane/internal/service/cascredential.go">
<violation number="1" location="app/controlplane/internal/service/cascredential.go:152">
P1: Use the authenticated requesting org when minting CAS credentials; deriving `OrgID` from `backend.OrganizationID` can incorrectly scope managed S3 access-point sessions to backend ownership instead of caller identity.</violation>
</file>
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Fix all with cubic
Re-trigger cubic
Two follow-ups from the PR review on chainloop-dev#3121: * The CAS JWT minted by cascredential.go, attestation.go and casredirect.go now embeds OrgID from the authenticated caller (entities.CurrentOrg / robotAccount.OrgID) instead of backend.OrganizationID. For managed S3 Access Point backends this OrgID drives the AssumeRole session name and the AP-policy aws:userid match; deriving it from the resolved row would weaken the cross-tenant guarantee if a future bug ever let a caller resolve a backend they don't own. * The S3AccessPoint proto message now carries a buf.validate CEL constraint that requires base_role_arn when dev_mode_use_ambient_credentials is false, surfacing the misconfiguration at config-load time rather than at first upload. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
|
Kusari PR Analysis rerun based on - cfa4ff4 performed at: 2026-05-15T19:34:39Z - link to updated analysis |
A `go mod tidy` while developing the s3accesspoint provider regressed several deps: * go-git/v6 downgraded alpha.3 -> alpha.2 (CVE-2026-45022, commit signature spoofing) * go-billy/v5 downgraded 5.9.0 -> 5.8.0 (CVE-2026-44973 path traversal, CVE-2026-44740 symlink-loop DoS) * go-billy/v6 swapped to an older snapshot * go-git/v5 downgraded 5.19.0 -> 5.18.0 * unrelated olekukonko/* and golang.org/x/* version churn that broke CI's go-module tidy check Restoring go.mod and go.sum to match origin/main resolves both the Kusari CVE alerts and the CI failures. aws-sdk-go-v2/service/sts (needed by the s3accesspoint provider) is already an indirect at v1.41.9 on main, so no go.mod change is required for the new code to build. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
jiparis
force-pushed
the
jiparis/managed-cas-s3-access-points
branch
from
cfa4ff4 to
7457ed2
Compare
|
@jiparis how does this work? Does it create a backend DB entry automatically if the managed setup is configured in the instance in a similar way we do it with inline? |
The proto message and its YAML field describe configuration for *managed* CAS backends (provisioned and operated by Chainloop), not generic blob storage. Rename: * proto message `BlobBackends` -> `ManagedCASBackends` * proto field `blob_backends` -> `managed_cas_backends` in both controlplane and artifact-cas Bootstrap messages * matching Go field on the regenerated `*conf.Bootstrap` (`ManagedCasBackends`) and references in wire.go / wire_gen.go * commented-out example block in both `config.devel.yaml` No behavioural change; the only deployments that read this block today are local-dev configs (gitignored config.local.yaml) which have been updated separately. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
CASBackendService.Create previously accepted any provider ID present in the loader's provider map, including AWS-S3-ACCESS-POINT. A sufficiently determined user could craft a Create request that half-provisioned a managed row pointing at an AP ARN they don't own, bypassing the platform reconciler's trust boundary. Add an explicit isManagedOnlyProvider() guard at the front of Create so the public RPC fails fast with `managed CAS backends cannot be created via this API`. The platform reconciler still creates managed rows by calling biz.CASBackendUseCase.Create directly, which is unaffected. Update/SoftDelete are already guarded against managed rows in the biz layer. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
Follow-ups from the PR review on chainloop-dev#3121: * JWT OrgID claim is restored to backend.OrganizationID (instead of the authenticated caller's currentOrg). For cross-org downloads (FindCASMappingForDownloadByUser may return a backend from any org the caller belongs to) the JWT must address the AP that actually owns the data; authorization is enforced earlier by the mapping lookup. Inline comments at those call sites were dropped — the reasoning lives in this commit and the design doc. * CASCredsOpts.OrgID is now uuid.UUID instead of string, matching every other org-id field in biz; the JWT boundary stringifies once and treats uuid.Nil as "no managed binding". * The s3accesspoint-specific ctx-key helper moves up to the pkg/blobmanager umbrella as backend.WithRequestingOrg / backend.RequestingOrgFromContext. Generic primitive, not tied to any one provider, and reusable for future managed backends. * Setting the requesting-org on ctx is now done by an auth-boundary middleware in app/artifact-cas/internal/server/auth.go (requestingOrgMiddleware for unary gRPC, jwtAuthFunc enrichment for stream gRPC, requestingOrgHTTPMiddleware for the download HTTP handler). The service layer no longer carries loadBackendForClaims; all four CAS service entry points are back to plain loadBackend. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
…ecret Move BaseRoleARN and Region from the deployment-level config into the per-tenant Credentials blob, drop SessionDuration in favour of a 1h constant, and read the dev-mode bypass from CHAINLOOP_S3_ACCESS_POINT_DEV_MODE instead of config. The ManagedCASBackends proto blocks, the corresponding wire plumbing, and the loader Options surface are all gone. The provider is now registered unconditionally; on-prem deployments without managed CAS simply never have managed rows. A single chainloop install can also serve tenants across multiple AWS accounts without a config change since BaseRoleARN is per-secret. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
Row creation in the database is out of scope for this PR. Here you can find just the provider implementation for S3 accesspoints. In practice, it will expect an AP and an AP resource policy generated for every organization. The creation of those resources are not for this PR since here we are not yet provisioning backends. |
Collapses protobuf bindings in PR diffs and excludes them from linguist language stats. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2 Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
jiparis
force-pushed
the
jiparis/managed-cas-s3-access-points
branch
from
000884f to
462db40
Compare
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
|
You're iterating quickly on this pull request. To help protect your rate limits, cubic has paused automatic reviews on new pushes for now—when you're ready for another review, comment |
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
Promote orgID to a required field on the CAS JWT alongside backendType, secretID, audience. Plumb it through CASClient.Upload/Download and the dispatcher's loadInputs so all call sites supply the org explicitly rather than relying on uuid.Nil as an "absent" sentinel. Non-managed providers ignore the claim; managed providers (AWS-S3-ACCESS-POINT) keep using it to scope per-tenant STS sessions. The token also gains audit traceability for free. Assisted-by: Claude Code Signed-off-by: Jose I. Paris <jiparis@chainloop.dev> Chainloop-Trace-Sessions: 234a03ed-b238-4506-95f0-235242842db2
2 participants
Summary
AWS-S3-ACCESS-POINTCAS backend that targets a single shared bucket via per-tenant S3 Access Points. Each request mints a scoped session viasts:AssumeRolewith a session policy andRoleSessionNamederived from the authenticated requesting org carried inctx.AccessPointARN,Region,BaseRoleARN) live in the per-tenant secret blob; there is no deployment-level config block. The provider is registered unconditionally, so on-prem deployments without managed CAS are unaffected — they simply never have managed rows.aws:userid.org-idclaim). An auth middleware inartifact-casreads the claim and enriches the requestctxviablobmanager.WithRequestingOrgbefore any service handler runs, so callers cannot accidentally skip the binding. Non-managed providers ignore the key.sts:AssumeRole, use the SDK's default credential chain) is opt-in via theCHAINLOOP_S3_ACCESS_POINT_DEV_MODEenv var.Managed=truerows, redacts AWS implementation details from any wire output: the AP ARN (Location) becomes"managed by Chainloop"and the provider ID (Provider) becomes"Chainloop"in both API responses and audit-event payloads. The DB and biz layer keep the real values.CASBackend.CreateRPC rejects the managed provider; managed rows are provisioned by the platform-side reconciler.AI Assistance
This change was developed with Claude Code; per-commit
Assisted-by:trailers record the specific commits.Closes #3114