From c8bfbd269516f5fc3edde6dcebe832c73f99631d Mon Sep 17 00:00:00 2001 From: Dimitri John Ledkov Date: Fri, 7 Aug 2026 00:20:20 +0000 Subject: [PATCH 1/3] Read CA bundle hash from the package stamp file, not a pinned value MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CertificateAudit pinned the SHA-256 of /etc/ssl/certs/ca-certificates.crt in the datastream, so every upstream ca-certificates roll turned a correctly-updated image into a STIG failure until the daily update-ca-cert workflow re-pinned the value and the PR merged. The ca-certificates package already ships the digest next to the bundle in /etc/ssl/certs/.ca-certificates.crt.sha256 (sha256sum format), so read the expected value from there instead: - new textfilecontent54 object obj:4 extracts the digest from the stamp file via `^([0-9a-fA-F]{64})[ \t]+\*?ca-certificates\.crt$` - new local_variable var:1 exposes that subexpression - ste:1 compares filehash58 against var:1 rather than a literal - new test tst:4 requires the stamp to exist and parse, so a missing or malformed stamp fails the rule instead of passing vacuously Note this is drift detection, not tamper-evidence: whoever can rewrite the bundle can rewrite the stamp beside it. What it still catches is a bundle modified outside the package (the cabundle-tampered fixture), since the stamp is package-owned. Corroborating the stamp against apk package integrity would be the stronger control. Verified with an image simulating an upstream roll (bundle changed, stamp regenerated): the released datastream fails it, this one passes, and both still fail the tampered fixture. Test coverage for the new dimension, in the offline matrix: fail_wrong_stamp_digest, fail_malformed_stamp, fail_missing_stamp. These needed overlay ops for replacing and removing a base entry; Apply's writer already handled removal. update-ca-cert.yaml no longer has a hash to re-pin. Its extract+sed steps are replaced by a `sha256sum -c` guard that fails the run if a future base image drops the stamp or ships one disagreeing with the bundle — the premise the pin used to guard. Fixture and test-pin re-pinning are unchanged. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/update-ca-cert.yaml | 145 ++++++++---------- .../ssg/content/ssg-chainguard-gpos-ds.xml | 31 +++- tests/e2e/fixtures/baseline-clean/Dockerfile | 3 +- .../e2e/fixtures/cabundle-tampered/Dockerfile | 7 +- .../fixtures/cabundle-tampered/expected.txt | 3 +- .../oscap-offline/internal/overlay/overlay.go | 26 ++++ .../internal/overlay/overlay_test.go | 56 +++++++ .../internal/scan/fixtures_test.go | 53 ++++++- 8 files changed, 223 insertions(+), 101 deletions(-) diff --git a/.github/workflows/update-ca-cert.yaml b/.github/workflows/update-ca-cert.yaml index 279027c..98938bd 100644 --- a/.github/workflows/update-ca-cert.yaml +++ b/.github/workflows/update-ca-cert.yaml @@ -1,4 +1,4 @@ -name: Update CA cert hash in datastream +name: Verify CA bundle stamp and re-pin fixture base images on: workflow_dispatch: schedule: @@ -8,7 +8,6 @@ on: - main paths: - ".github/workflows/update-ca-cert.yaml" - - "gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml" - "tests/e2e/fixtures/*/Dockerfile" - "tests/oscap-offline/internal/scan/fixtures_test.go" concurrency: @@ -28,7 +27,6 @@ jobs: issues: write env: IMAGE_REF: cgr.dev/chainguard/wolfi-base:latest - DATASTREAM_PATH: gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml TESTS_PATH: tests FIXTURES_GLOB: tests/e2e/fixtures/*/Dockerfile TEST_PIN_FILE: tests/oscap-offline/internal/scan/fixtures_test.go @@ -69,7 +67,7 @@ jobs: echo "::error::Image signature verification failed" exit 1 } - - name: Extract CA certificate SHA + - name: Verify CA bundle stamp file id: ca env: STEPS_IMAGE_OUTPUTS_FULL_REF: ${{ steps.image.outputs.full_ref }} @@ -77,59 +75,38 @@ jobs: run: | set -euo pipefail - SHA=$(crane export "${STEPS_IMAGE_OUTPUTS_FULL_REF}" - | \ - tar -xO etc/ssl/certs/ca-certificates.crt | \ - sha256sum | cut -d' ' -f1) + # CertificateAudit no longer pins a CA bundle hash in the datastream: + # the OVAL reads the expected digest out of the stamp file the + # ca-certificates package ships next to the bundle + # (/etc/ssl/certs/.ca-certificates.crt.sha256, sha256sum format) and + # compares the bundle against it. Nothing in this repo needs updating + # when the upstream bundle rolls. + # + # This step guards the premise that replaced the pin: if a future + # base image drops the stamp file, renames it, or ships one that + # disagrees with the bundle, the rule would start failing on clean + # images. Fail loudly here — daily — instead of finding out via a red + # E2E run. + + workdir=$(mktemp -d) + crane export "${STEPS_IMAGE_OUTPUTS_FULL_REF}" - | \ + tar -C "${workdir}" -x etc/ssl/certs/ca-certificates.crt etc/ssl/certs/.ca-certificates.crt.sha256 + + if ! (cd "${workdir}/etc/ssl/certs" && sha256sum -c .ca-certificates.crt.sha256); then + echo "::error::CA bundle does not match its stamp file in ${IMAGE_REF}; CertificateAudit will fail on clean images" + exit 1 + fi + SHA=$(sha256sum "${workdir}/etc/ssl/certs/ca-certificates.crt" | cut -d' ' -f1) echo "sha=${SHA}" >> "$GITHUB_OUTPUT" cat >> "$GITHUB_STEP_SUMMARY" <]*hash>)[^<]*(]*hash>)|\1${CA_CERT_SHA}\2|g" "${DATASTREAM_PATH}" - - # Check if file changed - ds_changed=false - if diff -q "${DATASTREAM_PATH}.bak" "${DATASTREAM_PATH}" > /dev/null; then - echo "No datastream changes needed - SHA already up to date" - echo "datastream_changed=false" >> "$GITHUB_OUTPUT" - rm "${DATASTREAM_PATH}.bak" - else - ds_changed=true - echo "Updated datastream with new SHA: ${CA_CERT_SHA}" - echo "datastream_changed=true" >> "$GITHUB_OUTPUT" - rm "${DATASTREAM_PATH}.bak" - fi - - # Show what was updated for the summary - echo "### Datastream Update Summary" >> "$GITHUB_STEP_SUMMARY" - echo "- **File**: \`${DATASTREAM_PATH}\`" >> "$GITHUB_STEP_SUMMARY" - echo "- **New SHA**: \`${CA_CERT_SHA}\`" >> "$GITHUB_STEP_SUMMARY" - if [ "${ds_changed}" == "true" ]; then - echo "- **Status**: Updated" >> "$GITHUB_STEP_SUMMARY" - else - echo "- **Status**: Already up-to-date" >> "$GITHUB_STEP_SUMMARY" - fi - name: Re-pin E2E fixture base-image digests id: fixtures env: @@ -138,10 +115,10 @@ jobs: set -euo pipefail # Re-pin every `FROM @sha256:` in tests/e2e/fixtures/*/Dockerfile - # to the digest that matches the CA bundle hash computed above. This keeps - # the datastream hash and the fixture base images atomically in sync, so - # the CertificateAudit assertion in e.g. baseline-clean can't flake due to - # drift between the two values. + # to the current digest, verified above to carry a self-consistent CA + # bundle and stamp file. The fixtures no longer have to track a hash in + # the datastream, so this is ordinary base-image freshness: it keeps the + # E2E fixtures scanning a recent image rather than an ageing pin. # # For each Dockerfile: # - Parse the existing FROM line's `image` (everything before `@sha256:`). @@ -155,10 +132,9 @@ jobs: # # The offline harness (tests/oscap-offline) reads its pinned base image # from tests/e2e/fixtures/baseline-clean/Dockerfile, so re-pinning that - # fixture here also keeps the offline CertificateAudit pass fixture in - # lockstep with the datastream hash. The one remaining copy of the - # digest — the `pinned` constant in TestParseWolfiBaseRef — is - # re-pinned by the next step. + # fixture here also moves the offline CertificateAudit fixtures. The one + # remaining copy of the digest — the `pinned` constant in + # TestParseWolfiBaseRef — is re-pinned by the next step. fixtures_changed=false updated_files=() @@ -266,12 +242,11 @@ jobs: - name: Aggregate change status id: changed env: - DS_CHANGED: ${{ steps.update.outputs.datastream_changed }} FX_CHANGED: ${{ steps.fixtures.outputs.fixtures_changed }} TP_CHANGED: ${{ steps.testpin.outputs.test_pin_changed }} run: | set -euo pipefail - if [ "${DS_CHANGED}" = "true" ] || [ "${FX_CHANGED}" = "true" ] || [ "${TP_CHANGED}" = "true" ]; then + if [ "${FX_CHANGED}" = "true" ] || [ "${TP_CHANGED}" = "true" ]; then echo "changed=true" >> "$GITHUB_OUTPUT" else echo "changed=false" >> "$GITHUB_OUTPUT" @@ -286,14 +261,16 @@ jobs: with: token: ${{ steps.octo-sts.outputs.token }} commit-message: | - chore(oscap): re-pin CA bundle hash and fixture base-image digests + chore(oscap): re-pin fixture base-image digests - Atomically updates the CA bundle SHA in the OSCAP datastream, the - digest-pinned FROM lines in tests/e2e/fixtures/*/Dockerfile, and the - pinned wolfi-base digest in the offline harness test - (TestParseWolfiBaseRef) so the three values can never drift out of - sync (which would flake the CertificateAudit E2E assertions or - break the offline unit tests). + Re-pins the digest-pinned FROM lines in tests/e2e/fixtures/*/Dockerfile + and the pinned wolfi-base digest in the offline harness test + (TestParseWolfiBaseRef) together, so the two can never drift out of + sync and break the offline unit tests. + + The base image was checked first: its CA bundle matches the digest in + /etc/ssl/certs/.ca-certificates.crt.sha256, which is what + CertificateAudit compares against. Image: ${{ env.IMAGE_REF }} Digest: ${{ steps.image.outputs.digest }} @@ -302,32 +279,34 @@ jobs: Signed-off-by: github-actions[bot] branch: update-ca-cert-${{ steps.ca.outputs.sha }} delete-branch: true - title: "chore(oscap): re-pin CA bundle hash and fixture base-image digests" + title: "chore(oscap): re-pin fixture base-image digests" body: | - ## CA Certificate + Fixture Base-Image Update + ## Fixture Base-Image Update - Atomically re-pins three values that must stay in lockstep: + Re-pins two values that must stay in lockstep: - 1. The `` under `oval:org.CABundleHash:ste:1` in the OSCAP - datastream (`${{ env.DATASTREAM_PATH }}`). - 2. The `FROM cgr.dev/chainguard/wolfi-base:latest@sha256:...` line in + 1. The `FROM cgr.dev/chainguard/wolfi-base:latest@sha256:...` line in every `tests/e2e/fixtures/*/Dockerfile`. - 3. The `pinned` wolfi-base digest constant asserted by + 2. The `pinned` wolfi-base digest constant asserted by `TestParseWolfiBaseRef` in `${{ env.TEST_PIN_FILE }}`. - If these drift (e.g. Dependabot bumps the fixture digest before this - workflow refreshes the datastream hash, or vice versa), the - `baseline-clean` / `cabundle-tampered` E2E CertificateAudit check - fails because the fixture's CA bundle no longer matches the hash the - datastream asserts, and the offline harness unit tests break against - the stale test pin. This workflow is the authoritative update point - for all three values together; `TestParseWolfiBaseRef` was run - against the updated tree before this PR was opened. + If these drift (e.g. Dependabot bumps the fixture digest without the + test pin following), the offline harness unit tests break. This + workflow is the authoritative update point for both together; + `TestParseWolfiBaseRef` was run against the updated tree before this + PR was opened. + + CertificateAudit itself no longer pins a CA bundle hash — the OVAL + reads the expected digest from the ca-certificates stamp file + (`/etc/ssl/certs/.ca-certificates.crt.sha256`) inside the scanned + image. The `Verify CA bundle stamp file` step above asserts that + stamp is present and agrees with the bundle, so a base image that + dropped it would fail this run rather than silently red the E2E + CertificateAudit assertions. - **Image**: `${{ env.IMAGE_REF }}` - **Digest**: `${{ steps.image.outputs.digest }}` - - **New CA SHA256**: `${{ steps.ca.outputs.sha }}` - - **Datastream changed**: `${{ steps.update.outputs.datastream_changed }}` + - **CA SHA256**: `${{ steps.ca.outputs.sha }}` - **Fixtures changed**: `${{ steps.fixtures.outputs.fixtures_changed }}` - **Test pin changed**: `${{ steps.testpin.outputs.test_pin_changed }}` labels: | diff --git a/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml b/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml index 85e3212..b7f3578 100644 --- a/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml +++ b/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml @@ -6224,9 +6224,11 @@ or certificate store maintains a list of trusted root certificates. Script Verification: To manually verify, Ensure the ca-certificates package has not been - modified using apk audit or by ensuring the sha256 value matches the trusted - value, and that the SSL_CERT_FILE environment variable configured on the - image or container is set to /etc/ssl/certs/ca-certificates.crt. + modified using apk audit, or by running 'cd /etc/ssl/certs && + sha256sum -c .ca-certificates.crt.sha256' so the bundle matches the digest + the package recorded for it, and that the SSL_CERT_FILE environment variable + configured on the image or container is set to + /etc/ssl/certs/ca-certificates.crt. CCI-004909 @@ -6682,14 +6684,15 @@ Validate SHA-256 hash of CA bundle - Passes only if the CA bundle exists, has the correct hash, and the SSL_CERT_FILE environment variable configured on the image or container is set to /etc/ssl/certs/ca-certificates.crt. + Passes only if the CA bundle exists, its SHA-256 matches the digest recorded in the ca-certificates package stamp file /etc/ssl/certs/.ca-certificates.crt.sha256, and the SSL_CERT_FILE environment variable configured on the image or container is set to /etc/ssl/certs/ca-certificates.crt. Chainguard - + + @@ -6698,10 +6701,13 @@ - + + + + @@ -6719,16 +6725,27 @@ SSL_CERT_FILE + + /etc/ssl/certs + .ca-certificates.crt.sha256 + ^([0-9a-fA-F]{64})[ \t]+\*?ca-certificates\.crt$ + 1 + SHA-256 - 61efbd6d3f829f71039c57b29dd37d15ac7f33c4ece861aaef8c7d7a519cd1d9 + /etc/ssl/certs/ca-certificates.crt + + + + + diff --git a/tests/e2e/fixtures/baseline-clean/Dockerfile b/tests/e2e/fixtures/baseline-clean/Dockerfile index b9644d4..2fa1b9d 100644 --- a/tests/e2e/fixtures/baseline-clean/Dockerfile +++ b/tests/e2e/fixtures/baseline-clean/Dockerfile @@ -9,7 +9,8 @@ # - LibraryPermissions: /usr/lib owned by root:root # - VarLogPermissions: /var/log owned by root:root # - NoUsers: no interactive user accounts beyond the image default -# - CertificateAudit: /etc/ssl/certs/ca-certificates.crt matches the pinned SHA-256 +# - CertificateAudit: /etc/ssl/certs/ca-certificates.crt matches the SHA-256 in +# /etc/ssl/certs/.ca-certificates.crt.sha256 # # Expected result: a clean scan with no failures attributable to these rules. FROM cgr.dev/chainguard/wolfi-base:latest@sha256:30f03343947c7ae3581fda727a6e2aa7b8ce7009b7bfc2ab8d5c9483ace5812f diff --git a/tests/e2e/fixtures/cabundle-tampered/Dockerfile b/tests/e2e/fixtures/cabundle-tampered/Dockerfile index 088a664..678ce40 100644 --- a/tests/e2e/fixtures/cabundle-tampered/Dockerfile +++ b/tests/e2e/fixtures/cabundle-tampered/Dockerfile @@ -3,9 +3,10 @@ # # CertificateAudit violation fixture. # -# Appends a bogus trust anchor to /etc/ssl/certs/ca-certificates.crt so -# the SHA-256 of the baked bundle diverges from the pinned value the -# CertificateAudit OVAL check expects. The rule must FAIL. +# Appends a bogus trust anchor to /etc/ssl/certs/ca-certificates.crt without +# touching the ca-certificates stamp file, so the SHA-256 of the baked bundle +# diverges from the digest recorded in /etc/ssl/certs/.ca-certificates.crt.sha256 +# that the CertificateAudit OVAL check compares it against. The rule must FAIL. FROM cgr.dev/chainguard/wolfi-base:latest@sha256:30f03343947c7ae3581fda727a6e2aa7b8ce7009b7bfc2ab8d5c9483ace5812f LABEL dev.orbstack.add-ca-certificates=false diff --git a/tests/e2e/fixtures/cabundle-tampered/expected.txt b/tests/e2e/fixtures/cabundle-tampered/expected.txt index 3be96e8..fdf04ea 100644 --- a/tests/e2e/fixtures/cabundle-tampered/expected.txt +++ b/tests/e2e/fixtures/cabundle-tampered/expected.txt @@ -1,5 +1,6 @@ # Fixture: cabundle-tampered # # /etc/ssl/certs/ca-certificates.crt has extra content appended, so its -# SHA-256 no longer matches the pinned hash. CertificateAudit must FAIL. +# SHA-256 no longer matches the digest recorded in the package's stamp file +# /etc/ssl/certs/.ca-certificates.crt.sha256. CertificateAudit must FAIL. xccdf_mil.disa.stig_rule_SV-263659r982563_rule=fail diff --git a/tests/oscap-offline/internal/overlay/overlay.go b/tests/oscap-offline/internal/overlay/overlay.go index 32529cf..f350b2f 100644 --- a/tests/oscap-offline/internal/overlay/overlay.go +++ b/tests/oscap-offline/internal/overlay/overlay.go @@ -86,6 +86,32 @@ func AppendFile(path string, extra []byte) Op { } } +// ReplaceFile overwrites an existing entry's content, leaving its header +// (mode, ownership, position in the tar) intact. The path must already exist +// and be a regular file, otherwise Apply returns a wrapped ErrNotFound or +// ErrNotRegular. +func ReplaceFile(path string, content []byte) Op { + return func(p *plan) { + e := p.requireReg(path) + if e == nil { + return + } + e.data = bytes.Clone(content) + e.hdr.Size = int64(len(e.data)) + } +} + +// RemoveFile drops an existing entry from the produced tar. The path must +// already exist, otherwise Apply returns a wrapped ErrNotFound. +func RemoveFile(path string) Op { + return func(p *plan) { + if p.require(path) == nil { + return + } + delete(p.byName, path) + } +} + // AddFile adds a new regular-file entry after the base entries. The path must // not already exist, otherwise Apply returns a wrapped ErrExists. func AddFile(path string, content []byte, mode int64, uid, gid int) Op { diff --git a/tests/oscap-offline/internal/overlay/overlay_test.go b/tests/oscap-offline/internal/overlay/overlay_test.go index bbbe792..85bedc0 100644 --- a/tests/oscap-offline/internal/overlay/overlay_test.go +++ b/tests/oscap-offline/internal/overlay/overlay_test.go @@ -163,6 +163,62 @@ func TestAppendFileRejectsNonRegular(t *testing.T) { } } +func TestReplaceFile(t *testing.T) { + t.Parallel() + + base := buildTar(t, map[string]fileSpec{ + "etc/f": {content: randBytes(40), mode: 0o444, uid: 3, gid: 5}, + }) + + content := randBytes(12) + got := apply(t, base, ReplaceFile("etc/f", content)) + _, byName := readEntries(t, got) + + want := fileSpec{content: content, mode: 0o444, uid: 3, gid: 5} + if diff := cmp.Diff(want, byName["etc/f"], cmp.AllowUnexported(fileSpec{})); diff != "" { + t.Errorf("ReplaceFile changed more than content (-want,+got):\n%s", diff) + } +} + +func TestRemoveFile(t *testing.T) { + t.Parallel() + + base := buildTar(t, map[string]fileSpec{ + "etc/keep": {content: []byte("k"), mode: 0o644, uid: 0, gid: 0}, + "etc/drop": {content: []byte("d"), mode: 0o644, uid: 0, gid: 0}, + }) + + got := apply(t, base, RemoveFile("etc/drop")) + order, _ := readEntries(t, got) + + if diff := cmp.Diff([]string{"etc/keep"}, order); diff != "" { + t.Errorf("order mismatch after RemoveFile (-want,+got):\n%s", diff) + } +} + +// TestReplaceAndRemoveMissingPath proves both ops surface ErrNotFound for a +// path absent from the base rather than silently no-op'ing. +func TestReplaceAndRemoveMissingPath(t *testing.T) { + t.Parallel() + + base := buildTar(t, map[string]fileSpec{ + "etc/f": {content: []byte("x"), mode: 0o644, uid: 0, gid: 0}, + }) + + for name, op := range map[string]Op{ + "ReplaceFile": ReplaceFile("etc/missing", []byte("y")), + "RemoveFile": RemoveFile("etc/missing"), + } { + t.Run(name, func(t *testing.T) { + t.Parallel() + var out bytes.Buffer + if err := Apply(bytes.NewReader(base), []Op{op}, &out); !errors.Is(err, ErrNotFound) { + t.Fatalf("Apply error = %v, want errors.Is ErrNotFound", err) + } + }) + } +} + func TestChown(t *testing.T) { t.Parallel() diff --git a/tests/oscap-offline/internal/scan/fixtures_test.go b/tests/oscap-offline/internal/scan/fixtures_test.go index bbc724b..022f877 100644 --- a/tests/oscap-offline/internal/scan/fixtures_test.go +++ b/tests/oscap-offline/internal/scan/fixtures_test.go @@ -304,9 +304,24 @@ var plainHTTPRepo = []byte("http://insecure.example.com/alpine\n") var commentedAndHTTPSRepos = []byte("# http://insecure.example.com/alpine\nhttps://packages.example.com/alpine\n") // caTamper appends content to the CA bundle so its SHA-256 diverges from the -// datastream's pinned hash, failing the filehash58 state. +// digest recorded in the stamp file, failing the filehash58 state. var caTamper = []byte("\n# tamper\n-----BEGIN CERTIFICATE-----\nTAMPERED\n-----END CERTIFICATE-----\n") +// caStampPath is the sha256sum-format stamp file the ca-certificates package +// ships next to the bundle. CertificateAudit reads the expected digest out of +// it instead of pinning one in the datastream. +const caStampPath = "etc/ssl/certs/.ca-certificates.crt.sha256" + +// caStampWrongDigest is a well-formed stamp naming a digest the untouched +// bundle cannot have (all zeroes), isolating the comparison from the stamp +// side: the bundle is clean, the expected value is not. +var caStampWrongDigest = []byte(strings.Repeat("0", 64) + " ca-certificates.crt\n") + +// caStampMalformed has no line matching the object's +// `^([0-9a-fA-F]{64})[ \t]+\*?ca-certificates\.crt$` pattern, so the digest +// variable collects nothing and the stamp-file existence test fails. +var caStampMalformed = []byte("not a checksum line\n") + // activeShadowEntry is an /etc/shadow line whose password field is a // traditional DES crypt hash (not "!" or "*"), which the UserPasswordConfigured // pattern `^[^:]+:(?![!*])[^:\n]*:` matches, failing the none_exist test. @@ -409,11 +424,12 @@ func matrixCases() []matrixCase { want: map[string]results.Result{rulePackageSignature: results.Pass}, }, - // CertificateAudit is an AND of three criteria: the CA bundle exists, its - // SHA-256 matches the datastream's pinned hash (the base ref is read from - // the same pin the update-ca-cert workflow keeps in lockstep with that - // hash), and SSL_CERT_FILE is set to the bundle path. The pass fixture - // therefore also sets SSL_CERT_FILE; the fail fixtures each isolate one + // CertificateAudit is an AND of four criteria: the CA bundle exists, the + // ca-certificates stamp file exists and yields one SHA-256 digest, the + // bundle's SHA-256 equals that digest, and SSL_CERT_FILE is set to the + // bundle path. Nothing here depends on a hash pinned in the datastream, + // so no fixture needs to move when the upstream bundle rolls. The pass + // fixture also sets SSL_CERT_FILE; the fail fixtures each isolate one // failing dimension while holding the others valid. { name: "certificate_audit/pass_clean", @@ -427,6 +443,31 @@ func matrixCases() []matrixCase { containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, want: map[string]results.Result{ruleCertificateAudit: results.Fail}, }, + { + // Hash dimension from the stamp side: clean bundle, stamp claims a + // digest it cannot have. + name: "certificate_audit/fail_wrong_stamp_digest", + ops: []overlay.Op{overlay.ReplaceFile(caStampPath, caStampWrongDigest)}, + containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, + want: map[string]results.Result{ruleCertificateAudit: results.Fail}, + }, + { + // Stamp dimension: stamp present but unparseable, so there is no + // expected digest to compare against. + name: "certificate_audit/fail_malformed_stamp", + ops: []overlay.Op{overlay.ReplaceFile(caStampPath, caStampMalformed)}, + containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, + want: map[string]results.Result{ruleCertificateAudit: results.Fail}, + }, + { + // Stamp dimension: image ships no stamp file at all (e.g. a base + // image whose ca-certificates predates it). The rule must not pass + // vacuously. + name: "certificate_audit/fail_missing_stamp", + ops: []overlay.Op{overlay.RemoveFile(caStampPath)}, + containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, + want: map[string]results.Result{ruleCertificateAudit: results.Fail}, + }, { // Env dimension: bundle valid, SSL_CERT_FILE points elsewhere. name: "certificate_audit/fail_wrong_ssl_cert_file", From e0faacb6cc026f6928c09cd3e95442a42403d915 Mon Sep 17 00:00:00 2001 From: Dimitri John Ledkov Date: Fri, 7 Aug 2026 00:25:50 +0000 Subject: [PATCH 2/3] Verify the Java truststore against its stamp file where one exists MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Java-based images ship a truststore at /etc/ssl/certs/java/cacerts with the same style of package-recorded digest beside it (/etc/ssl/certs/java/.cacerts.sha256, sha256sum format). CertificateAudit ignored it, so a tampered truststore passed the rule even though the system CA bundle it checks is the equivalent file for non-Java trust. Extend the definition to cover it, conditionally — the truststore only exists on Java images, and a non-Java image must not fail for lacking one. The new criteria is an OR: - tst:5, unix:file_test with check_existence="none_exist": no truststore at all, i.e. a non-Java image; or - tst:6 AND tst:7: the stamp file exists and parses (obj:6, pattern `^([0-9a-fA-F]{64})[ \t]+\*?cacerts$`), and the truststore's SHA-256 equals the digest it names, via var:2 and ste:3. Pairing tst:6 with tst:7 is what stops the OR from being a loophole: a truststore shipped without a stamp fails rather than falling through to the absent branch. Verified against real images with oscap-docker: chainguard-base with a Java truststore passes, jdk:latest passes, chainguard-base without Java passes via the absent branch, a truststore tampered without touching its stamp fails, and a truststore legitimately updated with its stamp regenerated passes. The offline base has no Java, so the three new offline fixtures synthesize the pair with AddFile: pass_java_truststore, fail_java_wrong_stamp, fail_java_missing_stamp. The truststore's real format does not matter to a SHA-256 comparison, so opaque bytes exercise the check exactly as a JKS would. Co-Authored-By: Claude Opus 5 (1M context) --- .../ssg/content/ssg-chainguard-gpos-ds.xml | 44 +++++++++++- .../internal/scan/fixtures_test.go | 69 +++++++++++++++++-- 2 files changed, 105 insertions(+), 8 deletions(-) diff --git a/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml b/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml index b7f3578..f2d6ce3 100644 --- a/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml +++ b/gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml @@ -6228,7 +6228,10 @@ sha256sum -c .ca-certificates.crt.sha256' so the bundle matches the digest the package recorded for it, and that the SSL_CERT_FILE environment variable configured on the image or container is set to - /etc/ssl/certs/ca-certificates.crt. + /etc/ssl/certs/ca-certificates.crt. On Java-based images, which additionally + ship a truststore at /etc/ssl/certs/java/cacerts, also run 'cd + /etc/ssl/certs/java && sha256sum -c .cacerts.sha256'. Images with no + /etc/ssl/certs/java/cacerts have no truststore to verify. CCI-004909 @@ -6684,7 +6687,7 @@ Validate SHA-256 hash of CA bundle - Passes only if the CA bundle exists, its SHA-256 matches the digest recorded in the ca-certificates package stamp file /etc/ssl/certs/.ca-certificates.crt.sha256, and the SSL_CERT_FILE environment variable configured on the image or container is set to /etc/ssl/certs/ca-certificates.crt. + Passes only if the CA bundle exists, its SHA-256 matches the digest recorded in the ca-certificates package stamp file /etc/ssl/certs/.ca-certificates.crt.sha256, and the SSL_CERT_FILE environment variable configured on the image or container is set to /etc/ssl/certs/ca-certificates.crt. Java images additionally ship a JKS/PKCS12 truststore at /etc/ssl/certs/java/cacerts; when that file is present it must likewise match the digest in /etc/ssl/certs/java/.cacerts.sha256. Images without Java carry no truststore and are unaffected. Chainguard @@ -6694,6 +6697,13 @@ + + + + + + + @@ -6712,6 +6722,16 @@ + + + + + + + + + + @@ -6731,6 +6751,19 @@ ^([0-9a-fA-F]{64})[ \t]+\*?ca-certificates\.crt$ 1 + + /etc/ssl/certs/java/cacerts + + + /etc/ssl/certs/java + .cacerts.sha256 + ^([0-9a-fA-F]{64})[ \t]+\*?cacerts$ + 1 + + + /etc/ssl/certs/java/cacerts + SHA-256 + @@ -6740,11 +6773,18 @@ /etc/ssl/certs/ca-certificates.crt + + SHA-256 + + + + + diff --git a/tests/oscap-offline/internal/scan/fixtures_test.go b/tests/oscap-offline/internal/scan/fixtures_test.go index 022f877..827baf1 100644 --- a/tests/oscap-offline/internal/scan/fixtures_test.go +++ b/tests/oscap-offline/internal/scan/fixtures_test.go @@ -2,6 +2,8 @@ package scan_test import ( "bytes" + "crypto/sha256" + "encoding/hex" "errors" "fmt" "os" @@ -322,6 +324,25 @@ var caStampWrongDigest = []byte(strings.Repeat("0", 64) + " ca-certificates.crt // variable collects nothing and the stamp-file existence test fails. var caStampMalformed = []byte("not a checksum line\n") +// The Java truststore half of CertificateAudit is conditional: it only applies +// to images that ship /etc/ssl/certs/java/cacerts. The offline base is +// wolfi-base, which has no Java, so these fixtures synthesize the pair. The +// truststore's real format is irrelevant here — the check is a SHA-256 +// comparison, so opaque bytes exercise it exactly as a real JKS would. +const ( + javaTrustStorePath = "etc/ssl/certs/java/cacerts" + javaStampPath = "etc/ssl/certs/java/.cacerts.sha256" +) + +var javaTrustStore = []byte("synthetic truststore bytes, not a real JKS") + +// javaStamp returns a sha256sum-format stamp naming content's digest, matching +// what the package ships next to the truststore: " cacerts". +func javaStamp(content []byte) []byte { + sum := sha256.Sum256(content) + return []byte(hex.EncodeToString(sum[:]) + " cacerts\n") +} + // activeShadowEntry is an /etc/shadow line whose password field is a // traditional DES crypt hash (not "!" or "*"), which the UserPasswordConfigured // pattern `^[^:]+:(?![!*])[^:\n]*:` matches, failing the none_exist test. @@ -424,13 +445,15 @@ func matrixCases() []matrixCase { want: map[string]results.Result{rulePackageSignature: results.Pass}, }, - // CertificateAudit is an AND of four criteria: the CA bundle exists, the + // CertificateAudit is an AND of five criteria: the CA bundle exists, the // ca-certificates stamp file exists and yields one SHA-256 digest, the - // bundle's SHA-256 equals that digest, and SSL_CERT_FILE is set to the - // bundle path. Nothing here depends on a hash pinned in the datastream, - // so no fixture needs to move when the upstream bundle rolls. The pass - // fixture also sets SSL_CERT_FILE; the fail fixtures each isolate one - // failing dimension while holding the others valid. + // bundle's SHA-256 equals that digest, SSL_CERT_FILE is set to the + // bundle path, and — only where /etc/ssl/certs/java/cacerts exists — the + // Java truststore likewise matches its own stamp file. Nothing here + // depends on a hash pinned in the datastream, so no fixture needs to move + // when an upstream bundle rolls. The pass fixture also sets + // SSL_CERT_FILE; the fail fixtures each isolate one failing dimension + // while holding the others valid. { name: "certificate_audit/pass_clean", containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, @@ -468,6 +491,40 @@ func matrixCases() []matrixCase { containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, want: map[string]results.Result{ruleCertificateAudit: results.Fail}, }, + { + // Java dimension: a synthesized truststore agreeing with its stamp + // must keep the rule passing. The base has no Java, so this is the + // only fixture exercising the "present and matching" OR branch. + name: "certificate_audit/pass_java_truststore", + ops: []overlay.Op{ + overlay.AddFile(javaTrustStorePath, javaTrustStore, 0o444, 0, 0), + overlay.AddFile(javaStampPath, javaStamp(javaTrustStore), 0o444, 0, 0), + }, + containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, + want: map[string]results.Result{ruleCertificateAudit: results.Pass}, + }, + { + // Java dimension: truststore present but its stamp names a digest + // for different content, i.e. a truststore modified in place. + name: "certificate_audit/fail_java_wrong_stamp", + ops: []overlay.Op{ + overlay.AddFile(javaTrustStorePath, javaTrustStore, 0o444, 0, 0), + overlay.AddFile(javaStampPath, javaStamp([]byte("other content")), 0o444, 0, 0), + }, + containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, + want: map[string]results.Result{ruleCertificateAudit: results.Fail}, + }, + { + // Java dimension: truststore present with no stamp beside it. The + // absent-truststore OR branch must not rescue this — there is a + // truststore, it just cannot be verified. + name: "certificate_audit/fail_java_missing_stamp", + ops: []overlay.Op{ + overlay.AddFile(javaTrustStorePath, javaTrustStore, 0o444, 0, 0), + }, + containerVars: []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}, + want: map[string]results.Result{ruleCertificateAudit: results.Fail}, + }, { // Env dimension: bundle valid, SSL_CERT_FILE points elsewhere. name: "certificate_audit/fail_wrong_ssl_cert_file", From 11840721d0307223d669ee0db5523db2eea40b85 Mon Sep 17 00:00:00 2001 From: Dimitri John Ledkov Date: Fri, 7 Aug 2026 00:30:11 +0000 Subject: [PATCH 3/3] Guard both trust-store stamps using the jre image The stamp-file guard added with the pinned-hash removal only inspected wolfi-base, which has no Java. That left the truststore criterion unguarded: a jdk/jre image that dropped /etc/ssl/certs/java/.cacerts.sha256 or shipped one disagreeing with cacerts would surface as a red E2E run rather than as a clear failure here. Point the guard at cgr.dev/chainguard/jre:latest instead. It carries both stamps CertificateAudit reads, so a single crane export covers the whole rule. Each stamp is checked separately rather than as one combined sha256sum -c, so the error names which trust store drifted, and a missing stamp is reported distinctly from a mismatched one. IMAGE_REF stays wolfi-base: it is what the E2E fixtures are pinned to, and re-pinning them is a separate concern from verifying the rule's premise. Both images are now resolved and cosign-verified through one helper. Verified by running the guard body against a real jre export: clean export passes, appending to cacerts fails naming the java directory, appending to the CA bundle fails naming /etc/ssl/certs, and removing .cacerts.sha256 fails as missing. Also confirmed `exit 1` inside the verify helper aborts the step rather than being swallowed by the command substitution. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/update-ca-cert.yaml | 139 +++++++++++++++++--------- 1 file changed, 91 insertions(+), 48 deletions(-) diff --git a/.github/workflows/update-ca-cert.yaml b/.github/workflows/update-ca-cert.yaml index 98938bd..c7c59c2 100644 --- a/.github/workflows/update-ca-cert.yaml +++ b/.github/workflows/update-ca-cert.yaml @@ -26,7 +26,13 @@ jobs: pull-requests: write issues: write env: + # The image the E2E fixtures are pinned to, re-resolved below. IMAGE_REF: cgr.dev/chainguard/wolfi-base:latest + # The image the stamp-file guard inspects. jre carries BOTH stamps + # CertificateAudit reads — /etc/ssl/certs/.ca-certificates.crt.sha256 and + # /etc/ssl/certs/java/.cacerts.sha256 — so one image covers the whole rule. + # wolfi-base has no Java and would leave the truststore half unguarded. + STAMP_IMAGE_REF: cgr.dev/chainguard/jre:latest TESTS_PATH: tests FIXTURES_GLOB: tests/e2e/fixtures/*/Dockerfile TEST_PIN_FILE: tests/oscap-offline/internal/scan/fixtures_test.go @@ -49,63 +55,94 @@ jobs: uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2 - name: Setup crane uses: imjasonh/setup-crane@feee3b6bb0d4c68370f256a4502498c9227e5c6b # v0.7 - - name: Pull and verify image + - name: Pull and verify images id: image run: | set -euo pipefail - DIGEST=$(crane digest "${IMAGE_REF}") - FULL_REF="${IMAGE_REF%:*}@${DIGEST}" - echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT" - echo "full_ref=${FULL_REF}" >> "$GITHUB_OUTPUT" + # Verify a tag's signature and echo its digest-pinned reference. + # cosign's own output is dropped so only the ref lands on stdout; the + # command substitution runs this in a subshell, so `exit 1` aborts the + # substitution and `set -e` then fails the step. + resolve_and_verify() { + local ref="$1" full + full="${ref%:*}@$(crane digest "${ref}")" + cosign verify \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + --certificate-identity-regexp "https://github.com/chainguard-images/images/.*" \ + "${full}" >/dev/null || { + echo "::error::Image signature verification failed: ${ref}" + exit 1 + } + echo "${full}" + } - # Verify signature - cosign verify \ - --certificate-oidc-issuer https://token.actions.githubusercontent.com \ - --certificate-identity-regexp "https://github.com/chainguard-images/images/.*" \ - "${FULL_REF}" || { - echo "::error::Image signature verification failed" - exit 1 - } - - name: Verify CA bundle stamp file + FULL_REF=$(resolve_and_verify "${IMAGE_REF}") + STAMP_FULL_REF=$(resolve_and_verify "${STAMP_IMAGE_REF}") + + echo "digest=${FULL_REF#*@}" >> "$GITHUB_OUTPUT" + echo "full_ref=${FULL_REF}" >> "$GITHUB_OUTPUT" + echo "stamp_digest=${STAMP_FULL_REF#*@}" >> "$GITHUB_OUTPUT" + echo "stamp_full_ref=${STAMP_FULL_REF}" >> "$GITHUB_OUTPUT" + - name: Verify trust-store stamp files id: ca env: - STEPS_IMAGE_OUTPUTS_FULL_REF: ${{ steps.image.outputs.full_ref }} - STEPS_IMAGE_OUTPUTS_DIGEST: ${{ steps.image.outputs.digest }} + STEPS_IMAGE_OUTPUTS_STAMP_FULL_REF: ${{ steps.image.outputs.stamp_full_ref }} + STEPS_IMAGE_OUTPUTS_STAMP_DIGEST: ${{ steps.image.outputs.stamp_digest }} run: | set -euo pipefail - # CertificateAudit no longer pins a CA bundle hash in the datastream: - # the OVAL reads the expected digest out of the stamp file the - # ca-certificates package ships next to the bundle - # (/etc/ssl/certs/.ca-certificates.crt.sha256, sha256sum format) and - # compares the bundle against it. Nothing in this repo needs updating - # when the upstream bundle rolls. + # CertificateAudit no longer pins any hash in the datastream: the OVAL + # reads each expected digest out of the stamp file the owning package + # ships next to the file it describes, in sha256sum format — + # /etc/ssl/certs/.ca-certificates.crt.sha256 (always) + # /etc/ssl/certs/java/.cacerts.sha256 (Java images only) + # — and compares the real file against it. Nothing in this repo needs + # updating when either rolls upstream. + # + # This step guards the premise that replaced the pins: if a future + # image drops a stamp file, renames it, or ships one that disagrees + # with the file it names, the rule starts failing on clean images. + # Fail loudly here — daily — instead of finding out via a red E2E run. # - # This step guards the premise that replaced the pin: if a future - # base image drops the stamp file, renames it, or ships one that - # disagrees with the bundle, the rule would start failing on clean - # images. Fail loudly here — daily — instead of finding out via a red - # E2E run. + # The jre image is used because it carries BOTH stamps, so a single + # export covers the whole rule. wolfi-base has no Java and would leave + # the truststore criterion unguarded. workdir=$(mktemp -d) - crane export "${STEPS_IMAGE_OUTPUTS_FULL_REF}" - | \ - tar -C "${workdir}" -x etc/ssl/certs/ca-certificates.crt etc/ssl/certs/.ca-certificates.crt.sha256 + crane export "${STEPS_IMAGE_OUTPUTS_STAMP_FULL_REF}" - | \ + tar -C "${workdir}" -x \ + etc/ssl/certs/ca-certificates.crt \ + etc/ssl/certs/.ca-certificates.crt.sha256 \ + etc/ssl/certs/java/cacerts \ + etc/ssl/certs/java/.cacerts.sha256 - if ! (cd "${workdir}/etc/ssl/certs" && sha256sum -c .ca-certificates.crt.sha256); then - echo "::error::CA bundle does not match its stamp file in ${IMAGE_REF}; CertificateAudit will fail on clean images" - exit 1 - fi + # Both stamps must be present and correct. Checking each separately + # (rather than one combined sha256sum -c) keeps the error message + # specific about which trust store drifted. + for pair in "etc/ssl/certs:.ca-certificates.crt.sha256" "etc/ssl/certs/java:.cacerts.sha256"; do + dir="${pair%%:*}" + stamp="${pair#*:}" + if [ ! -f "${workdir}/${dir}/${stamp}" ]; then + echo "::error::${dir}/${stamp} missing from ${STAMP_IMAGE_REF}; CertificateAudit will fail on clean images" + exit 1 + fi + if ! (cd "${workdir}/${dir}" && sha256sum -c "${stamp}"); then + echo "::error::${dir} contents do not match ${stamp} in ${STAMP_IMAGE_REF}; CertificateAudit will fail on clean images" + exit 1 + fi + done SHA=$(sha256sum "${workdir}/etc/ssl/certs/ca-certificates.crt" | cut -d' ' -f1) + JAVA_SHA=$(sha256sum "${workdir}/etc/ssl/certs/java/cacerts" | cut -d' ' -f1) echo "sha=${SHA}" >> "$GITHUB_OUTPUT" cat >> "$GITHUB_STEP_SUMMARY" < @@ -296,16 +335,20 @@ jobs: `TestParseWolfiBaseRef` was run against the updated tree before this PR was opened. - CertificateAudit itself no longer pins a CA bundle hash — the OVAL - reads the expected digest from the ca-certificates stamp file - (`/etc/ssl/certs/.ca-certificates.crt.sha256`) inside the scanned - image. The `Verify CA bundle stamp file` step above asserts that - stamp is present and agrees with the bundle, so a base image that - dropped it would fail this run rather than silently red the E2E - CertificateAudit assertions. + CertificateAudit itself no longer pins any hash — the OVAL reads each + expected digest from the stamp file shipped beside the file it + describes inside the scanned image + (`/etc/ssl/certs/.ca-certificates.crt.sha256`, and on Java images + `/etc/ssl/certs/java/.cacerts.sha256`). The `Verify trust-store stamp + files` step above asserts both are present and agree, using + `${{ env.STAMP_IMAGE_REF }}` because it carries both, so an image + that dropped one would fail this run rather than silently red the + E2E CertificateAudit assertions. - **Image**: `${{ env.IMAGE_REF }}` - **Digest**: `${{ steps.image.outputs.digest }}` + - **Stamp image**: `${{ env.STAMP_IMAGE_REF }}` + - **Stamp image digest**: `${{ steps.image.outputs.stamp_digest }}` - **CA SHA256**: `${{ steps.ca.outputs.sha }}` - **Fixtures changed**: `${{ steps.fixtures.outputs.fixtures_changed }}` - **Test pin changed**: `${{ steps.testpin.outputs.test_pin_changed }}`