Skip to content

Drop apk audit from CertificateAudit verification guidance; document the checks - #158

Merged
stevebeattie merged 2 commits into
chainguard-dev:mainfrom
stevebeattie:fix-certaudit-verification-guidance
Aug 20, 2026
Merged

Drop apk audit from CertificateAudit verification guidance; document the checks#158
stevebeattie merged 2 commits into
chainguard-dev:mainfrom
stevebeattie:fix-certaudit-verification-guidance

Conversation

@stevebeattie

Copy link
Copy Markdown
Member

Summary

CertificateAudit's Script Verification text offers apk audit as an
equivalent alternative to comparing each trust store against the digest
recorded beside it. The two are not equivalent, and they disagree most on the
images the rule was recently changed to support.

This replaces that guidance with the sidecar comparison alone, and moves the
reasoning — which is not appropriate for assessor-facing STIG text — into a new
maintainer document.

Why apk audit had to go

All measured, not inferred:

  • It answers a different question. apk audit compares a file against the
    package that shipped it; the sidecar compares it against the built image. An
    image carrying additional trust anchors legitimately diverges from the
    packages, so sha256sum -c reports OK while apk audit reports
    U etc/ssl/certs/ca-certificates.crt. An assessor taking the apk audit
    branch raises a spurious finding on exactly the customised images the rule was
    changed to accommodate.
  • It exits 0 even when reporting differences, so it gives no usable
    pass/fail criterion. apk audit && echo compliant is a false pass.
  • "Empty output" is not the test either. Default mode lists unrelated A
    entries (resolv.conf, hostname, apko.json, ld.so.cache) — including
    A etc/ssl/certs/.ca-certificates.crt.sha256, because no package owns a
    sidecar. That A appears on every apko-built image, pristine or not.
  • Coverage is split across modes, and the intuitive flags are the wrong
    ones.
    Modification is reported only in default mode; deletion (X) only
    under --system. Neither --system nor --full reports a modified bundle.
  • Most images cannot run it. Distroless images ship no shell and no apk
    binary — jre:latest and static:latest carry only the database at
    usr/lib/apk/db/installed.

A correct apk audit instruction would need three invocations with different
flags, output parsing, and a carve-out for images with added anchors. The
sidecar check is one command.

Changes

Datastream (prose only; no OVAL logic touched). The verification text now
names the sidecar comparison as the single check and says up front that on
distroless images it is run against an extracted root filesystem rather than in
the image. It also corrects the attribution: the digest is recorded by the image
build, not by the ca-certificates package, which owns neither sidecar
(apk info -W reports "Could not find owner package" for both). The /kaniko,
SSL_CERT_FILE, and Java paragraphs are unchanged.

New docs/certificate-audit.md, linked from a new "Check implementation
notes" section in the README so the other checks can slot in later. It covers
what each of tst:1tst:13 asserts; why the sidecar-exists guards are paired
with the comparisons (a missing or malformed sidecar makes the comparison
evaluate to error, not false — false AND error = false is what makes the
failure definite, verified as tst:4 => false, tst:2 => error); where the
expected digests come from, including apko's write ordering; how additional
trust anchors come to be covered by the sidecars; the trust model; testing
notes; and known gaps.

The apk audit findings above are recorded there as a gotcha section, with the
A/U/X/m table and which mode reports each. It also flags explicitly that
protected_paths.d/ca-certificates.list is a red herring — it is shipped by
ca-certificates and so absent from bundle-only images such as wolfi-base,
but it makes no difference to detection: images with and without it both report
U on a modified bundle. That is the wrong inference I drew first, so it is
written down rather than left to be re-derived.

Deliberately not included

No consumer-facing remediation text. The mechanism by which added anchors reach
the sidecars is documented, but the build definition is an input to the image
build rather than something edited in the image under assessment, so the
supported path for a consumer needing additional trust anchors should be settled
before any remediation instruction is written. The document describes the
mechanism only and prescribes nothing.

Verification

  • xmllint --noout: clean.
  • oscap ds sds-validate: rc=0.
  • oscap xccdf validate: 235 errors, identical to origin/main — no new ones.
  • Zero changed lines touch OVAL logic elements; the datastream diff is prose
    inside one <html:pre> block.
  • Offline fixture matrix green (go test -run TestOfflineFixtureMatrix).

🤖 Generated with Claude Code

stevebeattie and others added 2 commits August 19, 2026 10:20
The rule's Script Verification text offered `apk audit` as an equivalent
alternative to comparing each trust store against the digest recorded
beside it. The two are not equivalent, and the difference matters most on
the images the rule was recently changed to support:

  - `apk audit` compares a file against the package that shipped it. An
    image built with approved additional trust anchors legitimately
    diverges from that, so it reports
    `U etc/ssl/certs/ca-certificates.crt` while `sha256sum -c` reports OK.
    Following the apk audit branch yields a spurious finding.
  - It exits 0 even when it reports differences, so it offers no usable
    pass/fail criterion, and its default output lists unrelated `A`
    entries — including the sidecar itself, which no package owns.
  - Coverage is split across modes: modification is reported only in
    default mode, deletion (`X`) only under `--system`, and neither
    `--system` nor `--full` reports a modified bundle.
  - Most images cannot run it. Distroless images ship no shell and no apk
    binary, only the database.

So the guidance now names the sidecar comparison as the single check, and
says up front that on distroless images it is run against an extracted
root filesystem rather than in the image. Also corrects the attribution:
the digest is recorded by the image build, not by the ca-certificates
package, which owns neither sidecar.

The rationale above, and the rest of the reasoning that is not
appropriate for assessor-facing STIG text, moves to
docs/certificate-audit.md: what each criterion asserts, why the
sidecar-exists guards are paired with the comparisons (a missing sidecar
makes the comparison evaluate to error, not false), where "approved" is
established, how to add trust anchors through apko's `certificates:`
stanza, and the Java truststore's non-reproducibility under a post-build
`update-ca-certificates`.

XCCDF changes are prose only; no OVAL logic is touched. `oscap ds
sds-validate` passes and `oscap xccdf validate` reports the same 235
pre-existing errors as origin/main. Offline fixture matrix green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The "Adding trust anchors" section read as instructions to whoever was
reading it — "add them via apko's certificates: stanza", "this is the
supported path", "do not add trust anchors post-build". That framing is
wrong for this audience: the apko build definition is an input to the
image build, not something a consumer of the image edits, so the section
was telling readers to do something they are not positioned to do.

Recast it as a description of the mechanism instead. It now explains how
an image that trusts anchors beyond the Mozilla set ends up
self-consistent — apko merges each certificate into every CA bundle and
Java truststore and only then writes the sidecars — so that a passing
result on a customised image is understood rather than mistaken for a
gap, and states the two consequences that matter when reading a result:
such an image passes, and a trust store modified after the build fails
because nothing regenerates the sidecars.

No claim about who performs any of this, and no instruction to do it.
Consumer-facing remediation guidance is deliberately still absent, here
and in the profile; it needs the supported path established first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@stevebeattie
stevebeattie requested review from egibs and xnox August 20, 2026 04:08

@xnox xnox left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope to one day move the public java cacerts to the reproducible unencrypted PKCS12 format.

Yes, the ecs certs is a gap.... in a way - they are used by ecs agents only; and not trusted system wide. But we likely should cover them everywhere too.

@stevebeattie
stevebeattie merged commit 570bc2e into chainguard-dev:main Aug 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants