Skip to content

#77 fix cname dedupe#78

Merged
matthewdevenny merged 2 commits into
mainfrom
matt/77-cname-chain-dedupe
Jul 8, 2026
Merged

#77 fix cname dedupe#78
matthewdevenny merged 2 commits into
mainfrom
matt/77-cname-chain-dedupe

Conversation

@matthewdevenny

@matthewdevenny matthewdevenny commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Closes #77

Why

cargowall summary dedupes connection events per step before the SaaS push, on a key of (process, dest, port, protocol, eventType) — and keeps the first-seen event wholesale. Transparent CNAME support (#63/#68/#73) attributes a CNAME-derived connection to its origin hostname (chain[0]), the same dest a direct connection to that origin reports. So a chain-bearing event and a chainless direct hit to the same origin collide on the dedup key, and whenever the chainless one sorts first, the chain-bearing event is discarded — the pushed/stored event has an empty cname_chain even though the firewall logged the chain correctly (see #77 for the observed auth.docker.io evidence). The same wholesale discard applies to auto_allowed_type, which is per-IP (CIDR pass of GetAutoAllowedType) while the dedup key ignores DstIP when a hostname is set — round-robin DNS can produce same-key duplicates where only a later one carries the type.

What

  • cmd/summary.godeduplicateStepEvents: the seen map now stores the index of the retained representative (map[dedupKey]int). On a collision, instead of discarding the duplicate outright, backfill the distinguishing fields the representative is missing:

    • CNAMEChain — filled when the representative has none and the duplicate carries one;
    • AutoAllowedType — same fill-if-empty rule.

    Fill-if-empty only: the first-seen event stays the representative and an existing value is never overwritten. One row per destination is preserved (the merge approach preferred in summary: cname_chain dropped by deduplicateStepEvents (chainless duplicate to the same origin wins) #77, not the add-chain-to-key alternative, which would emit a redundant chainless row). No signature or caller changes — auditEventToProto picks up the now-populated fields via its existing non-empty guards.

    MatchedRule deliberately gets no backfill: only connection_late_allowed events carry it, and they always do (HasAllow() ⇒ non-empty rule), so with eventType in the dedup key a same-key group is uniformly populated or uniformly empty — the backfill branch could never fire. Documented in a comment.

  • cmd/summary_test.go: three new tests alongside the existing dedup group —

    • ChainBackfilledFromLaterDuplicate — the summary: cname_chain dropped by deduplicateStepEvents (chainless duplicate to the same origin wins) #77 repro shape: chainless direct hit first, chain-bearing CNAME-derived duplicate second → one row, chain preserved, first-seen timestamp retained;
    • ChainNotOverwritten — a later chainless duplicate doesn't clobber an existing chain;
    • AutoAllowedTypeBackfilled — per-IP round-robin shape: type backfilled from the first duplicate that carries one, a later different type doesn't overwrite.

Semantics note

For a merged row with mixed auto/non-auto duplicates, the auto-allow attribution now deterministically wins (previously the label was ordering-arbitrary, first-seen verbatim). This means totalAutoAllowed / AutoAllowedConnections count a destination as auto-allowed if any connection to it was — intended, since at least one connection genuinely took the auto-allow path, and the alternative silently hides that attribution.

Testing

  • All 10 TestSummary_DeduplicateStepEvents_* tests (7 existing + 3 new) and the full cmd package pass on linux (golang:1.25 container; the dev host is darwin and these files are //go:build linux).
  • GOOS=linux go build ./..., go vet, staticcheck, and gofumpt -l are clean.

🤖 Generated with Claude Code

Signed-off-by: Matthew DeVenny <matt@codecargo.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates CargoWall’s summary-stage event deduplication so that when multiple connection events collapse to the same dedup key, the retained “first-seen” representative row is enriched with missing distinguishing fields (notably CNAME chain and auto-allowed type), preventing loss of drill-down metadata during API push and summary rendering.

Changes:

  • Update deduplicateStepEvents to track the representative event index per dedup key and backfill CNAMEChain / AutoAllowedType from later duplicates when the representative is missing them.
  • Add focused unit tests covering CNAME-chain backfill, non-overwrite behavior, and auto-allowed-type backfill semantics.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
cmd/summary.go Enhances deduplication to preserve key metadata (CNAME chain / auto-allowed type) across same-key duplicates.
cmd/summary_test.go Adds regression tests for the new backfill behavior to prevent metadata loss in future changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/summary.go Outdated
@matthewdevenny matthewdevenny marked this pull request as ready for review July 8, 2026 15:53

@mtmk mtmk left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread cmd/summary.go
Comment thread cmd/summary.go
Comment thread cmd/summary_test.go
Comment thread cmd/summary.go Outdated
Signed-off-by: Matthew DeVenny <matt@codecargo.com>
@matthewdevenny matthewdevenny merged commit 49385df into main Jul 8, 2026
18 checks passed
@matthewdevenny matthewdevenny deleted the matt/77-cname-chain-dedupe branch July 8, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

summary: cname_chain dropped by deduplicateStepEvents (chainless duplicate to the same origin wins)

3 participants