[caclmgrd] Own the dhcp_server docker0 syslog INPUT exception by Xichen96 · Pull Request #412 · sonic-net/sonic-host-services

Xichen96 · 2026-07-16T12:32:15Z

Why I did it

dhcp_server is the only bridge-mode container, so its syslog reaches host rsyslog over docker0. caclmgrd flushes and rebuilds the INPUT chain on every control-plane ACL change and appends a catch-all DROP, which drops that syslog traffic. The dhcp_server container script used to add/remove this docker0 ACCEPT exception itself, but caclmgrd wiped it on the next rebuild.

Container-to-host syslog forwarding uses RELP over tcp/2514 (since sonic-net/sonic-buildimage#18113), so the exception is for tcp/2514. The RELP connection is initiated by the container, so its inbound SYN is a new connection that caclmgrd's conntrack ESTABLISHED,RELATED accept does not cover — it needs an explicit ACCEPT or the RELP session never opens.

Fixes sonic-net/sonic-buildimage#27584.

This is step 2 of 3:

[dhcp_server] Revert container-side docker0 syslog iptables add/del (#26637); caclmgrd to own the rule sonic-buildimage#28328 (merged) removed the old container-side add/del of the rule.
This PR makes caclmgrd own the rule.
[dhcp_server] Wait for caclmgrd-owned docker0 syslog rule on start/stop sonic-buildimage#28580 adds container-side feature-gated bounded waits on the rule presence/absence.

Work item tracking

Microsoft ADO (number only): 38745011

How I did it

Mirror the existing REDFISH FEATURE-gated mechanism:

Seed DhcpServerSyslogAllowed from FEATURE|dhcp_server state at init.
Re-emit the docker0/2514 ACCEPT rule on every rebuild while enabled, before any catch-all DROP (host namespace / IPv4 only). On feature-disable the rule is simply not re-emitted, so the next rebuild removes it.
Generalize the shared FEATURE-table subscriber handler (handle_redfish_feature_events -> handle_feature_state_events) to drain both redfish and dhcp_server keys in the one loop (a second popper would steal events from the redfish handling).

The rule string is byte-identical to the container-side -C check so the container can wait on presence/absence in the follow-up PR.

How to verify it

New unit test tests/caclmgrd/caclmgrd_dhcp_server_syslog_test.py covers: init seeding from FEATURE, rule present when enabled / absent when disabled, rule strictly before the catch-all DROP, host-namespace-only, and the FEATURE-event handler transitions.
Existing tests/caclmgrd/caclmgrd_redfish_acl_test.py updated for the renamed handler and still passes.

Hardware validation

Validated by manually hand-patching this caclmgrd change onto a running DUT (no image rebuild).

DUT / topology: bjw-can-720dt-2 (720dt, mx), dhcp_server enabled.
Transport: RELP tcp/2514 (host rsyslogd listens on 240.127.1.1:2514, the docker0 IP).
Precondition: a control-plane ACL rule was configured so caclmgrd programs its catch-all -A INPUT -j DROP.

Results:

Rule programmed above the catch-all DROP (feature enabled):

3:-A INPUT -i docker0 -p tcp -m tcp --dport 2514 -m comment --comment dhcp_server_syslog -j ACCEPT
46:-A INPUT -j DROP

End-to-end syslog delivery — docker exec dhcp_server logger ... reached host /var/log/syslog; the docker0 ACCEPT rule showed packet hits (2 pkts / 214 B).

Feature toggle — config feature state dhcp_server disabled removes the rule; enabled re-adds it. caclmgrd logs both transitions:

INFO caclmgrd: Disabled dhcp_server docker0 syslog control plane ACL handling
INFO caclmgrd: Enabled  dhcp_server docker0 syslog control plane ACL handling

Survives control-plane ACL re-walk (the root regression, sonic-buildimage#27584) — after modifying an unrelated control-plane ACL rule (forcing caclmgrd to flush + rebuild INPUT), the docker0/2514 ACCEPT rule is still present at position 3, above the catch-all DROP.

Note: this hardware validation was performed on a 202605 image (identical caclmgrd INPUT flush/rebuild + catch-all DROP path); the code change under test is the same logic being merged here.

Which release branch to backport (provide reason below if selected)

Backported as a coordinated set with sonic-buildimage#28328 (step 1) and #28580 (step 3) so the branch is never left with neither the container nor caclmgrd owning the rule.
Tracking issue/work item for backport/cherry-pick request (GitHub issue or Microsoft ADO): Microsoft ADO 38745011
Failure type: day-one issue (the container-owned docker0 syslog rule never survived a caclmgrd control-plane ACL rebuild)

Tested branch

Test result

202605: 20260510.07 — hand-patched HW validation on bjw-can-720dt-2 (720dt, mx); see "Hardware validation" above (rule programmed above the catch-all DROP, end-to-end RELP syslog delivered, feature toggle add/remove, survives an unrelated control-plane ACL re-walk — bug #27584 fix). Unit tests tests/caclmgrd/caclmgrd_dhcp_server_syslog_test.py and the updated caclmgrd_redfish_acl_test.py pass.
master: validation pending — the caclmgrd code path is identical to 202605; a master-image run is the only remaining validation step.

Description for the changelog

[caclmgrd] Own the dhcp_server docker0 syslog INPUT exception so it survives the flush-and-rebuild.

mssonicbld · 2026-07-16T12:32:23Z

/azp run

azure-pipelines · 2026-07-16T12:32:23Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

azure-pipelines · 2026-07-16T12:32:33Z

Azure Pipelines: Successfully started running 1 pipeline(s).

mssonicbld · 2026-07-16T12:45:00Z

/azp run

azure-pipelines · 2026-07-16T12:45:09Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Pull request overview

Updates caclmgrd (the host-side control-plane ACL manager that flushes/rebuilds iptables) to own the dhcp_server syslog exception rule so it reliably persists across ACL rebuilds, preventing UDP/514 syslog traffic arriving via docker0 from being dropped by the catch-all INPUT DROP.

Changes:

Add DhcpServerSyslogAllowed feature-gated flag (seeded from FEATURE|dhcp_server at init) and re-emit the docker0/UDP 514 ACCEPT rule during host-namespace IPv4 ACL rebuilds.
Generalize the FEATURE-table event handler from redfish-only to handle_feature_state_events, handling both redfish and dhcp_server state transitions.
Add a focused unit test suite for the new dhcp_server syslog exception behavior and update the existing redfish test to use the renamed handler.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
scripts/caclmgrd	Seeds/handles `FEATURE.dhcp_server` state and emits the host-only `docker0` UDP/514 `ACCEPT` exception before any catch-all `DROP`; generalizes FEATURE event handling for redfish + dhcp_server.
tests/caclmgrd/caclmgrd_dhcp_server_syslog_test.py	New unit tests validating seeding, rule presence/absence, ordering before catch-all DROP, host-namespace-only emission, and FEATURE event transitions.
tests/caclmgrd/caclmgrd_redfish_acl_test.py	Updates test to call the renamed generalized FEATURE event handler.

Xichen96 · 2026-07-16T13:04:51Z

@StormLiangMS @yxieca @lguohan could you please review when you get a chance? This is step 2 of 3 (caclmgrd now owns the dhcp_server docker0 syslog INPUT exception; buildimage#28328 removed the old container-side add/del). Thanks!

mssonicbld · 2026-07-17T11:20:34Z

This PR has backport request for branch(es): 202605.
Added label(s) for branch(es) 202605.

_{---Powered by SONiC BuildBot}

StormLiangMS

@Xichen96 — the master-branch implementation looks good.

caclmgrd now owns and re-emits the feature-gated docker0 UDP/514 exception before the catch-all INPUT DROP, including correct enable/disable rebuild behavior and host-namespace-only scoping. CI is green and I found no source-branch code blocker.

For the 202605 request, the current cherry-pick conflicts because that branch does not contain the master redfish FEATURE-event infrastructure or its test. The backport needs a target-aware adaptation that adds only the dhcp_server handling, plus explicit 202605 test evidence.

mssonicbld · 2026-07-21T12:19:14Z

This PR has backport request label(s) for branch(es): 202605, but is missing required test information. Please make sure you tick the tested branch(es) in the Tested branch section and provide test evidence (e.g., 202605: <test result>) in the Test result section as well in your PR description.

_{---Powered by SONiC BuildBot}

dhcp_server is the only bridge-mode container, so its syslog reaches host rsyslog over docker0. caclmgrd flushes and rebuilds the INPUT chain on every control-plane ACL change, appending a catch-all DROP, which drops that syslog traffic. Previously the dhcp_server container script added/removed this exception itself, but caclmgrd wiped it on the next rebuild (sonic-net/sonic-buildimage#27584). Container-to-host syslog forwarding uses RELP over tcp/2514 (since sonic-net/sonic-buildimage#18113), so the exception is for tcp/2514. The connection is initiated by the container, so the inbound SYN is a new connection that the conntrack ESTABLISHED,RELATED accept does not cover -- it needs an explicit ACCEPT or the RELP session never opens. Make caclmgrd own the exception, mirroring the existing REDFISH FEATURE-gated mechanism: - Seed DhcpServerSyslogAllowed from FEATURE|dhcp_server state at init. - Re-emit the docker0/2514 ACCEPT rule on every rebuild while enabled, before any catch-all DROP (host namespace / IPv4 only). On feature-disable the rule is not re-emitted, so the next rebuild removes it. - Generalize the shared FEATURE-table subscriber handler (handle_redfish_feature_events -> handle_feature_state_events) to drain both redfish and dhcp_server keys in the one loop; a second popper would steal events from the redfish handling. The rule string is byte-identical to the container-side -C check so the container can wait on presence/absence (follow-up PR). Step 2 of 3 (buildimage#28328 removed the old container-side add/del; a follow-up adds container-side waits). ADO: 38745011. Signed-off-by: Xichen Lin <lukelin0907@gmail.com> Signed-off-by: Xichen96 <lukelin0907@gmail.com>

mssonicbld · 2026-07-22T04:45:36Z

/azp run

azure-pipelines · 2026-07-22T04:45:45Z

Azure Pipelines: Successfully started running 1 pipeline(s).

yyynini

LGTM

StormLiangMS · 2026-07-22T09:01:42Z

hi @Xichen96 Do we have a SONiC-MGMT test that verifies this functionality? If no, pls open an issue to track it as a test gap, then we can merge this PR.

Xichen96 · 2026-07-22T15:39:49Z

Hi @StormLiangMS — thanks for the review.

There is existing sonic-mgmt coverage: tests/cacl/test_cacl_application.py::test_cacl_application asserts the full caclmgrd INPUT ruleset, including the dhcp_server docker0 syslog exception (added in #24696). Because this change moves ownership of that rule into caclmgrd and corrects the transport to RELP tcp/2514 (feature-gated on FEATURE.dhcp_server), that existing assertion needs a corresponding update.

I've opened a tracking issue for it: sonic-net/sonic-mgmt#26470.

This has been hardware-validated by manual patch on an mx (720dt) DUT: the rule is programmed above the catch-all DROP when the feature is enabled, container syslog reaches host rsyslog over tcp/2514, it toggles correctly on feature enable/disable, and it survives a control-plane ACL re-walk (the root regression in sonic-buildimage#27584). Happy to merge once you're good.

mssonicbld · 2026-07-22T16:17:30Z

The change is not in 202605 yet. @Xichen96, please manually create the cherry pick PR for branch 202605.
You can ping the release branch owner(github account: vaibhavhd) to approve your cherry pick PR.
If this change is already in 202605, please comment "already in 202605". Thanks!

_{---Powered by SONiC BuildBot}

Step 3 of 3 of moving the dhcp_server docker0 syslog INPUT exception ownership to caclmgrd (fixes #27584). PR1 (#28328) removed the container-side add/del; PR2 (sonic-net/sonic-host-services#412) made caclmgrd own the rule. This replaces the removed add/del with feature-gated bounded waits that synchronize with caclmgrd: - preStartAction: when FEATURE|dhcp_server is enabled, wait up to 10s for the docker0 tcp/2514 ACCEPT rule to be present before the container starts, so the RELP client's first connection is not dropped by the control-plane catch-all DROP. - stop(): when FEATURE|dhcp_server is disabled, wait up to 10s for the rule to be removed. Plain restarts / shutdown / CONFIG_DB-down skip the wait (positive-state gate). The iptables -C existence check matches the canonical stored form of the rule caclmgrd installs (identical to `iptables -S INPUT` output), so the wait observes the rule reliably across iptables variants. Signed-off-by: Xichen96 <lukelin0907@gmail.com>

The dhcp_server docker0 syslog INPUT exception changed from UDP 514 to RELP over tcp/2514, and its ownership moved from the container start/stop script to caclmgrd, which re-installs it on every control-plane ACL flush-and-rebuild while FEATURE|dhcp_server is enabled (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584). Update generate_expected_rules() to assert the tcp/2514 exception instead of the old udp/514 rule, and gate the expectation on the FEATURE|dhcp_server state that caclmgrd keys on (read via sonic-db-cli), rather than on mere container presence. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>

Add test_caclmgrd_dhcp_server_syslog verifying the docker0 RELP tcp/2514 INPUT exception that caclmgrd now owns (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584): * present above the control-plane catch-all DROP while FEATURE|dhcp_server is enabled, * survives a control-plane ACL flush-and-rebuild (forced via the dummy_acl_rules fixture / acl-loader), and * removed on feature-disable and restored on feature-enable. The test skips when dhcp_server is not enabled on the DUT. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>

Xichen96 requested review from Blueve, StormLiangMS and lguohan July 16, 2026 12:32

Xichen96 force-pushed the dev/xichenlin/caclmgrd-own-dhcp-server-syslog branch from 64c85b3 to e8dac04 Compare July 16, 2026 12:44

Xichen96 requested a review from Copilot July 16, 2026 12:45

Copilot started reviewing on behalf of Xichen96 July 16, 2026 12:45 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Xichen96 requested review from yxieca and removed request for Blueve July 16, 2026 13:11

mssonicbld added the Request for 202605 Branch label Jul 17, 2026

mssonicbld added the Cherry Pick Conflict_202605 label Jul 17, 2026

StormLiangMS reviewed Jul 21, 2026

View reviewed changes

StormLiangMS requested a review from yyynini July 21, 2026 08:29

StormLiangMS approved these changes Jul 21, 2026

View reviewed changes

shreyansh-nexthop reviewed Jul 21, 2026

View reviewed changes

Comment thread scripts/caclmgrd Outdated

Xichen96 force-pushed the dev/xichenlin/caclmgrd-own-dhcp-server-syslog branch from e8dac04 to f913bf9 Compare July 22, 2026 03:54

yyynini approved these changes Jul 22, 2026

View reviewed changes

Xichen96 merged commit 60775ab into sonic-net:master Jul 22, 2026
6 checks passed

Xichen96 added the Tested for 202605 branch Tested for 202605 branch label Jul 22, 2026

Xichen96 mentioned this pull request Jul 22, 2026

[dhcp_server] Wait for caclmgrd-owned docker0 syslog rule on start/stop sonic-net/sonic-buildimage#28580

Open

19 tasks

Xichen96 mentioned this pull request Jul 22, 2026

[cacl][test] Expect dhcp_server docker0 syslog rule on tcp/2514 sonic-net/sonic-mgmt#26477

Open

21 tasks

Xichen96 mentioned this pull request Jul 22, 2026

[cacl][test] Add coverage for caclmgrd-owned dhcp_server syslog rule sonic-net/sonic-mgmt#26478

Open

21 tasks

Uh oh!

Conversation

Xichen96 commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

Work item tracking

How I did it

How to verify it

Hardware validation

Which release branch to backport (provide reason below if selected)

Tested branch

Test result

Description for the changelog

Uh oh!

mssonicbld commented Jul 16, 2026

Uh oh!

azure-pipelines Bot commented Jul 16, 2026

Uh oh!

azure-pipelines Bot commented Jul 16, 2026

Uh oh!

mssonicbld commented Jul 16, 2026

Uh oh!

azure-pipelines Bot commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Xichen96 commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mssonicbld commented Jul 17, 2026

Uh oh!

StormLiangMS left a comment

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Jul 21, 2026

Uh oh!

Uh oh!

mssonicbld commented Jul 22, 2026

Uh oh!

azure-pipelines Bot commented Jul 22, 2026

Uh oh!

yyynini left a comment

Choose a reason for hiding this comment

Uh oh!

StormLiangMS commented Jul 22, 2026

Uh oh!

Xichen96 commented Jul 22, 2026

Uh oh!

Uh oh!

mssonicbld commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Xichen96 commented Jul 16, 2026 •

edited

Loading

Xichen96 commented Jul 16, 2026 •

edited

Loading