Skip to content

feat: controller-runtime/operator migration and CI modernization - #681

Open
MaxRink wants to merge 4 commits into
k8snetworkplumbingwg:masterfrom
telekom:upstream-pr-ready
Open

feat: controller-runtime/operator migration and CI modernization#681
MaxRink wants to merge 4 commits into
k8snetworkplumbingwg:masterfrom
telekom:upstream-pr-ready

Conversation

@MaxRink

@MaxRink MaxRink commented Mar 9, 2026

Copy link
Copy Markdown

What Changed

  • Migrated the control-plane from legacy controller binaries to a controller-runtime based operator architecture.
  • Added dedicated reconcilers for IPPool, NodeSlicePool, and OverlappingRange reservations.
  • Added validating webhooks for CRDs with cert bootstrap/rotation and readiness gating.
  • Reworked API/CRD generation paths and regenerated clientsets/informers/listers/applyconfig/openapi artifacts.
  • Updated command/layout/build chain (cmd/*, Makefile, Dockerfile, codegen/update scripts).
  • Updated deployment assets (kustomize config tree + Helm templates/values) to match the new operator/webhook architecture.
  • Expanded unit/integration coverage across controller, webhook, validation, allocator, config, and storage paths.

Why This Is Better

  • Reconciliation is now centralized, explicit, and testable under controller-runtime instead of split legacy binaries.
  • Admission validation is enforced consistently via webhooks, reducing invalid CR state entering reconciliation.
  • Certificate and webhook readiness handling is deterministic, reducing startup race conditions.
  • Deployment and release paths are more consistent across kustomize/Helm/CI.
  • Test coverage is broader in both edge conditions and real e2e behavior, reducing regression risk.

Bugs Fixed

  • Vendor consistency drift around vendored go-openapi README content.
  • Release pipeline digest parsing failure in multi-arch manifest assembly (Image push release).
  • Release pipeline attestation-stage failure caused by SBOM release-asset permission constraints.
  • Multiple controller/webhook validation/reconcile edge-case defects surfaced by newly added tests.

Limitations Removed

  • Removed dependency on the old standalone controller binaries by consolidating logic in one operator runtime.
  • Removed brittle webhook startup ordering by introducing cert readiness gating before registration.
  • Removed release-stage fragility in image metadata/digest flow.
  • Removed several e2e blind spots (dual-stack edge behavior, workload lifecycle stress paths, node-slice interactions).

Internal (Non User-Facing) Logic Changes

  • Added a patch-helper based reconcile update pattern for safer spec/status mutation handling.
  • Standardized shared validation helpers and webhook validation behavior.
  • Improved internal retry/error-handling patterns in storage + reconcile paths.
  • Added richer metrics instrumentation and corresponding tests.

Added / Expanded E2E Coverage

  • Core/basic and lifecycle scenarios under both direct and Helm-installed flows.
  • Workload-focused scenarios (ReplicaSet/StatefulSet behavior and pool consistency checks).
  • Dual-stack and address-family specific validation paths.
  • Node-slice and edge-case scenarios (drain/eviction/churn related behavior).

Notes

  • This PR is intentionally a single commit on top of upstream master for easier review/cherry-pick handling.

@MaxRink
MaxRink requested review from dougbtv and maiqueb as code owners March 9, 2026 19:46
@gemini-code-assist

Copy link
Copy Markdown

Warning

Gemini is experiencing higher than usual traffic and was unable to create the summary. Please try again in a few hours by commenting /gemini summary.

@MaxRink
MaxRink force-pushed the upstream-pr-ready branch from 434113f to d8707c2 Compare March 9, 2026 22:13
@MaxRink

MaxRink commented Mar 9, 2026

Copy link
Copy Markdown
Author

Reviewer guide (high-signal file map)

  1. Operator/runtime architecture
  • cmd/operator/main.go
  • cmd/operator/controller.go
  • internal/controller/setup.go
  1. Reconcilers
  • internal/controller/ippool_controller.go
  • internal/controller/nodeslice_controller.go
  • internal/controller/overlappingrange_controller.go
  • internal/controller/patchhelper.go
  1. Webhooks + cert readiness
  • internal/webhook/setup.go
  • internal/webhook/ippool_webhook.go
  • internal/webhook/nodeslicepool_webhook.go
  • internal/webhook/overlappingrange_webhook.go
  • internal/webhook/certrotator/certrotator.go
  1. API/CRD + generated surface
  • api/whereabouts.cni.cncf.io/v1alpha1/*
  • config/crd/bases/*
  • pkg/generated/*
  1. CNI + install flow
  • cmd/whereabouts/main.go
  • cmd/install-cni/main.go
  • config/daemonset/*
  1. Helm/deploy manifests
  • deployment/whereabouts-chart/templates/operator.yaml
  • deployment/whereabouts-chart/templates/validatingwebhookconfiguration.yaml
  • deployment/whereabouts-chart/templates/operator-clusterrole.yaml
  • deployment/whereabouts-chart/values.yaml
  1. E2E coverage additions
  • e2e/e2e_test.go
  • e2e/e2e_node_slice/e2e_node_slice_test.go
  • e2e/poolconsistency/*
  1. CI/release hardening
  • .github/workflows/build.yml
  • .github/workflows/test.yml
  • .github/workflows/image-push-release.yml
  • .github/workflows/binaries-upload-release.yml
  • .github/workflows/chart-push-release.yml

@MaxRink
MaxRink force-pushed the upstream-pr-ready branch from d8707c2 to 2a765b9 Compare July 31, 2026 13:09
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 194 files, which is 94 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 11a975a2-824a-483e-b109-cfa47f91793a

📥 Commits

Reviewing files that changed from the base of the PR and between 2c10c76 and 3c72f5f.

⛔ Files ignored due to path filters (37)
  • doc/logo-sticker.svg is excluded by !**/*.svg
  • doc/logo.png is excluded by !**/*.png
  • doc/logo.svg is excluded by !**/*.svg
  • go.sum is excluded by !**/*.sum
  • pkg/generated/applyconfiguration/internal/internal.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/utils.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/ipaddressallocation.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/ipallocation.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/ippool.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/ippoolspec.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/ippoolstatus.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/nodesliceallocation.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/nodeslicepool.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/nodeslicepoolspec.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/nodeslicepoolstatus.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservation.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservationspec.go is excluded by !**/generated/**
  • pkg/generated/applyconfiguration/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservationstatus.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/fake/clientset_generated.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/fake/register.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/scheme/register.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/fake/fake_ippool.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/fake/fake_nodeslicepool.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/fake/fake_overlappingrangeipreservation.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/ippool.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/nodeslicepool.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservation.go is excluded by !**/generated/**
  • pkg/generated/clientset/versioned/typed/whereabouts.cni.cncf.io/v1alpha1/whereabouts.cni.cncf.io_client.go is excluded by !**/generated/**
  • pkg/generated/informers/externalversions/factory.go is excluded by !**/generated/**
  • pkg/generated/informers/externalversions/generic.go is excluded by !**/generated/**
  • pkg/generated/informers/externalversions/whereabouts.cni.cncf.io/v1alpha1/ippool.go is excluded by !**/generated/**
  • pkg/generated/informers/externalversions/whereabouts.cni.cncf.io/v1alpha1/nodeslicepool.go is excluded by !**/generated/**
  • pkg/generated/informers/externalversions/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservation.go is excluded by !**/generated/**
  • pkg/generated/listers/whereabouts.cni.cncf.io/v1alpha1/ippool.go is excluded by !**/generated/**
  • pkg/generated/listers/whereabouts.cni.cncf.io/v1alpha1/nodeslicepool.go is excluded by !**/generated/**
  • pkg/generated/listers/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservation.go is excluded by !**/generated/**
  • pkg/generated/openapi/zz_generated.openapi.go is excluded by !**/generated/**
📒 Files selected for processing (194)
  • .dockerignore
  • .github/dependabot.yml
  • .github/workflows/binaries-upload-release.yml
  • .github/workflows/build.yml
  • .github/workflows/chart-push-release.yml
  • .github/workflows/image-build.yml
  • .github/workflows/image-push-master.yml
  • .github/workflows/image-push-release.yml
  • .github/workflows/stale.yml
  • .github/workflows/test.yml
  • .gitignore
  • .golangci.yml
  • CHANGELOG.md
  • CLAUDE.md
  • CONTRIBUTING.md
  • Dockerfile
  • Dockerfile.arm64
  • Makefile
  • PROJECT
  • README.md
  • api/whereabouts.cni.cncf.io/v1alpha1/conditions.go
  • api/whereabouts.cni.cncf.io/v1alpha1/doc.go
  • api/whereabouts.cni.cncf.io/v1alpha1/groupversion_info.go
  • api/whereabouts.cni.cncf.io/v1alpha1/ippool_types.go
  • api/whereabouts.cni.cncf.io/v1alpha1/nodeslicepool_types.go
  • api/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservation_types.go
  • api/whereabouts.cni.cncf.io/v1alpha1/zz_generated.deepcopy.go
  • cmd/controlloop/controlloop.go
  • cmd/install-cni/main.go
  • cmd/install-cni/main_test.go
  • cmd/nodeslicecontroller/node_slice_controller.go
  • cmd/operator/controller.go
  • cmd/operator/main.go
  • cmd/reconciler/main.go
  • cmd/whereabouts.go
  • cmd/whereabouts/main.go
  • cmd/whereabouts/main_test.go
  • config/crd/bases/whereabouts.cni.cncf.io_ippools.yaml
  • config/crd/bases/whereabouts.cni.cncf.io_nodeslicepools.yaml
  • config/crd/bases/whereabouts.cni.cncf.io_overlappingrangeipreservations.yaml
  • config/crd/kustomization.yaml
  • config/daemonset/cni_role.yaml
  • config/daemonset/daemonset.yaml
  • config/daemonset/kustomization.yaml
  • config/default/kustomization.yaml
  • config/manager/kustomization.yaml
  • config/manager/manager.yaml
  • config/prometheus/kustomization.yaml
  • config/prometheus/monitor.yaml
  • config/rbac/kustomization.yaml
  • config/rbac/leader_election_role.yaml
  • config/rbac/leader_election_role_binding.yaml
  • config/rbac/role.yaml
  • config/rbac/role_binding.yaml
  • config/rbac/service_account.yaml
  • config/samples/kustomization.yaml
  • config/samples/whereabouts_v1alpha1_ippool.yaml
  • config/samples/whereabouts_v1alpha1_nodeslicepool.yaml
  • config/samples/whereabouts_v1alpha1_overlappingrangeipreservation.yaml
  • config/webhook/kustomization.yaml
  • config/webhook/manifests.yaml
  • config/webhook/service.yaml
  • deployment/whereabouts-chart/crds/whereabouts.cni.cncf.io_ippools.yaml
  • deployment/whereabouts-chart/crds/whereabouts.cni.cncf.io_nodeslicepools.yaml
  • deployment/whereabouts-chart/crds/whereabouts.cni.cncf.io_overlappingrangeipreservations.yaml
  • deployment/whereabouts-chart/templates/_helpers.tpl
  • deployment/whereabouts-chart/templates/cluster_role.yaml
  • deployment/whereabouts-chart/templates/configmap.yaml
  • deployment/whereabouts-chart/templates/daemonset.yaml
  • deployment/whereabouts-chart/templates/node-slice-controller.yaml
  • deployment/whereabouts-chart/templates/operator-clusterrole.yaml
  • deployment/whereabouts-chart/templates/operator-clusterrolebinding.yaml
  • deployment/whereabouts-chart/templates/operator-pdb.yaml
  • deployment/whereabouts-chart/templates/operator-serviceaccount.yaml
  • deployment/whereabouts-chart/templates/operator.yaml
  • deployment/whereabouts-chart/templates/reconciler.yaml
  • deployment/whereabouts-chart/templates/validatingwebhookconfiguration.yaml
  • deployment/whereabouts-chart/templates/webhook-service.yaml
  • deployment/whereabouts-chart/values.yaml
  • doc/architecture.md
  • doc/crds/daemonset-install.yaml
  • doc/crds/node-slice-controller.yaml
  • doc/crds/reconciler-deployment.yaml
  • doc/crds/whereabouts.cni.cncf.io_ippools.yaml
  • doc/crds/whereabouts.cni.cncf.io_nodeslicepools.yaml
  • doc/crds/whereabouts.cni.cncf.io_overlappingrangeipreservations.yaml
  • doc/developer_notes.md
  • doc/extended-configuration.md
  • doc/metrics.md
  • doc/proposals/operator-controller-runtime-migration.md
  • doc/sample_config.json
  • e2e/client/ippool.go
  • e2e/client/nodeslicepool.go
  • e2e/client/pod.go
  • e2e/client/replicaset.go
  • e2e/client/statefulset.go
  • e2e/client/whereabouts.go
  • e2e/e2e_node_slice/e2e_node_slice_test.go
  • e2e/e2e_test.go
  • e2e/entities/helpers.go
  • e2e/poolconsistency/checker.go
  • e2e/poolconsistency/node_slice_checker.go
  • e2e/poolconsistency/poolconsistency_test.go
  • e2e/retrievers/pod.go
  • e2e/testenvironment/config.go
  • e2e/util/util.go
  • go.mod
  • hack/build-go.sh
  • hack/cni-install.yml
  • hack/e2e-get-test-tools.sh
  • hack/e2e-setup-kind-cluster-helm.sh
  • hack/e2e-setup-kind-cluster.sh
  • hack/generate-code.sh
  • hack/install-kubebuilder-tools.sh
  • hack/openapi-violations.list
  • hack/test-go.sh
  • hack/tools/openapi-schema/main.go
  • hack/update-codegen.sh
  • hack/verify-codegen.sh
  • internal/controller/conditions.go
  • internal/controller/controller_extended_test.go
  • internal/controller/ippool_controller.go
  • internal/controller/ippool_controller_test.go
  • internal/controller/metrics.go
  • internal/controller/metrics_test.go
  • internal/controller/nodeslice_controller.go
  • internal/controller/nodeslice_controller_test.go
  • internal/controller/overlappingrange_controller.go
  • internal/controller/overlappingrange_controller_test.go
  • internal/controller/patchhelper.go
  • internal/controller/patchhelper_test.go
  • internal/controller/setup.go
  • internal/controller/suite_test.go
  • internal/validation/validation.go
  • internal/validation/validation_extended_test.go
  • internal/validation/validation_test.go
  • internal/webhook/certrotator/certrotator.go
  • internal/webhook/certrotator/certrotator_test.go
  • internal/webhook/certrotator/suite_test.go
  • internal/webhook/ippool_webhook.go
  • internal/webhook/ippool_webhook_test.go
  • internal/webhook/metrics.go
  • internal/webhook/metrics_test.go
  • internal/webhook/nodeslicepool_webhook.go
  • internal/webhook/nodeslicepool_webhook_test.go
  • internal/webhook/overlappingrange_webhook.go
  • internal/webhook/overlappingrange_webhook_test.go
  • internal/webhook/setup.go
  • internal/webhook/suite_test.go
  • pkg/allocate/allocate.go
  • pkg/allocate/allocate_test.go
  • pkg/api/whereabouts.cni.cncf.io/register.go
  • pkg/api/whereabouts.cni.cncf.io/v1alpha1/ippool_types.go
  • pkg/api/whereabouts.cni.cncf.io/v1alpha1/nodeslicepool_types.go
  • pkg/api/whereabouts.cni.cncf.io/v1alpha1/overlappingrangeipreservation_types.go
  • pkg/config/config.go
  • pkg/config/config_extended_test.go
  • pkg/config/config_test.go
  • pkg/controlloop/dummy_controller.go
  • pkg/controlloop/entity_generators.go
  • pkg/controlloop/pod.go
  • pkg/controlloop/pod_controller_test.go
  • pkg/iphelpers/iphelpers.go
  • pkg/iphelpers/iphelpers_extended_test.go
  • pkg/iphelpers/iphelpers_test.go
  • pkg/logging/logging.go
  • pkg/logging/logging_extended_test.go
  • pkg/logging/logging_test.go
  • pkg/node-controller/controller.go
  • pkg/node-controller/controller_test.go
  • pkg/node-controller/signals/signals.go
  • pkg/node-controller/signals/signals_posix.go
  • pkg/reconciler/config.go
  • pkg/reconciler/config_test.go
  • pkg/reconciler/ip.go
  • pkg/reconciler/ip_test.go
  • pkg/reconciler/iploop.go
  • pkg/reconciler/wrappedPod.go
  • pkg/reconciler/wrappedPod_test.go
  • pkg/storage/kubernetes/client.go
  • pkg/storage/kubernetes/errors.go
  • pkg/storage/kubernetes/extended_test.go
  • pkg/storage/kubernetes/ipam.go
  • pkg/storage/kubernetes/ipam_test.go
  • pkg/storage/kubernetes/upstream_bugs_test.go
  • pkg/storage/storage.go
  • pkg/storage/storage_test.go
  • pkg/types/types.go
  • pkg/types/types_test.go
  • pkg/version/version.go
  • pkg/version/version_test.go
  • script/install-cni.sh
  • script/lib.sh
  • script/token-watcher.sh

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The migration removes pkg/reconciler and the gocron dependency, and drops the
bin/ip-reconciler build. cmd/reconciler/main.go still imports both, so
go build ./... fails. Nothing references the binary any more. IPPoolReconciler
now performs the orphan cleanup this command used to run on a cron schedule.
PatchHelper wrote spec changes with a plain merge patch and no resourceVersion
precondition. The CNI plugin writes the same Allocations map and guards its own
JSON patch with a test on /metadata/resourceVersion. A reconcile that read a
stale pool could therefore delete an allocation key that the plugin had already
reassigned to a new pod. That frees an in-use IP and allows a duplicate
allocation.

Send the spec patch with MergeFromWithOptimisticLock. On conflict the reconcile
requeues and reruns against a fresh read. Add IsConflictError, because Patch
returns errors through kerrors.NewAggregate and that aggregate has no Unwrap,
so apierrors.IsConflict alone does not see the wrapped conflict.
@MaxRink
MaxRink force-pushed the upstream-pr-ready branch from a2a8997 to b4e9822 Compare August 20, 2026 20:16
@MaxRink
MaxRink force-pushed the upstream-pr-ready branch from b4e9822 to 3c72f5f Compare August 20, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant