Ingress becomes unreachable after changing Service type from LoadBalancer to NodePort when sharing the same VIP and port

[dhkim07]

Describe the bug

When a Kubernetes Service of type LoadBalancer and an Ingress resource share the same VIP (frontend-ip) and port, traffic is served correctly. However, when the Service type is changed from LoadBalancer to NodePort, the NSIC deletes the csvserver created for the LoadBalancer Service, but fails to create a new csvserver for the Ingress because the initial Ingress csvserver creation had already failed with errorcode=273 ("Resource already exists") during the initial setup — and this is never retried after the conflicting csvserver is removed.

As a result, the Ingress loses its frontend csvserver entirely and becomes unreachable, even though the Kubernetes Ingress resource still exists and reports the correct ADDRESS.

Root Cause Analysis

1. During initial creation, both the LoadBalancer Service and the Ingress attempt to create a csvserver on <VIP>:80.
2. The LoadBalancer Service csvserver is created first.
3. The Ingress csvserver creation fails with: Failed to create CSvserver with IP: <VIP>, Another Vserver already exists with same IP:Port. errorcode=273
4. However, since the LoadBalancer csvserver was also handling Ingress traffic, there was no service disruption at this point.
5. When the Service type is changed to NodePort, NSIC correctly deletes the LoadBalancer csvserver.
6. However, NSIC does not re-attempt to create the Ingress csvserver that previously failed — leaving the Ingress without a frontend, causing a complete service outage.
7. Additionally, repeated errorcode=258 ("No such resource") errors appear as NSIC tries to bind members to already-deleted servicegroups.

To Reproduce

1. Deploy a Pod, a Service of type LoadBalancer, and an Ingress that share the same VIP and port:

kind: Pod
apiVersion: v1
metadata:
  name: apple-app
  labels:
    app: apple
spec:
  containers:
    - name: apple-app
      image: <REGISTRY>/library/echo-server:latest
      args:
        - "-text=apple"
---
apiVersion: v1
kind: Service
metadata:
  name: apple-svc
  labels:
    name: apple
spec:
  type: LoadBalancer
  loadBalancerIP: "<VIP>"
  ports:
  - name: http
    port: 80
    targetPort: 8080
  selector:
    app: apple
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: apple-ingress
  annotations:
    ingress.citrix.com/frontend-ip: "<VIP>"
    ingress.citrix.com/insecure-port: "80"
spec:
  rules:
   - host: <DOMAIN>
     http:
      paths:
      - backend:
          service:
            name: apple-svc
            port:
              number: 80
        path: /apple
        pathType: Prefix

2. Verify traffic is working:

$ curl -I http://<DOMAIN>/apple --resolve <DOMAIN>:80:<VIP>
HTTP/1.1 200 OK

3. Change the Service type from LoadBalancer to NodePort:

apiVersion: v1
kind: Service
metadata:
  name: apple-svc
  labels:
    name: apple
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 8080
  selector:
    app: apple

4. Verify Kubernetes state looks correct:

$ kubectl get ing
NAME            CLASS    HOSTS      ADDRESS   PORTS   AGE
apple-ingress   citrix   <DOMAIN>   <VIP>     80      4m24s

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
apple-svc    NodePort    <CLUSTER-IP>     <none>        80:32628/TCP   6m3s

5. Observe that traffic is now broken:

$ curl -I http://<DOMAIN>/apple --resolve <DOMAIN>:80:<VIP>
curl: (28) Failed to connect to <DOMAIN> port 80 after 75001 ms: Couldn't connect to server

• Version of the NetScaler Ingress Controller: 3.3.2
• Kubernetes Version: v1.32.10

Expected behavior

When the Service type is changed from LoadBalancer to NodePort:

1. The NSIC should delete the LoadBalancer-associated csvserver (this already works correctly).
2. The NSIC should then reconcile the Ingress configuration and create the Ingress csvserver on the now-available VIP:Port, since the conflicting csvserver no longer exists.
3. Traffic to the Ingress should continue to work without interruption.

In general, if an Ingress resource is actively referencing a Service, changes to the Service type should not cause the Ingress to become permanently unreachable. The controller should re-evaluate and recover any previously failed configurations when the blocking condition is resolved.

Logs

Logs after initial creation (LoadBalancer + Ingress — traffic works)

Key error during Ingress csvserver creation:

ERROR - [nitrointerface.py:configure_ns_cs_app:7015] (MainThread)
Exception during configuraton of CS/LB apps: Failed to create CSvserver with IP: <VIP>,
Another Vserver already exists with same IP:Port. Please use a different IP address.
errorcode=273, message=Resource already exists

Logs after changing Service type to NodePort (traffic breaks)

LoadBalancer csvserver deleted successfully:

INFO - [nitrointerface.py:_delete_nsapp_cs_vserver:3465] (MainThread)
csvserver u695c92a6a-apple-svc_80_default_svc is deleted successfully

Repeated errors trying to bind to already-deleted servicegroups:

ERROR - [nitrointerface.py:_configure_services_nondesired:3430] (MainThread)
Nitro Exception while binding group member to servicegroup
u695c92a6a-apple-svc_80_sgp_kj3ur2tfsiuuxrxaruikcazoai5izv7l
errorcode=258 message=No such resource [serviceGroupName, ...]

ERROR - [nitrointerface.py:_configure_services_nondesired:3430] (MainThread)
Nitro Exception while binding group member to servicegroup
u695c92a6a-apple-svc_80_sgp_elynfyxsgku5b5jj6tbvtdfbbcnnuai6
errorcode=258 message=No such resource [serviceGroupName, ...]

(This error repeats 10+ times for the same deleted servicegroups.)

No log entry shows a re-attempt to create the Ingress csvserver after the conflicting LoadBalancer csvserver was removed.

Additional context

Feature / Improvement Request

We understand that using the same VIP and port for both a LoadBalancer Service and an Ingress is not a recommended configuration. However, users may not be aware of this constraint and can easily create this situation unintentionally.

Requested improvements:

1. Reconciliation on conflict resolution: When a csvserver creation fails due to errorcode=273 (IP:Port conflict), the NSIC should track this failure and re-attempt the creation when the conflicting csvserver is deleted.
2. Graceful Service type transitions: If an Ingress is actively using a backend Service, changing the Service type from LoadBalancer to NodePort should not cause a permanent outage for the Ingress. The controller should recover the Ingress frontend configuration.
3. Stale servicegroup cleanup: The repeated errorcode=258 errors for non-existent servicegroups suggest that NSIC maintains stale references after deletion. These should be cleaned up to avoid unnecessary error logging.

If you need faster resolution or need to share logs, please feel free to reach out to us via email at netscaler-appmodernization@cloud.com. We'll be happy to assist you promptly.