bug(router): HTTPConnector and NIXLConnector store per-request state on connector struct, causing data races and request corruption under concurrency

[bhuvan-somisetty]

Description

In pkg/kthena-router/connectors/http.go and pkg/kthena-router/connectors/nixl.go (and by extension mooncake.go), the HTTPConnector and NIXLConnector structs maintain mutable request state (prefillRequest, decodeRequest, and decodeRequestBody) as struct fields across calls to Proxy().

When Kthena Router handles disaggregated prefill-decode requests (proxyToPDDisaggregated), connector instances are created by the connector factory (r.connectorFactory.GetConnector(connectorType)) and reused to handle all incoming traffic for that connector type.

When two or more requests are routed through HTTPConnector, NIXLConnector, or MoonCakeConnector concurrently, multiple goroutines concurrently mutate and read h.prefillRequest, h.decodeRequest, n.prefillRequest, and n.decodeRequestBody on the shared connector instance without synchronization. This causes a severe data race and request corruption: one request's decode payload (n.decodeRequestBody or h.decodeRequest) is overwritten by another concurrent request, causing the wrong prompt, messages, and parameters to be sent to the decode engine.

Steps to reproduce the issue

1. Deploy Kthena with a ModelServer configured for PD disaggregation (e.g. using kvConnector: {type: "nixl"} or type: "http" or type: "mooncake").
2. Send two concurrent chat completion requests with different prompts/models simultaneously (e.g. Request 1 from User A with prompt "User A prompt", Request 2 from User B with prompt "User B prompt").
3. Request 1 and Request 2 call connector.Proxy() simultaneously on separate goroutines.
4. Request 1 sets n.decodeRequestBody to Request 1's payload.
5. Request 2 sets n.decodeRequestBody to Request 2's payload before Request 1 begins its decode phase.
6. Request 1 proceeds to the decode phase (n.buildDecodeRequest(c, n.decodeRequestBody, kvTransferParams)) and reads n.decodeRequestBody, which now contains Request 2's payload.

Evidence and production path

Production execution path:

1. Router invokes proxyToPDDisaggregated:
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/router/router.go#L1363-L1420

2. In HTTPConnector.Proxy:
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/connectors/http.go#L30-L33
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/connectors/http.go#L74-L88
   h.decodeRequest and h.prefillRequest are stored directly on the HTTPConnector struct instance.

3. In NIXLConnector.Proxy (and MoonCakeConnector which aliases NIXLConnector):
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/connectors/nixl.go#L35-L39
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/connectors/nixl.go#L65-L67
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/connectors/nixl.go#L79
   https://github.com/volcano-sh/kthena/blob/713a4a5814526d19b45781a742ea3521d9657065/pkg/kthena-router/connectors/nixl.go#L107-L108
   n.prefillRequest and n.decodeRequestBody are stored on the NIXLConnector struct instance and later read in n.prefill() and n.buildDecodeRequest().

Under concurrent traffic:

• Goroutine 1 writes n.decodeRequestBody = body1
• Goroutine 2 writes n.decodeRequestBody = body2
• Goroutine 1 reads n.decodeRequestBody in n.buildDecodeRequest and sends body2 to the decode pod.

Describe the results you received and expected

Expected Behavior:
KVConnector implementations should be stateless and thread-safe. Prefill and decode requests/bodies should be kept as local variables within Proxy() and passed as arguments, ensuring zero shared state across concurrent requests.

Actual Behavior:
Shared struct fields cause data races and cross-request contamination where one request's decode phase uses another request's body payload.

What version of Kthena are you using?

v0.2.0 (main branch commit 713a4a5814526d19b45781a742ea3521d9657065)

Any other relevant information

Affects HTTPConnector, NIXLConnector, and MoonCakeConnector. By contrast, SGLangConnector (pkg/kthena-router/connectors/sglang.go) correctly uses local variables and avoids shared struct fields.

[bhuvan-somisetty]

/assign