feat(sglang): pool DeepSeek-V4 SWA KV cache - #400
Draft
shipiyouniao wants to merge 4 commits into
Draft
Conversation
shipiyouniao
force-pushed
the
feat/sglang-dsv4-kv-pool-bridge
branch
2 times, most recently
from
July 24, 2026 05:08
45a0105 to
90f8247
Compare
shipiyouniao
force-pushed
the
feat/sglang-dsv4-kv-pool-bridge
branch
3 times, most recently
from
August 19, 2026 11:08
0a8a044 to
af6d1c7
Compare
shipiyouniao
force-pushed
the
feat/sglang-dsv4-kv-pool-bridge
branch
2 times, most recently
from
August 26, 2026 10:22
af6d1c7 to
91df367
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a stacked follow-up to #376 and part of #369. It adds an opt-in SGLang DeepSeek-V4 bridge that lets kvcached own the uncompressed SWA KV buffers and their physical page lifecycle.
This PR must not merge before #376. The current branch contains the #376 commits because the allocator compatibility and runtime-reservation accounting are hard dependencies. After #376 merges, I will rebase this PR so that only the SWA pooling commit remains.
Ownership boundary
kvcached owns:
SGLang continues to own:
The full-token allocator used by SGLang's composite SWA allocator is logical-only after the bridge is active. It does not reserve a second physical KV payload.
Activation and failure behavior
The bridge is disabled by default and requires:
The per-layer layout is required because the current DSV4 kernels expect each layer's SWA buffer to be contiguous. With
KVCACHED_REQUIRE=1, an incompatible runtime shape fails startup. Otherwise the adapter logs the incompatibility and leaves the native SGLang buffers in place.The runtime-reservation patch from #376 excludes the native SWA reservation only after the bridge marks the SWA pool as successfully managed. Compressed/indexer/state reservations remain accounted as runtime-owned memory.
Validation
Focused regression tests:
The full upstream pre-commit and typing matrix was reproduced on Linux and passed:
The tests verify opt-in behavior, replacement of the real SWA buffers, preservation of runtime-owned side pools, physical allocation only on TP rank 0, logical allocation on peer TP ranks, reservation accounting, and idempotent patching.
This PR does not claim complete DeepSeek-V4 pool ownership or a performance improvement. GPU runtime validation remains required before the opt-in bridge is considered production-ready.
Closes no issue; #369 should remain open for the remaining DeepSeek-V4 pool families and runtime coverage.