Skip to content

Add Kimi-K2.5 MXFP4 PD disaggregation recipe and update configurations#1575

Open
Jasen2201 wants to merge 4 commits into
mainfrom
Jasen/kimi-recipe
Open

Add Kimi-K2.5 MXFP4 PD disaggregation recipe and update configurations#1575
Jasen2201 wants to merge 4 commits into
mainfrom
Jasen/kimi-recipe

Conversation

@Jasen2201

Copy link
Copy Markdown
Contributor

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Jasen2201 and others added 2 commits July 13, 2026 06:38
Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 13, 2026 06:44
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 1575 --add-label <label>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new mesh recipe documenting how to run Kimi-K2.5 (MXFP4) in a 1P+1D prefill/decode-disaggregated setup using ATOM + Mooncake KV transfer + atomesh routing.

Changes:

  • Added a single-node (8-GPU) PD-disaggregation serving recipe for amd/Kimi-K2.5-MXFP4.
  • Included verification (curl), GSM8K accuracy (lm_eval), and serving benchmark commands via the atomesh router.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread recipes/mesh/kimi-k2.md
Comment on lines +126 to +128
## Serving Benchmark (via Router)

```bash
From PR #1580: enable MXFP4 intermediate output for INT4 quick reduce.

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 13, 2026 12:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment thread recipes/mesh/kimi-k2.md
@@ -0,0 +1,148 @@
# Kimi-K2.5 PD Disaggregation with atomesh
Comment thread recipes/mesh/kimi-k2.md
Comment on lines +1 to +5
# Kimi-K2.5 PD Disaggregation with atomesh

PD-disaggregated serving for Kimi-K2.5 (MXFP4, text-only backbone) using the
ATOM native backend, Mooncake RDMA KV transfer, and atomesh routing. Single-node
1P+1D topology.
@zufayu zufayu requested a review from yhl-amd July 14, 2026 05:10
Copilot AI review requested due to automatic review settings July 14, 2026 06:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants