Skip to content

[AMD] Add dsv4-fp4-mi355x-atom-disagg-mtp config#2294

Open
Oseltamivir wants to merge 3 commits into
mainfrom
dsv4-fp4-mi355x-atom-disagg-mtp
Open

[AMD] Add dsv4-fp4-mi355x-atom-disagg-mtp config#2294
Oseltamivir wants to merge 3 commits into
mainfrom
dsv4-fp4-mi355x-atom-disagg-mtp

Conversation

@Oseltamivir

Copy link
Copy Markdown
Collaborator

Summary

Adds a DeepSeek-V4-Pro FP4 MI355X ATOM disaggregated MTP benchmark, dsv4-fp4-mi355x-atom-disagg-mtp (spec-decoding=mtp, num-speculative-tokens=3 via DECODE_MTP_SIZE).

This is config-only — no new benchmark script:

  • Mirrors the dsv4-fp4-mi355x-atom-disagg topology: 2P1D DPA+TP8 (conc 256–2048) and 1P1D TP8 (conc 4–128).
  • Reuses benchmarks/multi_node/dsv4_fp4_mi355x_atom-disagg.sh; the multinode launcher builds the script name without a _mtp suffix, so the same script serves STP and MTP.
  • MTP is driven by the DeepSeek-V4-Pro mtp_flags already present in benchmarks/multi_node/amd_utils/models_atom.yaml (--method mtp --num-speculative-tokens) plus DECODE_MTP_SIZE in the decode additional-settings; server_atom.sh composes the spec args.
  • Results are disambiguated from the base config by the spec-mtp segment in RESULT_FILENAME (no collision).

Image

Bumped to rocm/atom-dev:nightly_202607201550 (2026-07-20) from the base config's stale 2026-06-10 nightly. Verified on-node that this image carries the dsv4 MTP path (deepseek_v4 -> deepseek_v4_mtp -> DeepseekV4MTPModel) and the atomesh disagg router (/usr/local/bin/atomesh).

Draft depth

DECODE_MTP_SIZE=3 on both arms, matching the single-node dsv4_fp4_mi355x_atom_mtp.sh recipe. The Atom MTP head runs autoregressively N times, so N=3 is valid even though dsv4 ships one nextn layer. A shallower high-concurrency depth (per the dsr1-fp4-mi355x-sglang-disagg-mtp concurrency taper) may follow once results land.

Validation

  • generate_sweep_configs.py test-config — passes Pydantic, generates 2 MTP arms, max-model-len 9472 auto-injected.
  • pytest utils/matrix_logic/ — 224 passed.
  • Not yet run on real GPUs (MI355X cluster was saturated); intended to run through the normal run-sweep path.

Note: perf-changelog.yaml pr-link will be updated to this PR's URL in a follow-up commit.

Add a DeepSeek-V4-Pro FP4 MI355X ATOM disaggregated MTP benchmark
(spec-decoding=mtp, num-speculative-tokens=3 via DECODE_MTP_SIZE).

Config-only: mirrors the dsv4-fp4-mi355x-atom-disagg topology (2P1D DPA+TP8
conc 256-2048, 1P1D TP8 conc 4-128) and reuses the existing launch script
benchmarks/multi_node/dsv4_fp4_mi355x_atom-disagg.sh. MTP is driven by the
DeepSeek-V4-Pro mtp_flags already in models_atom.yaml plus DECODE_MTP_SIZE in
the decode additional-settings; the multinode launcher reuses the base script
(no _mtp suffix), so no new benchmark script is needed.

Image bumped to rocm/atom-dev:nightly_202607201550 (from the base config's
stale 2026-06-10 nightly): verified on-node it carries the dsv4 mtp path
(deepseek_v4 -> deepseek_v4_mtp -> DeepseekV4MTPModel) and the atomesh router.

Draft depth 3 on both arms matches the single-node dsv4_fp4_mi355x_atom_mtp.sh
recipe; a shallower high-conc depth may follow the dsr1-fp4-mi355x-sglang-disagg-mtp
concurrency taper once results land.
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

Point the new changelog entry at PR #2294 (was XXX placeholder).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant