[AMD] Add dsv4-fp4-mi355x-atom-disagg-mtp config#2294
Conversation
Add a DeepSeek-V4-Pro FP4 MI355X ATOM disaggregated MTP benchmark (spec-decoding=mtp, num-speculative-tokens=3 via DECODE_MTP_SIZE). Config-only: mirrors the dsv4-fp4-mi355x-atom-disagg topology (2P1D DPA+TP8 conc 256-2048, 1P1D TP8 conc 4-128) and reuses the existing launch script benchmarks/multi_node/dsv4_fp4_mi355x_atom-disagg.sh. MTP is driven by the DeepSeek-V4-Pro mtp_flags already in models_atom.yaml plus DECODE_MTP_SIZE in the decode additional-settings; the multinode launcher reuses the base script (no _mtp suffix), so no new benchmark script is needed. Image bumped to rocm/atom-dev:nightly_202607201550 (from the base config's stale 2026-06-10 nightly): verified on-node it carries the dsv4 mtp path (deepseek_v4 -> deepseek_v4_mtp -> DeepseekV4MTPModel) and the atomesh router. Draft depth 3 on both arms matches the single-node dsv4_fp4_mi355x_atom_mtp.sh recipe; a shallower high-conc depth may follow the dsr1-fp4-mi355x-sglang-disagg-mtp concurrency taper once results land.
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Point the new changelog entry at PR #2294 (was XXX placeholder).
Summary
Adds a DeepSeek-V4-Pro FP4 MI355X ATOM disaggregated MTP benchmark,
dsv4-fp4-mi355x-atom-disagg-mtp(spec-decoding=mtp,num-speculative-tokens=3viaDECODE_MTP_SIZE).This is config-only — no new benchmark script:
dsv4-fp4-mi355x-atom-disaggtopology: 2P1D DPA+TP8 (conc 256–2048) and 1P1D TP8 (conc 4–128).benchmarks/multi_node/dsv4_fp4_mi355x_atom-disagg.sh; the multinode launcher builds the script name without a_mtpsuffix, so the same script serves STP and MTP.mtp_flagsalready present inbenchmarks/multi_node/amd_utils/models_atom.yaml(--method mtp --num-speculative-tokens) plusDECODE_MTP_SIZEin the decodeadditional-settings;server_atom.shcomposes the spec args.spec-mtpsegment inRESULT_FILENAME(no collision).Image
Bumped to
rocm/atom-dev:nightly_202607201550(2026-07-20) from the base config's stale 2026-06-10 nightly. Verified on-node that this image carries the dsv4 MTP path (deepseek_v4 -> deepseek_v4_mtp -> DeepseekV4MTPModel) and the atomesh disagg router (/usr/local/bin/atomesh).Draft depth
DECODE_MTP_SIZE=3on both arms, matching the single-nodedsv4_fp4_mi355x_atom_mtp.shrecipe. The Atom MTP head runs autoregressively N times, so N=3 is valid even though dsv4 ships one nextn layer. A shallower high-concurrency depth (per thedsr1-fp4-mi355x-sglang-disagg-mtpconcurrency taper) may follow once results land.Validation
generate_sweep_configs.py test-config— passes Pydantic, generates 2 MTP arms,max-model-len 9472auto-injected.pytest utils/matrix_logic/— 224 passed.