Skip to content

[Bug Fix]Fix m_indices argument passing in BF16 grouped contiguous warmup - #27002

Closed
laurxh wants to merge 1 commit into
sgl-project:mainfrom
laurxh:fix/deep-gemm-warmup
Closed

[Bug Fix]Fix m_indices argument passing in BF16 grouped contiguous warmup#27002
laurxh wants to merge 1 commit into
sgl-project:mainfrom
laurxh:fix/deep-gemm-warmup

Conversation

@laurxh

@laurxh laurxh commented Jun 2, 2026

Copy link
Copy Markdown

Motivation

DeepGEMM >= 2.5.0 (PR deepseek-ai/DeepGEMM#280) renamed the 4th parameter of
m_grouped_bf16_gemm_nt_contiguous from m_indices to grouped_layout.
The BF16 grouped contiguous warmup executor in SGLang uses keyword argument
m_indices=, which causes a TypeError on startup:

TypeError: m_grouped_bf16_gemm_nt_contiguous(): incompatible function arguments. The following argument types are supported:                                            
   1. (a: torch.Tensor, b: torch.Tensor, d: torch.Tensor, grouped_layout: torch.Tensor, compiled_dims: str = 'nk', use_psum_layout: bool = False,                      
expected_m_for_psum_layout: typing.SupportsInt | None = None) -> None

Reproduce:

python -c "
from sglang.srt.layers.deep_gemm_wrapper.compile_utils import _BF16GroupedContWarmupExecutor
executor = _BF16GroupedContWarmupExecutor(max_m=128, n=128, k=128, num_groups=4)
executor.execute(m=128)
"

Modifications

  • Changed m_indices=self.m_indices[:m] (keyword) to self.m_indices[:m] (positional)
    in _BF16GroupedContWarmupExecutor.execute() to match the DeepGEMM pybind11 API.

  • The parameter functionality is unchanged — it is still an int32 tensor indicating which
    expert group each row belongs to. Only the parameter name was changed upstream.

Checklist


CI States

Latest PR Test (Base): ❌ Run #26806013714
Latest PR Test (Extra): ❌ Run #26806013517

The deep_gemm API changed m_indices from a keyword argument to a
positional argument. Pass it positionally to avoid TypeError.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@laurxh
laurxh marked this pull request as draft June 2, 2026 05:18
@laurxh
laurxh marked this pull request as ready for review June 2, 2026 05:58
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@laurxh
laurxh force-pushed the fix/deep-gemm-warmup branch from dd35da1 to cf0b822 Compare June 2, 2026 07:47
@laurxh laurxh closed this Jun 3, 2026
@laurxh
laurxh deleted the fix/deep-gemm-warmup branch June 9, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant