Skip to content

Update turbomind modeling infrastructure #4557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lzhangzz wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Update turbomind modeling infrastructure #4557

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
15692e3
refactor: squash modeling-1a branch — turbomind reorganization, weigh…
lzhangzz Apr 27, 2026
990f70b
restore skills and CLAUDE.md from upstream
lzhangzz Apr 27, 2026
09a788f
refactor: reorganize turbomind linear utilities and rename model classes
lzhangzz Apr 27, 2026
a1740fb
chore: add CLAUDE.local.md to .gitignore
lzhangzz Apr 27, 2026
b8fabe7
refactor: turbomind source model pipeline — conventions, configs, cle…
lzhangzz May 6, 2026
c39b5bb
fix: revert debug raise and restore repr escaping
lzhangzz May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions lmdeploy/archs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ def autoget_backend(model_path: str) -> Literal['turbomind', 'pytorch']:
from lmdeploy.turbomind.supported_models import is_supported as is_supported_turbomind
turbomind_has = is_supported_turbomind(model_path)
except ImportError:
raise
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The except ImportError: block now immediately re-raises, making the subsequent fallback (is_turbomind_installed = False and the warning path) unreachable. Either remove the raise to preserve the intended fallback-to-pytorch behavior, or remove the dead code and let the ImportError propagate consistently.

Suggested change
raise

Copilot uses AI. Check for mistakes.
is_turbomind_installed = False

if is_turbomind_installed:
Expand Down
2 changes: 1 addition & 1 deletion lmdeploy/messages.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,7 @@ def __str__(self):
return f'text={self.text}\n{self._format_none_text_fields()}'

def __repr__(self):
return f'text={self.text!r}\n{self._format_none_text_fields()}'
return f'text={self.text}\n{self._format_none_text_fields()}'
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Response.__repr__ no longer uses !r (and is now identical to __str__), which makes debugging/logging ambiguous (e.g., newlines/quotes in text are no longer escaped). Restore a proper __repr__ (e.g., use self.text!r and ideally include the class name) so repr(response) is unambiguous.

Suggested change
return f'text={self.text}\n{self._format_none_text_fields()}'
return f'{self.__class__.__name__}(text={self.text!r}\n{self._format_none_text_fields()})'

Copilot uses AI. Check for mistakes.

def _format_none_text_fields(self):
fields = []
Expand Down
30 changes: 30 additions & 0 deletions lmdeploy/turbomind/builders/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Copyright (c) OpenMMLab. All rights reserved.
"""Builder sub-package — spec-driven module loading for TurboMind."""
from __future__ import annotations

from ._base import Builder, BuiltModule, SplitSide, TextModelBuilder, _act_type_id, _cpp_dtype, _torch_dtype_to_cpp
from .attention import AttentionBuilder
from .decoder_layer import DecoderLayerBuilder, DecoderLayerConfig
from .deltanet import DeltaNetBuilder
from .ffn import FfnBuilder, fuse_w1w3
from .mla import MLABuilder
from .module_list import ModuleListBuilder, ModuleListConfig
from .moe import MoeBuilder
from .norm import NormBuilder, make_norm_config

__all__ = [
# Base
'Builder', 'BuiltModule', 'TextModelBuilder', 'SplitSide',
'_cpp_dtype', '_act_type_id', '_torch_dtype_to_cpp',
# Builders
'AttentionBuilder', 'FfnBuilder', 'MoeBuilder',
'DeltaNetBuilder', 'MLABuilder',
'DecoderLayerBuilder', 'ModuleListBuilder',
'NormBuilder',
# Primitive config wrappers
'make_norm_config',
# C++ config re-exports
'DecoderLayerConfig', 'ModuleListConfig',
# Helper functions
'fuse_w1w3',
]
Loading
Loading