[Bug] MiMo tool-call streaming: text after the last tool call is silently dropped; split bot-token flushes markup into content

[tancheng33]

Checklist

• I searched related issues but found no solution.
• The bug persists in the latest version.
• Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
• If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
• Please use English. Otherwise, it will be closed.

Describe the bug

Found while differential-testing detect_and_parse vs parse_streaming_increment across function-call detectors (same input, one-shot vs chunked, results must agree). MiMoDetector.parse_streaming_increment (srt/function_call/mimo_detector.py) has two divergences from its own one-shot parse:

1. Any normal text after the last tool call is silently dropped (production-realistic).

Once a tool call has been parsed (current_tool_id > 0), every increment whose buffer contains no further <tool_call> returns normal_text="" and keeps buffering forever:

start = current_text.find(self.bot_token)
if start == -1:
    if self.current_tool_id > 0:
        return StreamingParseResult(normal_text="")   # held forever

Text between two calls is only flushed when the next <tool_call> arrives, and text after the final call is never flushed at all — nothing in the streaming serving path drains the detector buffer at end of stream. So in streaming mode, anything MiMo says after its last tool call (explanations, summaries) is lost. This does not depend on chunk boundaries.

d = MiMoDetector()
d.parse_streaming_increment(
    "<tool_call>\n<function=search>\n<parameter=query>ls</parameter>\n</function>\n</tool_call>", tools
)  # call parsed correctly
r = d.parse_streaming_increment("Done, the files are listed.", tools)
# r.normal_text == ''  — the sentence is stuck in d._buffer permanently

2. A <tool_call> marker split across increments destroys the tool call (robustness).

When the buffer holds only a prefix of the bot token, the no-marker branch flushes and clears the whole buffer as normal text, so the marker can never be recognized:

d = MiMoDetector()
d.parse_streaming_increment("I will run it.\n<tool", tools)   # flushes '...<tool' as content
d.parse_streaming_increment("_call>\n<function=search>\n<parameter=query>ls</parameter>\n</function>\n</tool_call>", tools)
# no call detected; the entire raw markup is streamed to the user as content

BaseFormatDetector._ends_with_partial_token exists exactly for this and sibling detectors with the same <tool_call> marker use it (qwen25_detector.py L98; Glm47MoeDetector survives 1-char chunking) — MiMo skips it. MiMo's vocab does make <tool_call> an added token (151657), so SGLang's own detokenizer usually delivers it atomically; this half is mainly about matching the robustness bar the repo has applied elsewhere (e.g. the Trinity split-marker fix, Step3 '<'-in-values fix).

Expected: streamed results (concatenated normal_text + accumulated calls) match detect_and_parse for the same input, as they do for Qwen25/GLM-4.7.

Reproduction

CPU-only, no server:

from sglang.srt.function_call.mimo_detector import MiMoDetector
from sglang.srt.entrypoints.openai.protocol import Function, Tool

tools = [Tool(type="function", function=Function(
    name="search", parameters={"type": "object", "properties": {"query": {"type": "string"}}}))]

# Bug 1 — trailing content loss
d = MiMoDetector()
d.parse_streaming_increment("<tool_call>\n<function=search>\n<parameter=query>ls</parameter>\n</function>\n</tool_call>", tools)
r = d.parse_streaming_increment("Done, the files are listed.", tools)
assert r.normal_text == ""          # sentence lost (held in d._buffer forever)

# Bug 2 — split marker leaks markup
d2 = MiMoDetector()
r1 = d2.parse_streaming_increment("I will run it.\n<tool", tools)
r2 = d2.parse_streaming_increment("_call>\n<function=search>\n<parameter=query>ls</parameter>\n</function>\n</tool_call>", tools)
assert r1.normal_text.endswith("<tool") and r2.calls == []   # markup streamed as content, call gone

Happy to send a fix PR with regression tests: adopt _ends_with_partial_token (as in qwen25_detector) so only a potential marker suffix is held back, and flush non-marker text instead of buffering it unconditionally after the first call.

Environment

Current main, CPU-only reproduction (functions are pure Python), Python 3.12.

---