Fix crash when a streaming gr.ChatInterface function yields nothing - #13671
Merged
Conversation
A streaming gr.ChatInterface crashed with "RuntimeError: async generator raised StopAsyncIteration" when the chat function returned before yielding anything, instead of producing no bot response. _stream_fn guarded the first-item fetch with `except StopIteration`, but nothing in that block can raise it: async_iteration is `await anext(iterator)`, and for sync chat functions SyncToAsyncIterator has already converted StopIteration into StopAsyncIteration. The guard has been unreachable since #5116 converted _stream_fn to an async generator without updating the except clause. Because the branch was dead, it also never kept up with additional_outputs (#10071). The event outputs are [null_component, chatbot] + additional_outputs, so the extra slots are now padded with skip(), leaving those components untouched rather than clearing them. The yield itself has to stay. Yielding nothing sends None for the chatbot, and the frontend applies that as an empty chat: the substitution that keeps a normal generator's final payload intact never runs for a generator that yields nothing.
Collaborator
🪼 branch checks and previews
Install Gradio from this PR pip install https://huggingface.co/buckets/gradio/pypi-previews/resolve/5ab7552381aea5353f93918c3a4dfb535b6374bc/gradio-6.20.0-py3-none-any.whlInstall Gradio Python Client from this PR pip install "gradio-client @ git+https://github.com/gradio-app/gradio@5ab7552381aea5353f93918c3a4dfb535b6374bc#subdirectory=client/python"Import Gradio JS Client from this PR via CDN import { Client } from "https://huggingface.co/buckets/gradio/npm-previews/resolve/5ab7552381aea5353f93918c3a4dfb535b6374bc/browser.js"; |
Collaborator
🦄 change detectedThis Pull Request includes changes to the following packages.
|
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes a crash in Gradio’s backend ChatInterface streaming path when a (sync or async) generator returns without yielding any items, ensuring the API returns a valid “no bot response” payload (and preserves chat history) instead of raising RuntimeError: async generator raised StopAsyncIteration.
Changes:
- Update
_stream_fnto catchStopAsyncIterationwhen the first streamed item is missing, and yield a correctly shaped output tuple (includinggr.skip()for anyadditional_outputs). - Import and use
skip()inchat_interface.pyto keep additional outputs unchanged when the generator yields nothing. - Add API tests covering empty sync generator, empty async generator, and empty generator with
additional_outputs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
gradio/chat_interface.py |
Catches StopAsyncIteration during initial stream fetch and yields the correct number of outputs (including skip() for extra outputs). |
test/test_chat_interface.py |
Adds regression tests ensuring empty streaming generators don’t crash, preserve history, and behave correctly with additional_outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
self.additional_outputs is always a list, so splatting a per-output skip() covers the no-additional-outputs case too and the branch goes away. Build the list with a comprehension rather than `[skip()] * n`: the latter repeats one dict, and postprocess_data passes prediction dicts straight into delete_none and postprocess_update_dict, both of which mutate in place. Harmless while skip() has no "value" key, but there is no reason to share the dict.
hysts
marked this pull request as ready for review
July 27, 2026 09:19
Collaborator
Author
|
Thanks for the review @abidlabs ! Merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A streaming
gr.ChatInterfacecrashed withRuntimeError: async generator raised StopAsyncIterationwhenever the chat function returned before yielding anything, instead of simply producing no bot response._stream_fnfetches the first item withawait utils.async_iteration(generator)but guarded it withexcept StopIteration. Nothing in that block can raiseStopIteration:async_iterationisawait anext(iterator), which raisesStopAsyncIteration, and for sync chat functionsSyncToAsyncIterator.run_sync_iterator_asynchas already convertedStopIterationintoStopAsyncIteration. So the guard never fired, theStopAsyncIterationescaped the_stream_fnasync generator frame, and Python turned it into aRuntimeError.The guard has been dead since #5116, which converted
_stream_fnfrom a sync generator to an async one and changednext(generator)toawait async_iteration(generator)without updating theexceptclause alongside it. Before that commit the guard worked, so an empty generator producing no response was the original intended behaviour rather than something new.This corrects the exception type and, while the handler is finally reachable again, brings it in line with the rest of the method. The handler still yields the history, because that is what keeps the chatbot populated: if the handler yielded nothing at all,
postprocess_datawould emitNonefor every output, and the frontend applies anullchatbot value as an empty chat, wiping the conversation. A normal generator avoids that because the final payload of a generating run has itsNoneplaceholders replaced with the last real value, but a generator that yields nothing never enters generating mode, so that substitution never happens.The other change is the output count. The event's outputs are
[null_component, chatbot] + additional_outputs, and every other branch in_stream_fnalready varies its yield accordingly, but this branch always yielded two values. It predatesadditional_outputs(#10071) by more than a year, and since it was unreachable by then it was never updated. Withadditional_outputsconfigured it therefore raisedValueError: A function didn't return enough output valuesinstead of theRuntimeError. The additional outputs getgr.skip()so a turn that produced no response leaves them untouched rather than clearing them.Closes: #13663
Testing
Three cases added to
TestAPIintest/test_chat_interface.py, covering a sync generator, an async generator, and a generator used withadditional_outputs. Each asserts that normal input still streams and that the empty case produces no response. The first two also assert that the skipped turn leaves the history intact and still records the user message, so the following turn sees three messages.All three fail on
mainwith theRuntimeErrorabove. They also fail on the two tempting partial fixes: catchingStopAsyncIterationbut returning instead of yielding drops the history (history_len=2where 3 is expected), and only widening theexceptclause leaves theadditional_outputscase raising theValueError.The reproduction in the issue is enough for a manual check: sending
skipnow leaves the message in the chat with no bot reply and the textbox re-enabled, instead of erroring out.Before / after Spaces
Same app on both. Send
hello, thenskip.AI Disclosure
We encourage the use of AI tooling in creating PRs, but the any non-trivial use of AI needs be disclosed. E.g. if you used Claude to write a first draft, you should mention that. Trivial tab-completion doesn't need to be disclosed. You should self-review all PRs, especially if they were generated with AI.
🎯 PRs Should Target Issues
Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.
Not adhering to this guideline will result in the PR being closed.
Testing and Formatting Your Code
PRs will only be merged if tests pass on CI. We recommend at least running the backend tests locally, please set up your Gradio environment locally and run the backed tests:
bash scripts/run_backend_tests.shPlease run these bash scripts to automatically format your code:
bash scripts/format_backend.sh, and (if you made any changes to non-Python files)bash scripts/format_frontend.sh