Skip to content

Fix crash when a streaming gr.ChatInterface function yields nothing - #13671

Merged
hysts merged 4 commits into
mainfrom
fix/chatinterface-empty-generator-crash
Jul 29, 2026
Merged

Fix crash when a streaming gr.ChatInterface function yields nothing#13671
hysts merged 4 commits into
mainfrom
fix/chatinterface-empty-generator-crash

Conversation

@hysts

@hysts hysts commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Description

A streaming gr.ChatInterface crashed with RuntimeError: async generator raised StopAsyncIteration whenever the chat function returned before yielding anything, instead of simply producing no bot response.

_stream_fn fetches the first item with await utils.async_iteration(generator) but guarded it with except StopIteration. Nothing in that block can raise StopIteration: async_iteration is await anext(iterator), which raises StopAsyncIteration, and for sync chat functions SyncToAsyncIterator.run_sync_iterator_async has already converted StopIteration into StopAsyncIteration. So the guard never fired, the StopAsyncIteration escaped the _stream_fn async generator frame, and Python turned it into a RuntimeError.

The guard has been dead since #5116, which converted _stream_fn from a sync generator to an async one and changed next(generator) to await async_iteration(generator) without updating the except clause alongside it. Before that commit the guard worked, so an empty generator producing no response was the original intended behaviour rather than something new.

This corrects the exception type and, while the handler is finally reachable again, brings it in line with the rest of the method. The handler still yields the history, because that is what keeps the chatbot populated: if the handler yielded nothing at all, postprocess_data would emit None for every output, and the frontend applies a null chatbot value as an empty chat, wiping the conversation. A normal generator avoids that because the final payload of a generating run has its None placeholders replaced with the last real value, but a generator that yields nothing never enters generating mode, so that substitution never happens.

The other change is the output count. The event's outputs are [null_component, chatbot] + additional_outputs, and every other branch in _stream_fn already varies its yield accordingly, but this branch always yielded two values. It predates additional_outputs (#10071) by more than a year, and since it was unreachable by then it was never updated. With additional_outputs configured it therefore raised ValueError: A function didn't return enough output values instead of the RuntimeError. The additional outputs get gr.skip() so a turn that produced no response leaves them untouched rather than clearing them.

Closes: #13663

Testing

Three cases added to TestAPI in test/test_chat_interface.py, covering a sync generator, an async generator, and a generator used with additional_outputs. Each asserts that normal input still streams and that the empty case produces no response. The first two also assert that the skipped turn leaves the history intact and still records the user message, so the following turn sees three messages.

All three fail on main with the RuntimeError above. They also fail on the two tempting partial fixes: catching StopAsyncIteration but returning instead of yielding drops the history (history_len=2 where 3 is expected), and only widening the except clause leaves the additional_outputs case raising the ValueError.

The reproduction in the issue is enough for a manual check: sending skip now leaves the message in the chat with no bot reply and the textbox re-enabled, instead of erroring out.

Before / after Spaces

Same app on both. Send hello, then skip.

AI Disclosure

We encourage the use of AI tooling in creating PRs, but the any non-trivial use of AI needs be disclosed. E.g. if you used Claude to write a first draft, you should mention that. Trivial tab-completion doesn't need to be disclosed. You should self-review all PRs, especially if they were generated with AI.

  • I used AI to investigate the root cause and implement the fix.
  • I did not use AI

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Testing and Formatting Your Code

  1. PRs will only be merged if tests pass on CI. We recommend at least running the backend tests locally, please set up your Gradio environment locally and run the backed tests: bash scripts/run_backend_tests.sh

  2. Please run these bash scripts to automatically format your code: bash scripts/format_backend.sh, and (if you made any changes to non-Python files) bash scripts/format_frontend.sh

A streaming gr.ChatInterface crashed with "RuntimeError: async
generator raised StopAsyncIteration" when the chat function returned
before yielding anything, instead of producing no bot response.

_stream_fn guarded the first-item fetch with `except StopIteration`,
but nothing in that block can raise it: async_iteration is
`await anext(iterator)`, and for sync chat functions
SyncToAsyncIterator has already converted StopIteration into
StopAsyncIteration. The guard has been unreachable since #5116
converted _stream_fn to an async generator without updating the
except clause.

Because the branch was dead, it also never kept up with
additional_outputs (#10071). The event outputs are
[null_component, chatbot] + additional_outputs, so the extra slots
are now padded with skip(), leaving those components untouched
rather than clearing them.

The yield itself has to stay. Yielding nothing sends None for the
chatbot, and the frontend applies that as an empty chat: the
substitution that keeps a normal generator's final payload intact
never runs for a generator that yields nothing.
@hysts hysts self-assigned this Jul 27, 2026
@hysts
hysts requested a review from Copilot July 27, 2026 08:07
@gradio-pr-bot

gradio-pr-bot commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

🪼 branch checks and previews

Name Status URL
Spaces ready! Spaces preview
Website ready! Website preview
🦄 Changes detected! Details

Install Gradio from this PR

pip install https://huggingface.co/buckets/gradio/pypi-previews/resolve/5ab7552381aea5353f93918c3a4dfb535b6374bc/gradio-6.20.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@5ab7552381aea5353f93918c3a4dfb535b6374bc#subdirectory=client/python"

Import Gradio JS Client from this PR via CDN

import { Client } from "https://huggingface.co/buckets/gradio/npm-previews/resolve/5ab7552381aea5353f93918c3a4dfb535b6374bc/browser.js";

@gradio-pr-bot

Copy link
Copy Markdown
Collaborator

🦄 change detected

This Pull Request includes changes to the following packages.

Package Version
gradio patch

  • Fix crash when a streaming gr.ChatInterface function yields nothing

Something isn't right?

  • Maintainers can change the version label to modify the version bump.
  • If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a crash in Gradio’s backend ChatInterface streaming path when a (sync or async) generator returns without yielding any items, ensuring the API returns a valid “no bot response” payload (and preserves chat history) instead of raising RuntimeError: async generator raised StopAsyncIteration.

Changes:

  • Update _stream_fn to catch StopAsyncIteration when the first streamed item is missing, and yield a correctly shaped output tuple (including gr.skip() for any additional_outputs).
  • Import and use skip() in chat_interface.py to keep additional outputs unchanged when the generator yields nothing.
  • Add API tests covering empty sync generator, empty async generator, and empty generator with additional_outputs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
gradio/chat_interface.py Catches StopAsyncIteration during initial stream fetch and yields the correct number of outputs (including skip() for extra outputs).
test/test_chat_interface.py Adds regression tests ensuring empty streaming generators don’t crash, preserve history, and behave correctly with additional_outputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

self.additional_outputs is always a list, so splatting a per-output
skip() covers the no-additional-outputs case too and the branch goes
away.

Build the list with a comprehension rather than `[skip()] * n`: the
latter repeats one dict, and postprocess_data passes prediction dicts
straight into delete_none and postprocess_update_dict, both of which
mutate in place. Harmless while skip() has no "value" key, but there
is no reason to share the dict.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@hysts
hysts marked this pull request as ready for review July 27, 2026 09:19
@hysts
hysts requested review from abidlabs and dawoodkhan82 July 27, 2026 09:19

@abidlabs abidlabs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works perfectly!

@hysts

hysts commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review @abidlabs ! Merging.

@hysts
hysts merged commit b844e74 into main Jul 29, 2026
26 checks passed
@hysts
hysts deleted the fix/chatinterface-empty-generator-crash branch July 29, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Empty generators in streaming gr.ChatInterface

4 participants