Fix: Fallback to CPU to resolve native engine creation crash (#557)#878
Open
Chethan-4255 wants to merge 393 commits into
Open
Fix: Fallback to CPU to resolve native engine creation crash (#557)#878Chethan-4255 wants to merge 393 commits into
Chethan-4255 wants to merge 393 commits into
Conversation
…, T5, T6, T7 PiperOrigin-RevId: 807892326
PiperOrigin-RevId: 807893513
Special thanks to lprazdnik for the code contribution! PiperOrigin-RevId: 808765830
… description PiperOrigin-RevId: 810058684
PiperOrigin-RevId: 810518301
PiperOrigin-RevId: 811965499
Corrects the file extension from ".litermlm" to ".litertlm" in the Core Features list.
docs: Fix typo in .litertlm file extension
PiperOrigin-RevId: 815959499
Note: the cancellation exception will be used in an upcoming release of the litertlm API. PiperOrigin-RevId: 817754886
PiperOrigin-RevId: 819451826
PiperOrigin-RevId: 820431462
PiperOrigin-RevId: 822768793
Temporary disable benchmark in Gallery. We will reenable it once a new Maven package is released. PiperOrigin-RevId: 823574437
ExperimentalFlags is newly added to `0.0.0-alpha05`. PiperOrigin-RevId: 823610298
PiperOrigin-RevId: 824676152
PiperOrigin-RevId: 831537495
- Long message cannot be scrolled manually. - Copying message text can only copy partial text. Also, show error messages caught from the engine better. PiperOrigin-RevId: 831690400
…is no longer the default empty model. PiperOrigin-RevId: 832399534
…ages PiperOrigin-RevId: 832463657
…k's model list. PiperOrigin-RevId: 834335411
…owing up on launch (observed on certain vivo phones) PiperOrigin-RevId: 834838597
PiperOrigin-RevId: 836817743
…bump-to-v1.0.14 Update the allowlist for v1.0.14
PiperOrigin-RevId: 914466042
PiperOrigin-RevId: 914530896
PiperOrigin-RevId: 914549933
PiperOrigin-RevId: 914978962
PiperOrigin-RevId: 914988398
…ption opens a link to GitHub discussions for exploring community-contributed skills. PiperOrigin-RevId: 915028065
…ification skill. PiperOrigin-RevId: 915044738
PiperOrigin-RevId: 915130429
PiperOrigin-RevId: 915136269
…ew try-out chip for "Schedule Reminder". Several skills ("calculate-hash", "kitchen-adventure", "text-spinner", "send-email") are now disabled by default. The corresponding try-out chips for these have been removed.
PiperOrigin-RevId: 915151168
PiperOrigin-RevId: 915507718
Integrates the **Model Context Protocol (MCP)** into the AI Edge Gallery Android app, specifically within the Agent Chat task. It allows the AI assistant to use tools provided by remote MCP servers.
### Key Changes:
* **MCP Integration**:
* Introduces `McpManagerViewModel` to handle server connections, tool discovery, and state management.
* Updates `AgentTools` with a new `runMcpTool` that communicates with MCP servers.
* **AI Agent Updates**:
* Modifies the `AgentChatTask` system prompt to include a routing logic between "Skills" (instruction-based) and "MCP Tools" (API-based).
* Injects enabled MCP tool definitions (name, description, schema) directly into the system prompt.
* **User Interface**:
* **Management UI**: Adds `McpManagerBottomSheet` and `McpToolManagerBottomSheet` for users to add (via URL), toggle, or delete MCP servers and individual tools.
* **Configuration**: Implements `AddMcpServerFromUrlDialog` with support for custom request headers for authorization.
* **Chat Integration**: Updates the chat input area to display an MCP server toggle and enabled skill/mcp count.
* **Persistence**:
* Defines a new `mcp.proto` and uses **Proto DataStore** (`McpServersSerializer`) to persist configured servers and their enablement states.
* Extends `UserData` proto to store MCP authentication data (e.g., API keys in headers).
* **General Improvements**:
* Refactors `LlmChatViewModel` and `ModelManagerViewModel` to use the correct way to update ui state.
* Enhances common UI components like `SmallFilledTonalButton` and `ClickableLink` for better reusability in the new MCP screens.
* Update the Config Dialog to show min/max slider values on labels so sliders can align better.
PiperOrigin-RevId: 915539581
PiperOrigin-RevId: 915693518
…any tool call triggered by the agent prompts the user for approval unless the tool has been explicitly marked as "Always allow." ### Key Changes: * **Permission Dialog:** Adds `McpToolCallPermissionDialog` which displays the tool name and its arguments (formatted as JSON) when a tool is about to be called. It offers three options: "Always allow," "Allow once," and "Don't allow." * **Persistence:** Updates the `mcp.proto` and `McpManagerViewModel` to store the "always allow" preference for each tool. * **Management UI:** Enhances the MCP tool management bottom sheet to show which tools have "always allow" enabled and provides a way for users to revoke this permission. * **Action Handling:** Integrates a new `AskMcpToolCallPermissionAction` into the `AgentTools` execution flow. This suspends tool execution until the user provides a response via the UI. * **Code Cleanup:** Refactors `McpManagerViewModel` to use a centralized `persistServers` helper method, reducing code duplication when updating the DataStore. PiperOrigin-RevId: 915751277
PiperOrigin-RevId: 916210621
PiperOrigin-RevId: 916210820
PiperOrigin-RevId: 916336529
PiperOrigin-RevId: 917305569
PiperOrigin-RevId: 917406504
PiperOrigin-RevId: 917458688
PiperOrigin-RevId: 917488696
…formatted floating point numbers in non-US locales (e.g. numbers like "8186,03") by passing raw value to `StatRow`. PiperOrigin-RevId: 917488806
Add doc for mcp
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
Addresses #557. Resolves a critical native engine initialization crash (
INTERNAL: ERROR ... llm_litert_compiled_model_executor.cc) observed during model creation on high-end modern devices such as the Samsung Galaxy S26.Root Cause
The crash stems from the native C++ hardware delegation layer trying to compile specific tensor operations directly onto cutting-edge GPU/NPU architectures. Because mobile drivers on newly launched chipsets sometimes lack full operational compatibility matrices with predefined model weights graphs, the underlying compiler aborts setup, bubbling up an unhandled C++ runtime exception to the JNI initialization layer.
Solution
Introduced a highly stable application-layer fallback mapping. By enforcing
Backend.CPU()during instantiation routines, the engine routes vector processing through optimized multi-threaded SIMD instructions (via XNNPACK). This completely bypasses the unstable NPU/GPU drivers, preventing the fatal crash and ensuring deterministic model execution.Fixes #557