-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Cleaning up the Hosting native Android views page #13356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,24 +2,26 @@ | |
|
|
||
| Platform views in Flutter come with performance trade-offs. | ||
|
|
||
| For example, in a typical Flutter app, the Flutter UI is composed | ||
| on a dedicated raster thread. This allows Flutter apps to be fast, | ||
| as the main platform thread is rarely blocked. | ||
|
|
||
| While a platform view is rendered with hybrid composition, | ||
| the Flutter UI is composed from the platform thread, | ||
| which competes with other tasks like handling OS or plugin messages. | ||
|
|
||
| Prior to Android 10, hybrid composition copied each Flutter frame | ||
| out of the graphic memory into main memory, and then copied it back | ||
| to a GPU texture. As this copy happens per frame, the performance of | ||
| the entire Flutter UI might be impacted. In Android 10 or above, the | ||
| graphics memory is copied only once. | ||
|
|
||
| Virtual display, on the other hand, | ||
| makes each pixel of the native view | ||
| flow through additional intermediate graphic buffers, | ||
| which cost graphic memory and drawing performance. | ||
| In a typical Flutter app, the Flutter UI is composed on a dedicated raster thread, | ||
| while platform code runs on the UI/platform thread. | ||
| This separation keeps Flutter rendering fast and fluid. | ||
|
|
||
| However, when a platform view is rendered on Android using **Hybrid | ||
| Composition**, Flutter merges the raster and UI threads into a single thread to | ||
| ensure correct synchronization between the native Android views and the Flutter canvas. | ||
| Because of this thread merging, rendering complex Flutter widgets | ||
| alongside a platform view can compete with OS messages and plugin interactions, | ||
| potentially causing lower application FPS and frame drops. | ||
|
|
||
| **Hybrid Composition++ (HCPP)** minimizes this overhead by using native | ||
| transaction synchronization on supported devices (Android API 34+ with Vulkan), | ||
| allowing superior performance without the heavy costs of original hybrid | ||
| composition. | ||
|
|
||
| **Virtual display** (used by the texture layer mode), on the other hand, | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. VD and TLHC are distinct modes, I don't think we should conflate them here.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have updated this section, please check it. Also, I don't know why the Performance section was created as an include, because it wasn't included anywhere else in the site. I've fixed that, too. |
||
| avoids thread merging but forces every pixel of the native Android view to flow | ||
| through additional intermediate graphic buffers, which increases graphic memory | ||
| usage and can cause jank during high-frequency updates like fast scrolling. | ||
|
|
||
| For complex cases, there are some techniques that | ||
| can be used to mitigate these issues. | ||
|
|
@@ -31,7 +33,7 @@ platform view is rendered, | |
| then consider taking a screenshot of the | ||
| native view and rendering it as a texture. | ||
|
|
||
| For more information, see: | ||
| For more information, visit: | ||
|
|
||
| * [`TextureLayer`][] | ||
| * [`TextureRegistry`][] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,18 +31,25 @@ visit [Hosting native macOS views][]. | |
| Platform Views on Android have several implementations. | ||
| They come with tradeoffs both in terms of performance and fidelity. | ||
|
|
||
| ## Hybrid composition {: #hybrid-composition } | ||
| ### Choosing an implementation | ||
|
|
||
| The following matrix summarizes the different implementations and their trade-offs: | ||
|
|
||
| | Mode | Benefits | Considerations | Enabler | | ||
| | :--- | :--- | :--- | :--- | | ||
| | **Texture layer** | • Best Flutter performance<br>• Full widget transforms work | • Janky during quick scrolling<br>• SurfaceViews break | Default behavior or `AndroidView` | | ||
| | **Hybrid Composition** | • Full native fidelity<br>• Correct accessibility and SurfaceViews | • Causes thread merging, which degrades Flutter FPS | `PlatformViewLink` with `AndroidViewSurface` | | ||
| | **Hybrid Composition++ (HCPP)** (Experimental) | • Full fidelity and performance<br>• Solves original sync overhead | • Requires Android API 34+ and Vulkan support | `<meta-data>` and `PlatformViewLink` in `AndroidManifest.xml` | | ||
|
sfshaza2 marked this conversation as resolved.
Outdated
|
||
|
|
||
| {:.table .table-striped} | ||
|
|
||
|
|
||
| ## Hybrid Composition {: #hybrid-composition } | ||
|
|
||
| Platform Views are rendered as they are normally. | ||
| Flutter content is rendered into a texture. | ||
| SurfaceFlinger composes the Flutter content and the platform views. | ||
|
|
||
| * `+` best performance and fidelity of Android views. | ||
| * `-` Flutter performance suffers. | ||
| * `-` FPS of application will be lower. | ||
| * `-` Certain transformations that can be applied to Flutter widgets | ||
| won't work when applied to platform views. | ||
|
|
||
| ## Hybrid Composition++ (HCPP) {: #hcpp } | ||
|
|
||
| :::note | ||
|
|
@@ -58,7 +65,7 @@ It is currently available as an opt-in feature. | |
|
|
||
| * **Android API 34 or later**: Required for native transaction | ||
| synchronization capabilities. | ||
| * **Vulkan Rendering**: The device must be capable of rendering with Vulkan. | ||
| * **Vulkan rendering**: The device must be capable of rendering with Vulkan. | ||
|
|
||
| If these requirements are not met on the end-user device, | ||
| Flutter will automatically fall back to the existing platform view strategy | ||
|
|
@@ -72,7 +79,7 @@ it's enabled through configuration rather than standard Dart initialization meth | |
|
|
||
| You can enable HCPP using one of the following methods: | ||
|
|
||
| 1. **Command Line Flag (Run/Test)**: | ||
| 1. **Command line flag (run/test)**: | ||
| Pass the `--enable-hcpp` flag to your `flutter run` or `flutter test` command: | ||
|
|
||
| ```bash | ||
|
|
@@ -81,7 +88,7 @@ You can enable HCPP using one of the following methods: | |
|
|
||
| :::note | ||
| This flag is intended for local execution and testing. | ||
| It **can't** be passed to the `flutter build` commands. | ||
| **It can't be passed to the `flutter build` commands.** | ||
| For release builds, use the manifest configuration | ||
| as shown in the next step. | ||
| ::: | ||
|
|
@@ -98,34 +105,38 @@ You can enable HCPP using one of the following methods: | |
|
|
||
| ### Limitations and known issues | ||
|
|
||
| * **Complex Overlay Stacking**: | ||
| * **Complex overlay stacking**: | ||
| Transparent platform views won't display correctly | ||
| in layout stacks structured as: | ||
| Flutter canvas -> Platform View -> Overlay -> Transparent Platform View, | ||
| **Flutter canvas -> Platform View -> Overlay -> Transparent Platform View**, | ||
| when all four of these layers intersect. | ||
|
|
||
| To create a platform view on Android, use the following steps. | ||
|
|
||
| ## Texture layer { #texturelayerhybridcomposition } | ||
| ## Texture layer {: #texture-layer } | ||
|
|
||
| Platform Views are rendered into a texture. | ||
| Flutter draws the platform views (using the texture). | ||
| Flutter content is rendered directly into a Surface. | ||
|
|
||
| * `+` good performance for Android Views | ||
| * `+` best performance for Flutter rendering. | ||
| * `+` all transformations work correctly. | ||
| * `-` quick scrolling (such as a web view) will be janky | ||
| * `-` SurfaceViews are problematic in this mode and will be moved into a virtual | ||
| display (breaking a11y) | ||
| * `-` Text magnifier will break unless Flutter is rendered into a TextureView. | ||
| This approach provides: | ||
|
|
||
| * good performance for Android Views | ||
| * best performance for Flutter rendering | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. comment about best performance also applies here |
||
| * all transformations work correctly | ||
|
|
||
| However, this approach might cause: | ||
|
|
||
| * jankiness on quick scrolling (such as a web view) | ||
| * broken accessibility for `SurfaceView`s | ||
| * broken text magnification unless Flutter is rendered | ||
| into a `TextureView` | ||
|
|
||
| ## On the Dart side | ||
|
|
||
| On the Dart side, create a `Widget` | ||
| and add one of the following build implementations. | ||
| To create a platform view on Android, use the following steps. | ||
| First, on the Dart side, create a `Widget` and add one of the | ||
| following build implementations depending on your chosen strategy. | ||
|
|
||
| ### Hybrid composition | ||
| ### Hybrid Composition | ||
|
|
||
| In your Dart file, | ||
| for example `native_view_example.dart`, | ||
|
|
@@ -142,7 +153,7 @@ use the following instructions: | |
| import 'package:flutter/services.dart'; | ||
| ``` | ||
|
|
||
| 2. Implement a `build()` method: | ||
| 2. Implement a `build` method: | ||
|
|
||
| <?code-excerpt "lib/native_view_example_1.dart (hybrid-composition)"?> | ||
| ```dart | ||
|
|
@@ -179,7 +190,7 @@ use the following instructions: | |
| } | ||
| ``` | ||
|
|
||
| For more information, visit the API docs for: | ||
| For more information, visit the following API docs: | ||
|
|
||
| * [`PlatformViewLink`][] | ||
| * [`AndroidViewSurface`][] | ||
|
|
@@ -203,7 +214,7 @@ use the following instructions: | |
| import 'package:flutter/services.dart'; | ||
| ``` | ||
|
|
||
| 2. Implement a `build()` method: | ||
| 2. Implement a `build` method: | ||
|
|
||
| <?code-excerpt "lib/native_view_example_2.dart (virtual-display)"?> | ||
| ```dart | ||
|
|
@@ -222,9 +233,7 @@ use the following instructions: | |
| } | ||
| ``` | ||
|
|
||
| For more information, visit the API docs for: | ||
|
|
||
| * [`AndroidView`][] | ||
| For more information, visit the [`AndroidView`][] API page. | ||
|
|
||
| [`AndroidView`]: {{site.api}}/flutter/widgets/AndroidView-class.html | ||
|
|
||
|
|
@@ -485,19 +494,19 @@ android { | |
| ### Manual view invalidation | ||
|
|
||
| Certain Android Views don't invalidate themselves when their content changes. | ||
| Some example views include `SurfaceView` and `SurfaceTexture`. | ||
| When your Platform View includes these views, you are required to | ||
| manually invalidate the view after they have been drawn to | ||
| (or more specifically: after the swap chain is flipped). | ||
| Manual view invalidation is done by calling `invalidate` on the View | ||
| Some examples include `SurfaceView` and `SurfaceTexture`. | ||
| When your Platform View includes these views, you must | ||
| manually invalidate the view after it's been drawn to | ||
| (or, more specifically, after the swap chain is flipped). | ||
| Manual view invalidation is done by calling `invalidate` on the view | ||
| or one of its parent views. | ||
|
|
||
| [`AndroidViewSurface`]: {{site.api}}/flutter/widgets/AndroidViewSurface-class.html | ||
|
|
||
| ### Issues | ||
|
|
||
| [Existing Platform View issues][] | ||
| Check out the [existing Platform View issues][] on GitHub. | ||
|
|
||
| {% render "docs/platform-view-perf.md", site: site %} | ||
|
|
||
| [Existing Platform View issues]: {{site.github}}/flutter/flutter/issues?q=is%3Aopen+is%3Aissue+label%3A%22a%3A+platform-views | ||
| [existing Platform View issues]: {{site.github}}/flutter/flutter/issues?q=is%3Aopen+is%3Aissue+label%3A%22a%3A+platform-views | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to be changing the information that was presented before. Was the prior info (that there was a change around android 10) wrong?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hope I've fixed this. Please review.