Add Android backend using Chaquopy/Briefcase#1944
Add Android backend using Chaquopy/Briefcase#1944timrid wants to merge 43 commits intohbldh:developfrom
Conversation
timrid
force-pushed
the
android-backend-using-chaquopy
branch
from
2ee3b1b to
03573aa
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #1944 +/- ##
===========================================
+ Coverage 51.26% 56.50% +5.23%
===========================================
Files 39 48 +9
Lines 3938 4697 +759
Branches 488 572 +84
===========================================
+ Hits 2019 2654 +635
- Misses 1791 1864 +73
- Partials 128 179 +51
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
You've been very busy! 🤯 A few quick high-level comments.
|
timrid
force-pushed
the
android-backend-using-chaquopy
branch
from
bdde922 to
161e410
Compare
Done in #1959 and #1958. Now that also #1934 is merged, I rebased on the new develop branch.
Yes, that was my intention. It is theoretically possible to integrate the
I think the
Done.
I created https://github.com/timrid/chaquopy-stubs to outsource most of the stubs, so we just need to depend on chaquopy-stubs-android. But some of the stubs are special ones that can not be autogenerated so easy. So they need to be kept in |
timrid
force-pushed
the
android-backend-using-chaquopy
branch
from
161e410 to
ef72121
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a new Android backend based on BeeWare/Briefcase + Chaquopy, alongside an Android testbed app and CI job to run integration tests on an emulator.
Changes:
- Added a new
BleakBackend.ANDROIDimplementation (bleak.backends.android) with typed dispatcher/callback plumbing, permissions handling, scanning, and client support. - Added an Android “testbed” Briefcase app plus host-side runner scripts to execute pytest on emulator/real device and collect coverage/JUnit artifacts.
- Updated docs, changelog, typing configs/stubs, and CI workflow to reflect and validate Android support.
Reviewed changes
Copilot reviewed 34 out of 42 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| typings/toga_android/app.pyi | Adds stub for Toga Android permission request API used by backend. |
| typings/toga_android/init.pyi | Package marker for Toga Android stubs. |
| typings/org/beeware/android/init.pyi | Adds stub for MainActivity.singletonThis access. |
| typings/org/beeware/init.pyi | Package marker for org.beeware stubs. |
| typings/org/init.pyi | Package marker for org stubs. |
| tests/integration/test_issue_1885.py | Skips a flaky/timeout-prone test on Android. |
| tests/integration/test_client_pairing.py | Adds Android-only pairing test behavior. |
| tests/integration/README.rst | Documents how Android integration tests are executed (real device/emulator). |
| testbed/tests/bleak_testbed.py | Implements the in-app pytest runner + coverage artifact generation. |
| testbed/src/bleak_testbed/app.py | Minimal Toga app for the testbed. |
| testbed/src/bleak_testbed/main.py | Entry point to run the testbed app. |
| testbed/src/bleak_testbed/init.py | Package marker for testbed app module. |
| testbed/run_android_tests_real_device.py | Host runner for real-device tests with serial↔TCP bridge + Briefcase run. |
| testbed/run_android_tests_emulator.py | Host runner for emulator tests (netsim forwarding + dialog automation). |
| testbed/README.md | Brief documentation for the testbed app. |
| testbed/pyproject.toml | Briefcase configuration for building/running the testbed on Android (and desktop). |
| pyrightconfig-android.json | Pyright strict config targeting Android platform. |
| pyproject.toml | Adds test-android dependency group and excludes testbed from type checking. |
| mypy-android.ini | Adds a mypy config targeting Android platform and typings. |
| docs/index.rst | Updates supported platforms list to include Android via Chaquopy/Briefcase. |
| docs/images/android-permission-request.png | Adds screenshot of Android permission dialog used in docs. |
| docs/conf.py | Mocks java/org imports for Sphinx autodoc builds. |
| docs/backends/index.rst | Splits Android docs into BeeWare backend vs Python-for-Android backend pages. |
| docs/backends/android-p4a.rst | Retitles and adds a warning about Python-for-Android backend stability. |
| docs/backends/android-beeware.rst | New documentation page for the BeeWare/Chaquopy Android backend. |
| CHANGELOG.rst | Notes addition of the new Android backend. |
| bleak/backends/scanner.py | Selects Android scanner backend when BleakBackend.ANDROID is active. |
| bleak/backends/client.py | Selects Android client backend when BleakBackend.ANDROID is active. |
| bleak/backends/android/utils.py | Adds Android utility helpers and global activity/context accessors. |
| bleak/backends/android/status.py | Defines Android scan/GATT status constants/enums. |
| bleak/backends/android/scanner.py | Implements Chaquopy-based Android scanner + excessive scan throttling. |
| bleak/backends/android/scanner_callback.py | Implements ScanCallback proxy and bridges to asyncio via dispatcher. |
| bleak/backends/android/permissions.py | Implements runtime permission checks/requests via Toga. |
| bleak/backends/android/dispatcher.py | Adds typed callback dispatcher to await Java callback results safely. |
| bleak/backends/android/client.py | Implements Chaquopy-based Android GATT client operations. |
| bleak/backends/android/client_callback.py | Implements BluetoothGattCallback proxy and dispatches results/errors. |
| bleak/backends/android/broadcast.py | Adds Chaquopy-adapted BroadcastReceiver helper. |
| bleak/backends/android/init.py | Package marker for Android backend module. |
| bleak/backends/init.py | Adds BleakBackend.ANDROID and selects it when sys.platform == "android". |
| .gitignore | Ignores junit.xml artifact. |
| .github/workflows/build_and_test.yml | Adds Android emulator integration test job + mypy Android type-check step. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This was copied from the P4A backend, but I dont think that it is required, because we are not using batched mode, because we use `setReportDelay(0)`.
… not raising it to other futures or raising it to the event loop.
…API level 33 and above
According to the docs `getBluetoothLeScanner()` will return None when Bluetooth is turned off. This also updates chaquopy-stubs-android, because the type hint was updated so that `getBluetoothLeScanner()` is marked that it can return None
…t, because it is only used in rare cases.
|
I've integrated Copilot's comments into the code and added a few points of my own. The CI is currently generating the error |
Before:
"User denied access to jarray('Ljava/lang/String;')(['android.permission.BLUETOOTH_SCAN', 'android.permission.BLUETOOTH_CONNECT'])"
After:
"User denied access to ['android.permission.BLUETOOTH_SCAN', 'android.permission.BLUETOOTH_CONNECT']"
|
I've fixed all new Copilot comments. The CI failure was fixed in |
…luetooth permission handling
dlech
left a comment
dlech
left a comment
There was a problem hiding this comment.
This is huge, so will probably come back with more comments later. Here are a few hours worth...
| @@ -0,0 +1,4 @@ | |||
| from bleak_testbed.app import main | |||
There was a problem hiding this comment.
These testbed files could use some comments/docstrings at the start of each file to explain why they are here.
There was a problem hiding this comment.
Actually the files in testbed/src/bleak_testbed are not needed, because we only use the --test mode of briefcase. So I removed these files. But there has to be at least a folder called testbed/src/bleak_testbed or briefcase will raise a error.
For the testbed/tests/bleak_testbed.py I added a comment.
| "**/.venv/**", | ||
| "**/kivy/**", | ||
| "**/recipes/**", | ||
| "**/testbed/**", |
There was a problem hiding this comment.
Should testbed be still be excluded here?
| producing an error. Therefore, Bleak automatically tracks the starting times of scans | ||
| and waits the necessary time before a new scan can be started. |
There was a problem hiding this comment.
Not sure how I feel about this feature. Seems like it could have unintended consequences. Like if someone connects BleakScanner directly to UI that lets user start/stop scanning, it seems like it would make the UI unresponsive if they pressed the start/stop button a bunch. It would say that it was starting scanning, but would actually be much delayed with no user feedback.
I would be more inclined to just raise an error and let the application deal with it.
| # Enable KVM permissions for the emulator | ||
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \ | ||
| | sudo tee /etc/udev/rules.d/99-kvm4all.rules | ||
| sudo udevadm control --reload-rules |
There was a problem hiding this comment.
Modern systems have file watchers that does this automatically, so I've never needed to manually reload rules.
Maybe there could be a race condition though in a script, so I guess it doesn't hurt.
There was a problem hiding this comment.
Or since this is CI, just get rid of the udev stuff and just run sudo chmod directly.
There was a problem hiding this comment.
I just followed the recommendations from Github: https://github.blog/changelog/2024-04-02-github-actions-hardware-accelerated-android-virtualization-now-available/
I added the link also to the source code, so that the source of these commands is clear.
There was a problem hiding this comment.
Just because they recommended it doesn't mean they understood it. 😉
| gatt.writeCharacteristic(characteristic.obj, payload, write_type) | ||
| == 0 |
There was a problem hiding this comment.
Ideally, we would be capturing the return value and including it in the raised exception to help the user find out what went wrong. At the very least, we should debug log it.
There was a problem hiding this comment.
I refactored the dispatcher a bit, so that it is simplified and it is also possible to raise an exception in the dispatch_func. So it is possible to raise an error message here that includes the return status.
| def _do_write_desc(): # pragma: no cover # (CI is running on API level below 33) | ||
| # On API level 33 (Android 13) and above writeDescriptor returns int (GATT_SUCCESS=0 on success). | ||
| # Convert to bool so dispatch_result_indicates_status=True works correctly. | ||
| return gatt.writeDescriptor(descriptor.obj, payload) == 0 |
There was a problem hiding this comment.
Another case to capture the error.
| self._result_state_unthreadsafe, failure, callback_api, callback_result | ||
| ) | ||
|
|
||
| def _result_state_unthreadsafe( |
There was a problem hiding this comment.
| def _result_state_unthreadsafe( | |
| def _result_state( |
I think not-threadsafe can be generally assumed.
There was a problem hiding this comment.
I changed the method name.
| logger.debug(f"onCharacteristicWrite {status=}") | ||
| handle = characteristic.getInstanceId() | ||
| self.dispatcher.result_state_threadsafe( | ||
| BleakGATTProtocolError(int(status)) if status != GATT_SUCCESS else None, |
There was a problem hiding this comment.
| BleakGATTProtocolError(int(status)) if status != GATT_SUCCESS else None, | |
| BleakGATTProtocolError(status) if status != GATT_SUCCESS else None, |
Or type hint of status is wrong.
There was a problem hiding this comment.
The type hint is correct, so the explicit conversion to int is not necessary.
This is also documented in the chaquopy docs:
When a Python method is called from Java, the parameters and return value are converted as described in data types above.
timrid
force-pushed
the
android-backend-using-chaquopy
branch
from
92a7849 to
058c4e2
Compare
|
I accidentally synced my branch to the develop via the Github UI, which caused the PR to be closed... That was by accident. But now I am not sure how to reopen this PR. Sorry for the inconvenience... |
timrid
force-pushed
the
android-backend-using-chaquopy
branch
from
b54196b to
92a7849
Compare
|
I finally got it reopened. Sorry for the noise. |
…backend-using-chaquopy
I am not quite sure why this helps on Android, but with this change the test passes.
The dispatcher will return always the dispatch result. It is the responsibility of the caller to use the result or not. Additionaly the return of the dispatch_func is not checked by the dispatcher, because the return type of the dispatch_func ist not always a bool. It is not the responsitility of the dispatch_func to raise an error if the dispatch has not started correctly. This enables a more explicit exception message. Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
This PR adds Android support for use with briefcase and chaquopy as a separate backend.
Fixes #917
Backend changes
The existing python-for-android backend served as the foundation for the new backend, but several modifications were made. Some of them are:
ExcessiveUsageCheckerbecause Android enforces a limit of 5 start/stop scan cycles within 30 seconds. This was necessary to allow the integration tests to run, as they would otherwise exceed this limitBroadcastReceiver, which is part of python-for-android but not of chaquopyI also noticed your comment that you would prefer permission handling to be explicit rather than implicit at scan start. I deliberately decided against this, because CoreBluetooth also handles permissions implicitly when starting a scan, and provides no way to request permissions independently without initiating a scan. To keep the backends as consistent as possible, I prefer the same approach on Android.
The old python-for-android backend has been left untouched for reference. Removing it would be better handled in a separate PR.
Type hints
For the type hints of the Android API, I created the tool chaquopy-stubgen and created chaquopy-stubs-android. This enables type checking with mypy, and in the future with pyright once microsoft/pyright#11221 is released.
Integration tests
To run the integration tests, a briefcase test bed example has been added. This builds a test app that can be run on a real Android device or the Android emulator.
In CI, the integration tests run on the Android emulator, which is started automatically by briefcase. To simulate Bluetooth devices, Bumble is connected to the virtual Bluetooth controller provided by the Android emulator (see Android Emulator transport). Since this virtual Bluetooth controller is normally only accessible from the host, the port is forwarded into the Android emulator via adb. Additionally, permission and pairing dialogs in the Android emulator must be confirmed — this is automated in CI using
uiautomator2. All of this is encapsulated in therun_android_tests_emulator.pyscript and can be run withuv run --python 3.13 testbed/run_android_tests_emulator.py.To run the integration tests on a real Android device, use the
run_android_tests_real_device.pyscript. This requires an nRF52840 dongle with HCI-UART firmware and automatically forwards the UART to a TCP server that is tunneled to the device via adb. I tested this on a Samsung S23 running Android 13 (API level 33).Supported Python versions
The target is Python 3.13 and later, as this is the first version in which Android is officially supported via PEP 738. This is also the first version where
sys.platformreturnsandroid, which simplifies backend detection and makes conditional dependencies possible.Python 3.14 cannot yet be included in CI because some binary dependencies of Bumble are not yet available for Python 3.14. I am still working on this.
Miscellaneous