sclang: Don't use LID on Android

[rene-descartes2021]

Purpose and Motivation

I was able to build SuperCollider on Android/Termux. It works great on my Android :)

Ultimately I'll get Overtone working with this build, usable within Emacs on Android/Termux.

This PR makes SuperCollider a little easier to build on Android/Termux. Did not attempt to build SuperNova- but I presume that will work.

Details

On Termux Jack2 became available as of June 2024, uses OpenSLES backend: termux/termux-packages#20544
Jack2 launchable with: jackd -d opensles
Possibly launchable when Termux is launched via termux-services, looking at that later.

No proot or pulseaudio involved as mentioned here:
https://scsynth.org/t/my-attempt-to-build-supercollider-on-termux-android/1297

My build script:

rm -r build
mkdir build
cd build
export CFLAGS="-DBOOST_NO_CXX98_FUNCTION_BASE"
export CXXFLAGS="$CFLAGS"
cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DNATIVE=ON -DCMAKE_INSTALL_PREFIX=~/.local/share/SuperCollider -DAUDIOAPI=jack -DSC_QT=OFF -DSC_HIDAPI=OFF -DNO_X11=ON -DNO_AVAHI=ON -DSC_ABLETON_LINK=OFF

Types of changes

• New feature
  Easier to build on Android/Termux.

libusb LID symbols not available on Android/Termux. This PR removes the LID stuff so it will be easier to compile.

"To use HIDAPI on Android, it is only possible with HIDAPI-libusb, and HIDAPI is limited as much as libusb is limited on this matter"
See: https://github.com/libusb/libusb/wiki/Android

To-do list

• Code is tested
• All tests are passing
• Updated documentation
• This PR is ready for review

Additional build problems encountered this PR does not address and I won't fix

boost problem:

• https://stackoverflow.com/a/77503976
• fixed with -DBOOST_NO_CXX98_FUNCTION_BASE

external_libraries/boost/boost/thread/future.hpp L4672:

• rename x.that to x.that_
• https://bugs.gentoo.org/937221
• fixed in newer boost:
• https://bugs.gentoo.org/945916

lang/LangSource/Bison/lang11d_tab.cpp:

• commented out bcopy
• if lang11d_tab gets autogenerated by bison maybe the bcopy can not be generated if ANDROID

Builds now scsynth but had runtime error due to more boost problems, /tmp assumption:

• error: World_New: “No such file or directory”

Need to patch in external_libraries/boost/boost/interprocess/detail/

• os_file_functions.hpp and shared_dir_helpers.hpp:

// header #include <boost/filesystem/operations.hpp>
system::error_code e;
filesystem::path p = filesystem::detail::temp_directory_path(&e);
//with single use of p.string() following in each file where appropriate replacement based on the context, e.g. return p.string();

Much of the above besides the bison issue can possibly be fixed by updating boost or configuring CMake to not use the boost distributed within the repo- to source a newer version of boost using whatever CMake variables.

I read some of the boost::filesystem stuff was moved into std::filesystem so if maybe new stdlib standard used that problem may be fixed assuming they fixed the use of /tmp assumption in the interprocess code (e.g. look at $TMPDIR or whatever first).

[capital-G]

Hey, much thanks for this nice write-up - awesome to see that SC also runs on android!

• rename x.that to x.that_

Are you sure you are on a development branch? I fixed this 2 months ago on develop via edfac5e and we are on a rather recent boost version. Main/3.13 has an outdated boost version though.

[rene-descartes2021]

Hey, much thanks for this nice write-up - awesome to see that SC also runs on android!

• rename x.that to x.that_

Are you sure you are on a development branch? I fixed this 2 months ago on develop via edfac5e and we are on a rather recent boost version. Main/3.13 has an outdated boost version though.

Nope I wasn't sorry got mixed up. First I tried the latest release source tarball and edited that till it built, then thinking to make a PR I just moved the LID changes onto a commit on the develop branch. I've been tinkering with building the develop branch today, I'll write again if I find the same boost problems.

One more thing I noticed, 'Android' should be added to the list here so that SUPERNOVA is built by default:

supercollider/CMakeLists.txt

Lines 17 to 18 in 5c8b58d

	if(CMAKE_SYSTEM_NAME MATCHES "Linux|FreeBSD|DragonFly|OpenBSD|NetBSD")
	set(LINUX_OR_BSD 1)

I was also writing a PR for termux-packages so that a supercollider release is installable with apt-get install supercollider within the Termux app. Seems like an ok thing to do, meets their criteria as long as it builds. I read some more and the bcopy thing appears to be a Termux specific concern, not Android specific, so that's something I was putting in that PR, not something to worry about in this supercollider repo.

[rene-descartes2021]

Ok you're right most of the boost problems were fixed as of the newer packaged boost v1.86. The /tmp boost interprocess problem still exists though, with the scsynth server error:
Exception in World_New: No such file or directory.

I will try v1.87 boost as that's what is at present the Termux system's boost, by building with -DSYSTEM_BOOST=ON. Or I guess browse around the boost source repository & issues.

Ran into two problems building SUPERNOVA:
https://gist.github.com/rene-descartes2021/04b706d35c7f0cab5dcd83af46c6a563
Dunno what is causing those yet, might be a Termux specific thing again.

With respect to the PR there is a section on Termux in the below link, which mentions the SuperCollider software might be able to adapt to one of the Termux workarounds to navigate/use usb devices, so, just pointing out that is an option. I have no idea what the use case of SuperCollider using usb devices is though:
https://github.com/libusb/libusb/wiki/Android

EDIT: Maybe the above libusb Termux workaround is only necessary if the user has a device that isn't exposed/compatible through OpenSLES via Jack2. Dunno if there is a use case there.

[rene-descartes2021]

So I looked at the most recent boost at boostorg/interprocess, looks like it's not fixed there.

Turns out it's only shared_dir_helpers.hpp in boost that needs the fix- the other problem file I mentioned earlier isn't used by SuperCollider. I tested each fix individually. Had to rewrite it a bit for the newer boost, to use std::filesystem not boost::filesystem.

Here is a patch for the earlier mentioned issues, Android in CMakeLists.txt and external_libraries/boost/boost/interprocess/detail/shared_dir_helpers.hpp:
https://gist.github.com/rene-descartes2021/ea2e5c8976c75589c8a2a3821322a1e6

I guess that patch isn't on the topic of this PR so I won't add it to this PR.

[rene-descartes2021]

Well here is a commit ready for termux/termux-packages for making a SuperCollider release readily available in the Termux app on Android via apt-get install supercollider:
rene-descartes2021/termux-packages@05c9b95

That's all it looks like, a build.sh recipe based on the 3.14.0-rc1 release and all the patches to get it to build for Termux, like the bcopy thing. As the other patches get added to supercollider I can remove them from that termux/termux-packages package recipe. Or anyone else who wants to maintain it as new SuperCollider releases are made.

I'll suppose I'll wait for the 3.14.0 release though.

[dyfer]

@rene-descartes2021 thanks for your work on this, being able to install SC on Android is amazing!

I have no idea what the use case of SuperCollider using usb devices is though

IIUC sclang has a couple ways of communicating with devices like joysticks, mice, touchpads etc. One is Linux specific (the one addressed here - LID), the other is using hidapi library and provides a cross-platform interface for such devices. (The latter library also needs updating but that's not relevant here). This has nothing to do with audio devices and jack, btw.

Ran into two problems building SUPERNOVA:
https://gist.github.com/rene-descartes2021/04b706d35c7f0cab5dcd83af46c6a563
Dunno what is causing those yet, might be a Termux specific thing again.

This looks like an issue in nova-tt. Maybe a solution is to add a check for Android here?

Regarding this PR, one thing I wonder about is whether LID symbols are not available on any Android, or just in Termux? If it's the latter, maybe a better option would be to add a CMake option to disable LID optionally, and the user would be responsible for setting that when building?

---

For the termux build, since you need to provide a patch anyway, I think it makes sense to use the 3.14 release and patch everything there, including the LID stuff.

---

As for the boost stuff, it looks like the /tmp fix could be also submitted upstream, if you have bandwidth for that.

---

Also, as I wanted to check whether you submitted a PR to termux for final 3.14.0, I found your old PR for rc1 and wanted to clarify something:

2)There exists some sort of plugin to SuperCollider which requires the same version of boost to be used. I'm not really sure if this is a reason I haven't really looked into it, but I think it's the reason they bundle it in the first place.

This is not the reason we bundle boost. The issue with the plugin (namely FluCoMa) is a relatively recent development.

Boost was originally bundled because 1) back in the day boost needed a number of patches to build/work with SC and 2) it was difficult to install on some platforms. At some point SC was made compatible with the latest boost and the ability to build with system boost was added, but we still use bundled boost by default on macOS and Windows. On Linux that depends on the packagers.

The issue with the plugin compatibility only pertain to a single 3rd-party plugin suite and can be disregarded for the purposes of bringing SC to Android.

---

Again, thank you for your work on getting SC to run on Android! I'm looking forward to trying it out!

[rene-descartes2021]

@rene-descartes2021 thanks for your work on this, being able to install SC on Android is amazing!

I have no idea what the use case of SuperCollider using usb devices is though

IIUC sclang has a couple ways of communicating with devices like joysticks, mice, touchpads etc. One is Linux specific (the one addressed here - LID), the other is using hidapi library and provides a cross-platform interface for such devices. (The latter library also needs updating but that's not relevant here). This has nothing to do with audio devices and jack, btw.

Ran into two problems building SUPERNOVA:
https://gist.github.com/rene-descartes2021/04b706d35c7f0cab5dcd83af46c6a563
Dunno what is causing those yet, might be a Termux specific thing again.

This looks like an issue in nova-tt. Maybe a solution is to add a check for Android here?

Thank you for the orientation, a lot of this stuff beyond my knowledge.

Regarding this PR, one thing I wonder about is whether LID symbols are not available on any Android, or just in Termux? If it's the latter, maybe a better option would be to add a CMake option to disable LID optionally, and the user would be responsible for setting that when building?

I have no clue just got it to build really. Dunno who to ask about that either. My assumption would be not available on any Android. On Android all the native libs are in /system/libs/ and there is a file /system/etc/public.libraries.txt containing on my device:

# See https://android.googlesource.com/platform/ndk/+/master/docs/PlatformApis.md
libandroid.so
libaaudio.so
libamidi.so
libbinder_ndk.so
libc.so
libcamera2ndk.so
libdl.so
libEGL.so
libGLESv1_CM.so
libGLESv2.so
libGLESv3.so
libicu.so
libicui18n.so
libicuuc.so
libjnigraphics.so
liblog.so
libmediandk.so
libm.so
libnativehelper.so
libnativewindow.so
libneuralnetworks.so nopreload
libOpenMAXAL.so
libOpenSLES.so
libRS.so
libstdc++.so
libsync.so
libvulkan.so
libwebviewchromium_plat_support.so
libz.so

There are libraries not in that list in /system/libs/ but I take those to be not guaranteed in future. Termux adds some libraries on top of that in order to get stuff to work. For hardware access on Android the trend is to only allow access through their JNI API, guarded by permissions/security not through native C API calls. But, well, beyond my knowledge really.

For the termux build, since you need to provide a patch anyway, I think it makes sense to use the 3.14 release and patch everything there, including the LID stuff.

Oh, alrighty. So close this PR and keep the LID patch in the set of Termux supercollider package patches.

As for the boost stuff, it looks like the /tmp fix could be also submitted upstream, if you have bandwidth for that.

The Tom person here on my termux-packages PR said they'd look into it.

Also, as I wanted to check whether you submitted a PR to termux for final 3.14.0, I found your old PR for rc1 and wanted to clarify something:

Oh right I did update the PR from rc1 to 3.14.0 but didn't have permission to reopen the PR, I just pinged a maintainer there for attention to reopen it.

2)There exists some sort of plugin to SuperCollider which requires the same version of boost to be used. I'm not really sure if this is a reason I haven't really looked into it, but I think it's the reason they bundle it in the first place.

This is not the reason we bundle boost. The issue with the plugin (namely FluCoMa) is a relatively recent development.

Boost was originally bundled because 1) back in the day boost needed a number of patches to build/work with SC and 2) it was difficult to install on some platforms. At some point SC was made compatible with the latest boost and the ability to build with system boost was added, but we still use bundled boost by default on macOS and Windows. On Linux that depends on the packagers.

The issue with the plugin compatibility only pertain to a single 3rd-party plugin suite and can be disregarded for the purposes of bringing SC to Android.

Got it. Was just speculating there. I think Tom will handle it as I wrote above and the Termux package adjusted after that.

Again, thank you for your work on getting SC to run on Android! I'm looking forward to trying it out!

No problem! For install via Termux maybe a web page guide could be made containing essentially:

1. install Termux apk from GitHub
2. agt-get install supercollider emacs.
3. Probably other steps related to jack2 server.

It's possible to install a Termux:X11 app in order to use interfaces like the supercollider scide IDE, but I didn't make the attempt to make a separate Termux package containing that IDE for that use case. It's a chore to set up and use (launch x server, window manager, X11 app, etc), unlike a native Android app. Emacs makes sense for me, with a Bluetooth keyboard, but I realize others might wish to choose differently if it were streamlined for them.

[rene-descartes2021]

New Supercollider Termux-package PR here. Maybe a week or so after that is merged it'll be available for install through the package manager on Termux.

[rene-descartes2021]

Ran into two problems building SUPERNOVA:
https://gist.github.com/rene-descartes2021/04b706d35c7f0cab5dcd83af46c6a563
Dunno what is causing those yet, might be a Termux specific thing again.

This looks like an issue in nova-tt. Maybe a solution is to add a check for Android here?

@dyfer I believe the #ifdef __linux__ is true on Android, that's why I added the && is_defined(LID) or whatever in this patch as android didn't have that LID preproc defined.

Dunno what the first CPU_SET error was about, guessing it had to do with pthread_setaffinity_np missing.

I browsed around for pthread_setaffinity_np and found it here in pthread.h (sorry couldn't find a handy web link, it should be in the zip):
https://dl.google.com/android/repository/android-ndk-r28c-linux.zip

Looks like from the comments and preproc guard that symbol is not available until API level 36, i.e. Android 16. I'm still on Android 12 and it makes sense to target the oldest devices possible, so that's a no-go for me to use that symbol. Maybe some substitute could be used, like the suggested sched_setaffinity, and I'm not really familiar with the context. I'll just go without SUPERNOVA for now? Seemed fine when I tested it without.