A new ASIO backend implementation

[LastExceed]

I have recently published azo, a re-implementation of ASIO's underlying COM API. As mentioned in #1296, the project's goal is for it to replace asio-sys in cpal's ASIO backend. I'm creating this issue for discussion and tracking purposes.

Motivation

asio-sys is bound to the limitations of Steinberg's ASIO SDK (outlined below), which are unsolvable from the outside.

ASIO SDK Limitations

Licensing Issues

The ASIO SDK has been under a propietary license for most of its life, with GPL being added as an alternative just a few months ago. Neither of them is a great fit for cpal:

• The propietary license (under which ASIO SDK was introduced into cpal) prohibits any form of redistribution, including even the header files, which requires developers to either
  • use a build script that automatically downloads the SDK from Steinberg's website - which is questionable both legally and security wise
  • manually provide the SDK themselves - which is inconvenient
• The GPL license is copyleft, which I doubt is 100% compatible with cpal's permissive license either (IANAL, correct me if I'm wrong)

Limited Capability

Most notably, ASIO SDK is incapable of initializing more than 1 driver at a time. This makes using multiple devices in parallel impossible. The spec explicitly calls this out as a limitation of the implementation, rather than a design choice.¹ There are also some other minor limitations, like not supporting strings which exceed certain fixed buffers in length, but these have essentially become part of the standard at this point.

azo

solves both of these problems. It is permissively licensed, and has multi-driver support without limitations. As the author I am of course heavily biased towards it, but I'm willing to do my due diligence in terms of field testing. For that, my first step was to implement cpal's custom host API, and in doing so, I encountered some problems with its design that I would like to discuss here.

cpal Limitations

No channel Mapping

cpal doesn't have a mechanism for channel mapping at the moment, so requesting n channels always means 0..n. Combined with the fact that ASIO channels are always mono (by design), this effectively means that only the first input and output channel of each device can be used. The current implementation works around this by including a mixer, which combines mono channels to stereo using an intermediate buffer. However, I'm very apprehensive of going this route, as it misrepresents the device's topology. The ability to record/playback stereo streams on a device without stereo channels might cause confusion among dependents.

One might point out that other backends feature native mixers out of the box, such as WASAPI's system wide sound server, but this isn't quite comparable, because in those cases, the native mixer IS the origin/destination of the stream, as far as cpal is concerned.

No Bulk Stream Creation

Another problem I faced is that in cpals API, streams are created one at a time, but ASIO synchronizes all channels by design, including their buffer size, sample position, buffer callback, running state, etc. Therefore, it isn't even possible to use the 1 input and 1 output channel that are accessible at the same time, unless there is some serious spaghetti going on under the hood. (edit: the upcoming duplex API solves this)

Overwhelming trait requirements

The device and stream traits impose some requirements that I struggled to fulfill. For instance, devices being Clone was counter intuitive to me. I understand the notion of it being a device identifier, rather than an life handle, but this just raises the question where the actual life handle (driver instance in ASIO's case) should reside. I ended up putting it behind an Arc that all Device and Stream instances hold, but it felt like cheating the system.

The Send bound on Stream was also tricky. ASIO uses apartment threaded COM, and painfully doesn't support marshalling AT ALL, which is very hard to make thread safe. Maybe this more of a criticism of ASIO rather than of cpal, but it is also what we have to work with.

Conclusion

Maybe it is a skill issue, but I am unable to come up with a "correct" design of fitting ASIO into cpal's API to an extent that is practically useful. I don't know how well the other backends fared in this regard, would love to hear some opinions on this.

I'm actually considering to make myself familiar with the other backends before continuing with ASIO, so I can get a bigger picture of why cpal is the way it is, but I fear I may be underestimating the depth of this rabbit hole.

---

¹ASIO 2.3 Specification, section IV.1:

[...] However, the implementation is currently limited to one active driver. Also, it implies that a driver’s name will have max. 32 characters including the terminating '\0'.

[HEnquist]

I just switched CamillaDSP's ASIO backend from asio-sys to azo (PR).

Main reason was multi-device support, and it works: capture and playback can now use two different ASIO drivers at once, with independent clocks and async resampling between them. Not possible before.

Second was simpler builds. asio-sys downloads the SDK at build time and runs bindgen, so it needs LLVM and a C++ toolchain. azo is pure Rust, so my Windows CI lost a step and ASIO is now part of the normal Windows build instead of a separate binary. The licensing cleanup is a welcome side effect: the SDK forced my Windows builds to GPLv3 only, and now the normal dual license applies everywhere.

Unexpected bonus: sample rate changes got simpler. Some drivers reported the requested rate while still running at the old one, so the SDK needed a workaround of dummy buffer creation, start, stop, dispose and reload. With azo, dropping and recreating the instance is enough, apparently because azo pairs CoInitializeEx with CoUninitialize where the SDK path never called it at all. About 110 lines of workaround deleted.

Tested on a MOTU M Series and the Steinberg built-in driver.

Obviously it's a young crate, but it has been solid for me and is a much better base than the SDK. I can recommend making the switch. It's not difficult and the benefits are real.

[LastExceed]

Thank you for this feedback, this makes me very happy :D

I'll use this occasion to give a status update: I have created a custom host implementation based on azo that is technically complete (as in it implements all of cpals api), but practically useless at the moment, because I didn't include the workaround for #1301 that the current ASIO contains (see OP). So progress on this issue is now essentially blocked on #1301, which I want to tackle as part of the config api rework that is already on the v0.19 roadmap. This also gives azo a bit more time to mature.

If we also manage to cover #367 along the way, then this would also solve the "no bulk stream creation" concern outlined in the OP. Meanwhile the "overwhelming trait requirements" turned out to be a deliberate design choice, which I was able to deal with, so that point can already be crossed off the list.

@HEnquist If you have encountered any rough edges while working with azo, I'd love to hear about it. I want to make azo as good as can be. (but perhaps file that as an issue on azo's repo, as it wouldn't belong here)

[roderickvd]

@HEnquist thank you for taking the time and providing your feedback here, even when CamillaDSP doesn’t really use cpal anymore does it (sad panda faces)?

Very encouraging to azo and @LastExceed! When you feel ready, we could think about adding azo as an optional feature so people can already start experimenting with it and meanwhile help azo mature.

[HEnquist]

@LastExceed I only ran into two minor things, and both are already fixed (LastExceed/azo#2). Nothing else to report. The crate is much more ready than the pre-0.1.0 version number indicates.

@roderickvd np, happy to help. And correct, CamillaDSP doesn't use cpal any more. Turns out all those annoying platform api quirks and complications that cpal does such a good job of hiding actually contain some stuff I need, on CoreAudio and WASAPI in particular. It was working nicely for Jack, but I recently dropped that to keep things manageable.

[roderickvd]

Turns out all those annoying platform api quirks and complications that cpal does such a good job of hiding actually contain some stuff I need

Ah, I see. Compliments on your backend implementations which seem quite robust. I even gleaned into them to improve cpal's. I was thinking it's a shame for the limited Rust audio community to duplicate and fragment its efforts. I've been hoping it wasn't due to cpal having seen limited maintenance up to, say, 0.15 when I picked it up.

So I get your rationale and again truly appreciate you reaching out to leave feedback here. If you see ways to extend cpal to your needs - while keeping it cross-platform - and focus community efforts, do not hesitate to reach out.

[HEnquist]

@roderickvd What I was missing was really the platform specific things that are hard to fit in a cross-platform api. Two examples: device property change notifications, and being able to control custom device properties, like tuning the clock of the BlackHole virtual device on macOS. The notifications work very differently on each platform, and the event data is so different that I have no idea how one would write a good abstraction for it. So I'm not sure any of this belongs in cpal at all.

What would help is an escape hatch that hands out the native handle behind a device or stream. cpal already sits on coreaudio-rs and friends, so exposing that would let an app stay cross-platform and still reach through for the few things that only exist on one platform, instead of dropping cpal entirely like I did.

We're drifting quite far off topic here, but happy to discuss the idea more in a suitable place.

Edit: funny you should mention looking at my backends. I have looked a lot at the cpal code when building those :D

[LastExceed]

The crate is much more ready than the pre-0.1.0 version number indicates.

I plan to promote azo to 0.1.0 with the next release, which lines up perfectly with the current version being 0.0.9 (I actually didn't plan this, it just happens to coincide). In the beginning I didn't really know yet what I wanted my API to look like, and anticipated that I would be making breaking changes left and right during the early versions (which I did), so I chose to start from 0.0.1 just so I didn't have to think about what is and isn't breaking 😅 But I got it mostly figured out now, so its time to upgrade. 0.1.0 might still undersell the crate a bit, since most projects start on that version (due to cargo new defaulting to it), but its definitely too early for 1.0.0, and at the end of the day its just arbitrary numbers.

What would help is an escape hatch that hands out the native handle behind a device or stream

That would be #49

[edwloef]

However, I'm very apprehensive of going this route, as it misrepresents the device's topology.

Does this imply that a user of CPAL would need to open n mono output streams, and synchronize playback between them, to have n-channel output with azo (and vice versa with input)? I would argue that that falls under CPAL's responsibility, since it's not something you'd want to do with any other host, and CPAL is explicitly supposed to offer a cross-platform API.

Additionally, if forbidding multiple simultaneous ASIO streams on the same device goes through (as per #1313 (comment)), how is a user of CPAL meant to work with multi-channel audio when using azo?

[LastExceed]

Apologies for the confusion. My stance has changed, and I haven't properly communicated that. Specifically, #367 has changed my mind, and I'd like to retract my claim about misrepresenting device topology.

However, I'm very apprehensive of going this route, as it misrepresents the device's topology.

Does this imply that a user of CPAL would need to open n mono output streams

That is basically what I was thinking when I initially wrote this, yes. But I have since recognized the problems with that approach myself, so disregard that.

and synchronize playback between them

My initial thought process was that ASIO streams always being synchronized would make this trivial. But I eventually realized that it is actually rather a reason not to support multiple streams, as stopping one stream would automatically stop all others as well, which is very unintuitive.

I would argue that that falls under CPAL's responsibility

I now fully agree. What I considered to be out of scope (and still do) is specifically the conversion between interleaved and sequential data, and #367 would remove the need to do that.

if forbidding multiple simultaneous ASIO streams on the same device goes through (as per #1313 (comment)), how is a user of CPAL meant to work with multi-channel audio when using azo?

I aknowledge that I am essentially contradicting myself there. By the time I wrote that comment you linked, I had already scrapped the mono-only idea (in favor of #367), but failed to communicate this.

To summarize:

My current stance is that ASIO devices should only support 1 stream each, because ASIO implicitly synchronizes all channels. But in order to make that 1 stream flexible enough in terms of channel coverage, we need #367 and #1301. This is a major paradigm shift from what we currently have, but also creates the perfect opportunity to switch to azo: First we put all the necessary APIs in place, and then instead of updating the asio-sys based implementation, we switch to the azo based implementation, where I simply never implement multi-stream support in the first place.

[edwloef]

Thanks for clarifying, that makes sense to me :)