Consider aiofastnet-owned TLSConfig/native TLS backend for uv-managed Python

[river-walras]

One possible longer-term direction is to make TLS independent from Python's _ssl internals by
introducing an aiofastnet-owned TLSConfig / native TLS backend.

Right now aiofastnet needs to reuse the exact OpenSSL instance behind Python's ssl module, because it
operates on SSL_CTX* / SSL* created by Python. That makes uv-managed Python difficult to support
when _ssl is statically linked against OpenSSL and no matching libssl / libcrypto can be loaded.

A possible architecture could be:

• Keep the current backend for ssl.SSLContext, requiring dynamically linked Python OpenSSL.
• Add a separate TLSConfig API owned by aiofastnet.
• Compile/cache TLSConfig into aiofastnet-managed native TLS context objects.
• Create per-connection native TLS objects from that cached context.
• Do not mix Python _ssl objects with a different OpenSSL instance.

This would make the limitation explicit:

• ssl.SSLContext path: maximum Python compatibility, but requires dynamic OpenSSL.
• aiofastnet.TLSConfig path: works with uv/static Python builds, but only supports the TLS options
  aiofastnet implements.

[tarasko]

Yes, something like that is certainly possible and would be nice to have, although there are a few things to keep in mind:

In dynamic case, aiofastnet uses whatever python's ssl module has already loaded and use.
In static case aiofastnet would have to load and rely on system's OpenSSL, which may or may not be present, or can be very old. Right now aiofastnet requires at least openssl 1.1, and I was a bit struggling to make it work for 1.0. But it is certainly possible.

Libraries that use aiofastnet directly as a drop-in replacement speedup normally would try to import aiofastnet, and if import fails then use native loop methods as a fallback. Failed import is a very simple mental model. If import succeed, but then you need to check if SSLObject can be used or not, this is more complicated.

This approach is used in aiohttp, for example, for all speedup packages. And I'm currently working on a PR that adds aiofastnet to aiohttp's speedups extra. I like 'failed import model', it is simple and clean.

This can be mitigated with a special env var probably. Like, if AIOFASTNET_ALLOW_STATIC_LINKAGE=1 defined then import succeed even in case of statically linked OpenSSL, but user will be expected to use TLSConfig instead of SSLContext then.

Higher-level libraries get SSLContext from user, or the even higher level library. They might expect actual ssl.SSLContext type.

I can imagine scenarios like user code -> trading library -> binance connector library -> aiohttp or picows or websockets -> aiofastnet.
SSLContext or TLSConfig will have to be created at the user code level and will go through all the layers before it reaches aiofastnet. If some layer for some reason explicitly rely on SSLContext type or maybe create SSLContext themselfs, it may not work at all.

I haven't explored it yet, but maybe it would be possible to recreate SSL_CTX* from python ssl.SSLContext attributes? Or maybe with additional hacking of PySSLContext layout? I hope it didn't change too much between 3.9 and 3.15. Then TLSConfig won't be necessary.

If not, then I would try to make TLSConfig interface as close to ssl.SSLContext interface as possible, so users could use duck-typing.

[tarasko]

So, after giving it some thoughts I came up with the following plan:

• If user has passed regular ssl.SSLContext then SSL fallback implementation will be used for uv python. Fallback implementation uses regular ssl.SSLObject and ssl.MemoryBIO. (Implement SSL fallback for uv standalone python  #40). This change is relatively small and contained to providing alternative fallback implementation to SSLObject. It will be rolled out first. Fallback is slightly slower than native implementation used with regular python, but still significantly faster than stock SSLTransport in asyncio and just a little bit faster than SSLTransport in uvloop.

• If user passes aiofastnet.SSLContext then bundled OpenSSL will be always used, regardless of python version. (Add bundled OpenSSL backend for uv Python #39).

[river-walras]

So, after giving it some thoughts I came up with the following plan:

• If user has passed regular ssl.SSLContext then SSL fallback implementation will be used for uv python. Fallback implementation uses regular ssl.SSLObject and ssl.MemoryBIO. (Implement SSL fallback for uv standalone python  #40). This change is relatively small and contained to providing alternative fallback implementation to SSLObject. It will be rolled out first. Fallback is slightly slower than native implementation used with regular python, but still significantly faster than stock SSLTransport in asyncio and just a little bit faster than SSLTransport in uvloop.
• If user passes aiofastnet.SSLContext then bundled OpenSSL will be always used, regardless of python version. (Add bundled OpenSSL backend for uv Python #39).

Great, when will the new version being released?

[tarasko]

0.19.0 has been just released with this:

If user has passed regular ssl.SSLContext then SSL fallback implementation will be used for uv python. Fallback implementation uses regular ssl.SSLObject and ssl.MemoryBIO. (Implement SSL fallback for uv standalone python #40). This change is relatively small and contained to providing alternative fallback implementation to SSLObject. It will be rolled out first. Fallback is slightly slower than native implementation used with regular python, but still significantly faster than stock SSLTransport in asyncio and just a little bit faster than SSLTransport in uvloop.