fix(core): stop run_nvrtc corrupting the interned bytes singleton

[harjothkhara]

What does this PR do ?

Fixes a process-wide memory-corruption bug in run_nvrtc(): it allocated NVRTC output buffers as immutable b" " * size, and for an empty compile log (size == 1) CPython returns the interned, process-wide 1-byte singleton b" ". NVRTC then writes its C-string NUL terminator into that shared object, turning every b" " / bytes([32]) / 1-byte space slice in the whole process into b"\x00" permanently. Fix: use a mutable bytearray.

Collection: core (nemo/core/utils/cuda_python_utils.py)

Changelog

• nemo/core/utils/cuda_python_utils.py: allocate the NVRTC log buffer and PTX buffer as bytearray(size) / bytearray(ptxSize) instead of b" " * size (mutable, never interned).
• Convert with bytes(ptx) before np.char.array(...) so downstream behavior is byte-for-byte identical (passing a bytearray directly to np.char.array would be misinterpreted as a sequence of ints).
• tests/core/test_cuda_python_utils.py (new): GPU-free regression test that fakes the NVRTC/CUDA bindings and emulates NVRTC writing into the buffer via memoryview; asserts the buffers are mutable and the interned singleton is intact. Fails on the old code (TypeError: cannot modify read-only memory), passes on the fix. Skips when cuda-python is unavailable, matching tests/collections/asr/decoding/test_cuda_graph_rnnt_greedy_decoding.py.

Usage

Reproduction (self-contained, no GPU / no NeMo import — just cuda-python), from #15790:

from cuda.bindings import nvrtc
err, prog = nvrtc.nvrtcCreateProgram(b'extern "C" __global__ void k(){}\n', b"k.cu", 0, [], [])
(err,) = nvrtc.nvrtcCompileProgram(prog, 0, [])
err, size = nvrtc.nvrtcGetProgramLogSize(prog)   # == 1 for a clean compile
buf = b" " * size                                # the interned singleton b" "
(err,) = nvrtc.nvrtcGetProgramLog(prog, buf)     # NVRTC writes NUL into it
print(bytes([32]))                               # -> b'\x00'  (b" " is now corrupted process-wide)

GitHub Actions CI

CI has not been triggered yet (external fork). A maintainer can start it with /ok to test <sha> / "Approve and run" and the Run CICD label.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests? — added tests/core/test_cuda_python_utils.py
• Did you add or update any necessary documentation? — n/a (internal bugfix)
• Does the PR affect components that are optional to install? (cuda-python)
  • Reviewer: import guards are unchanged — run_nvrtc stays behind @cuda_python_required, and the new test pytest.skips when cuda-python is unavailable.

PR Type:

• New Feature
• Bugfix
• Documentation

Additional Information

• Fixes run_nvrtc() corrupts CPython's interned 1-byte bytes singleton b' ' when the NVRTC compile log is empty #15790
• Verification note: I do not have a CUDA machine, so I confirmed the bug mechanism and the test's pass/fail behavior with a standalone replica of the buffer handling; the in-repo test is expected to run under CI where cuda-python is present.
• cc @nithinraok (NeMo core) per the review guidelines.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.