Extend d8/v8 to read stdin to TypedArray, or just keep using os.system()?

[guest271314]

Right now I'm using QuickJS to read standard input stream to d8 with os.system(), to get around readline() not being capable of processing Uint32Array from Chrome.

How would I go about translating working QuickJS code to extending d8 to read standard input into TypedArrays?

In d8

function encodeMessage(str) {
  return new Uint8Array([...str].map((s) => s.codePointAt()));
}
// ...
    const stdin = os.system("./readFile.js", [
      `/proc/${pid.replace(/\D+/g, "")}/fd/0`,
    ]);
    if (stdin != undefined && stdin != null && stdin.length) {
      const data = encodeMessage(stdin); 
      const size = data.subarray(0, 4);
      const view = new DataView(size.buffer);
      const length = view.getUint32(0, true);
      sendMessage(encodeMessage(JSON.stringify({ length, size: [...size] })));
      const message = data.subarray(4);
// ...

#!/usr/bin/env -S /home/user/bin/qjs -m --std
// Read stdin to V8's d8, send to d8
// const stdin = os.system("./read_d8_stdin.js", [`/proc/${pid.replace(/\D+/g, "")}/fd/0`]);
function read_d8_stdin([, path] = scriptArgs) {
  try {
    // Uint32Array to read message length into
    const size = new Uint32Array(1);
    const err = {
      errno: 0
    };
    // Open /proc/PID/fd/0 for reading
    const pipe = std.open(
      path,
      "rb",
      err,
    );
    if (err.errno !== 0) {
      throw `${std.strerror(err.errno)}: ${path}`;
    }
    // Read /proc/PID/fd/0 into Uint32Array
    pipe.read(size.buffer, 0, 4);
    // Uint8Array to read message into
    const output = new Uint8Array(size[0]);
    // Read /proc/PID/fd/0 into Uint8Array
    pipe.read(output.buffer, 0, output.length);
    // Uint8Array containing Uint32Array buffer in Uint8Array and Uint8Array containing message
    const data = new Uint8Array([...new Uint8Array(size.buffer), ...output]);
    // Write Uint8Array to d8's stdin
    std.out.write(data.buffer, 0, data.length);
    std.out.flush();
    // Exit and start again initiated from d8
    std.exit(0);
  } catch (e) {
    // Handle error
    const err = {
      errno: 0
    };
    const file = std.open("qjsErr.txt", "w", err);
    if (err.errno !== 0) {
      file.puts(JSON.stringify(err));
      file.close();
      std.exit(1);
    }
    file.puts(JSON.stringify(e));
    file.close();
    std.out.puts(JSON.stringify(err));
    std.exit(1);
  }
}

read_d8_stdin();

[saghul]

Right now I'm using QuickJS to read standard input stream to d8 with os.system(), to get around readline() not being capable of processing Uint32Array from Chrome.

Can you please elaborate? It's not clear to me what you are trying to do here.

[guest271314]

I'll add the V8 folks do not appear interested in extending d8 shell to process stardard input to an ArrayBuffer https://groups.google.com/g/v8-users/c/NsnStT6bx3Y/m/Yr_Z1FwgAQAJ.

[saghul]

Rather than sending the binary data, did you consider sending a serialized version of the Uint32Array (JSON perhaps?) if you send that + \n then I suppose readline will work, and you can deserialize the payload on the other end.

[guest271314]

Tried that. d8s readline() is exiting right after reading the raw "32-bit message length in native byte order". d8's readline() and SpiderMonkey's readline() don't work the same.

[saghul]

I am not familiar with either of those, but I guess they might interpret what they are reading as newline and stopping there. In that case sticking to QuickJS might be the right option, other than switching to another engine like Deno or Bun.

[guest271314]

@saghul FWIW guest271314/v8@75ec4af#diff-3b74c96d943457c2c7022c0d1c1fb076f9faaf40704a76450b563b4d4b0513ac