feat: WebAssembly binding (full stack)

[gergelyvagujhelyi]

Description

JavaScript binding for NobodyWho — runs local LLMs in a browser tab (or any wasm host) via llama.cpp compiled to wasm32 with Emscripten. The same core engine that powers the Python, Godot, Flutter, and Uniffi bindings, exposed to JS through wasm-bindgen.

This PR consolidates the entire stack that was previously staged across #526, #529, #530 — those have been closed in favour of this single PR per maintainer request. All commits preserved.

What's delivered

Core surface

import createNobodyWhoModule from '@nobodywho/js';
const m = await createNobodyWhoModule();

// Chat — inference runs on a background pthread (Emscripten pthreads),
// same std::thread::spawn code path as native. Pass modelUrl (browser)
// or modelPath (Node).
const chat = await m.Chat.create({
  modelUrl: 'https://huggingface.co/.../model.gguf',
  systemPrompt: 'You are a helpful assistant.',
  templateVariables: { enable_thinking: false },
  sampler: new m.SamplerBuilder().temperature(0.7).topK(40).topP(0.95, 1).dist(),
  tools: [getWeather, lookupTime],
});

// Streaming: per-token async iteration
for await (const tok of chat.ask('What is the capital of Denmark?')) {
  process.stdout.write(tok);  // each token arrives as the model samples it
}

// Streaming: collect the full response
const reply = await chat.ask('Hi').completed();

// Both return a TokenStream with:
//   .next()       → Promise<{value: string, done: boolean}>  (async iterator)
//   .completed()  → Promise<string>  (buffered full response)
//   [Symbol.asyncIterator] support for `for await`

// Multimodal: pass an array of mixed string | Image | Audio parts
const img = m.Image.fromBytes(jpegBytes);
const desc = await chat.ask(['Describe this:', img]).completed();

// Tools — sync or Promise-returning callbacks both work
const tool = m.Tool.fromFn(
  'get_weather',
  'Get current weather for a city',
  { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
  async (args) => fetch(`https://api.example/weather?city=${args.city}`).then(r => r.text()),
);

// Structured output — constrain generation to a JSON schema
const jsonChat = await m.Chat.create({
  modelUrl: 'https://...',
  sampler: m.SamplerPresets.constrainWithJsonSchema({ type: 'object', properties: { city: { type: 'string' } } }),
});

// Embeddings + reranking — share the same Model instance
const model = await m.Model.load({ modelUrl: 'https://...' });
const encoder = new m.Encoder(model, 2048);
const vec = await encoder.encode('the quick brown fox');
const xenc = new m.CrossEncoder(model, 2048);
const ranked = await xenc.rankAndSort('query', ['doc1', 'doc2']);

API status

Surface	Status
Model.load({modelUrl | modelPath, mmprojUrl | mmprojPath})	✅ verified
Encoder.encode(text) → Float32Array	✅ verified
CrossEncoder.rank(query, docs) / rankAndSort(...)	✅ verified
Chat.create({modelUrl | modelPath, ...}) → Chat	✅ verified
Chat.ask(prompt) → TokenStream	✅ verified
for await (const tok of stream) (async iteration)	✅ verified
TokenStream.next() / .completed()	✅ verified
Chat.terminate() → Promise<void>	✅ verified (no-op, kept for API compat)
SamplerConfig / SamplerBuilder / SamplerPresets	✅ verified
Structured output via constrainWithJsonSchema() / constrainWithRegex() / constrainWithGrammar()	✅ verified
Tool calling (Tool.fromFn(...), sync + async callbacks)	✅ verified
Multimodal vision/audio (Image.fromBytes / Audio.fromBytes)	✅ verified
Chat.create({modelUrl}) — streaming download + Cache API	✅ verified
Chat.create({modelPath}) — NODEFS disk access (Node-only)	✅ verified
mmap-backed tensor loading (CPU_Mapped)	✅ verified

Threading model: Emscripten pthreads

The wasm build uses Emscripten pthreads (-pthread, +atomics,+bulk-memory,+mutable-globals Rust target features). This enables std::thread::spawn on wasm — the same code path as native. ChatHandleAsync::new() spawns a real pthread for inference, and llama.cpp's ggml threadpool uses available_parallelism() to pick n_threads (maps to navigator.hardwareConcurrency in browser, os.cpus().length in Node).

Browser requirement: serving origin must set Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp headers for SharedArrayBuffer.

Build requirement: Rust nightly with -Zbuild-std=std,panic_abort (pre-compiled std for wasm32-unknown-emscripten lacks atomics).

The Chat class directly wraps the core ChatHandleAsync — no Web Worker wrapper, no message protocol, no RPC bridge. Token streaming works through tokio channels across pthreads, same as native.

Build target: wasm32-unknown-emscripten

Build pipeline:

   nobodywho/core (Rust)  +  llama-cpp-2 fork (wasm-emscripten branch)
        |
        | cargo +nightly -Zbuild-std=std,panic_abort
        | emcc for C/C++ side, rustc for Rust side
        v
   wasm32-unknown-emscripten .wasm  (with -pthread, SharedArrayBuffer)
        |
        | patched wasm-bindgen-cli (pthreads-compatible)
        | + post-link emcc with -pthread + --js-library
        v
   pkg-bundler/
     ├── nobodywho_js.js          (Emscripten loader with pthread runtime)
     ├── nobodywho_js_bg.wasm     (linked wasm, shared memory, ~10 MB)
     ├── library_bindgen.js       (kept for debugging)
     └── pre.js                   (HEAP_DATA_VIEW shim, inlined by emcc)

Three temporary forks (all will go away when upstream merges):

• nobodywho-ooo/wasm-bindgen — descriptor-interpreter fixes + Emscripten output mode + pthreads compatibility (skip thread/multivalue transforms, synthesize stack pointer shim from emscripten exports)
• nobodywho-ooo/llama-cpp-rs — CMAKE_SYSTEM_PROCESSOR=wasm32, -matomics -mbulk-memory for shared-memory link compat, MA_NO_* defines for miniaudio, -fexceptions for mtmd
• walkingeyerobot/emscripten — the -sWASM_BINDGEN flag (PR Add wasm-bindgen support emscripten-core/emscripten#23493)

Multimodal (Path A — MEMFS-virtualized)

Vision and audio input work end-to-end through bytes. Image.fromBytes(uint8) / Audio.fromBytes(uint8) write to content-hashed MEMFS paths; llama.cpp reads them via strong syscall overrides in js/src/syscall_imports.rs.

Verified: Qwen3.5-0.8B vision, Qwen2-VL, Gemma 3 vision, Qwen3-ASR audio (WAV/MP3/FLAC).

Tool calling

Tool.fromFn(name, description, jsonSchema, callback) — the callback runs directly in the inference context (no RPC bridge needed since pthreads share the same wasm instance). Both sync and async (Promise-returning) callbacks work.

Structured output

Constraints via SamplerPresets.constrainWithJsonSchema() / constrainWithRegex() / constrainWithGrammar(). llguidance works on Emscripten (its clock_gettime requirement is satisfied by Emscripten's libc).

Core changes that affect other bindings

• tokio features split (rt-multi-thread native vs rt wasm); native-only deps gated behind cfg(not(target_family = "wasm")).
• Worker channel: std::sync::mpsc → tokio::sync::mpsc::unbounded_channel. Public API unchanged.
• std::thread::spawn used on all targets (including wasm, via Emscripten pthreads). No more spawn_local wasm path.
• WorkerGuard unified — single struct with JoinHandle on all targets.
• n_threads uses available_parallelism() on wasm (was hardcoded to 1).
• New mtmd cargo feature (default-on), get_model_from_path, get_model_from_bytes, Tool::new_async, mtmd_marker_string().

Native consumers (Python, Godot, Flutter, uniffi) see no API changes. cargo check --workspace passes cleanly.

v1 limitations

• 4 GiB ceiling — wasm32 hard limit. Models + KV cache + compute must fit.
• Browser COOP/COEP — pthreads require cross-origin isolation headers.
• Audio: WAV/MP3/FLAC only — Ogg not supported under Emscripten.
• OpenAI-typed-content chat templates (SmolVLM, some Phi-3-Vision) — not yet supported.

Tested

• cargo check --workspace: clean on native.
• cargo test -p nobodywho-js: lint tests pass.
• bash js/scripts/build-pkg-emscripten.sh: ~60s on M-series macOS.
• All 17 smoke tests pass under Node (forawait, sampler, tool, stop, history, setters, terminate, context-shift, constraint, audio, vision, modelpath, parity-extras, etc.)
• Browser demos verified with COOP/COEP headers.

Type of change

• New feature (non-breaking change which adds functionality)
• Breaking change
• Documentation update

[gergelyvagujhelyi]

/full-ci