diff --git a/.changeset/base64-accept-bytearray-passable.md b/.changeset/base64-accept-bytearray-passable.md new file mode 100644 index 0000000000..47d3a9a9e5 --- /dev/null +++ b/.changeset/base64-accept-bytearray-passable.md @@ -0,0 +1,21 @@ +--- +'@endo/base64': patch +--- + +`@endo/base64` now encodes a frozen `Uint8Array` byteArray passable (issue +#573) correctly, bringing it to parity with its `@endo/hex` twin. + +`jsEncodeBase64` and `encodeBase64` accept a `Uint8Array` (the narrowed +byteArray shape) and gate on `ArrayBuffer.isView`, the committed +genuine-vs-emulated distinguisher: a genuine view (mutable or immutable +buffer) is read in place, while an emulated `@endo/immutable-arraybuffer` +wrapper — a plain object reporting `isView === false`, whose `bytes[i]` reads +`undefined` — is thawed into a mutable `Uint8Array` first. +`encodeBase64` also dispatches to the native `Uint8Array.prototype.toBase64` +intrinsic (or the legacy `globalThis.Base64.encode` XS binding) only for +genuine views, whose bytes the native code can read; an emulated wrapper falls +through to the pure-JavaScript polyfill. Previously the polyfill's +integer-indexed read silently produced all-zero output for an emulated +byteArray, and the native path had no such guard. Not reached by an in-repo +passable today, but the byteArray narrowing that reached `@endo/hex` did not +reach its twin. diff --git a/.changeset/byte-array-hex-codecs.md b/.changeset/byte-array-hex-codecs.md new file mode 100644 index 0000000000..05a318e0ef --- /dev/null +++ b/.changeset/byte-array-hex-codecs.md @@ -0,0 +1,29 @@ +--- +'@endo/marshal': minor +--- + +A `byteArray` (a plain frozen `Uint8Array` backed by an immutable +`ArrayBuffer`) is now serializable through the capdata, smallcaps, +encode-passable, and marshal-justin codecs. + +- **capdata**: byteArray encodes as `{"@qclass":"byteArray","data":""}`. +- **smallcaps**: byteArray encodes as `"*"`. The reserved `*` prefix is + now assigned to byteArray. +- **encode-passable**: byteArray encodes as + `a:`. The Elias-delta length prefix gives + shortlex ordering (matching `compareRank`) with no arbitrary size cap, and + every character in the body is safe inside both `legacyOrdered` and + `compactOrdered` array framings. +- **marshal-justin**: renders byteArray as + `frozenBytes(decodeHex(""))`. + +Hex conversion uses `@endo/hex` (`encodeHex` / `decodeHex`), and the decoded +`Uint8Array` is converted into a passable byteArray with +`@endo/immutable-arraybuffer`'s `frozenBytes`. + +Syrup already supported this value; no change required there. + +Deploy sequencing: producers should not emit byteArrays until decoders are +upgraded. Older decoders reject the new encodings (unknown `@qclass`, +unknown smallcaps prefix, unknown encode-passable prefix); consumers must +ship the new decoder before producers begin emitting byteArray values. diff --git a/.changeset/consolidate-immutable-byte-utilities.md b/.changeset/consolidate-immutable-byte-utilities.md new file mode 100644 index 0000000000..cc07aac20a --- /dev/null +++ b/.changeset/consolidate-immutable-byte-utilities.md @@ -0,0 +1,29 @@ +--- +'@endo/immutable-arraybuffer': minor +'@endo/bytes': major +'@endo/marshal': major +'@endo/ocapn': patch +'@endo/thixotrope': patch +--- + +Consolidate the immutable byte utilities onto a single shared implementation +exported from `@endo/immutable-arraybuffer`, and rename them to `frozenBytes` +(previously `@endo/bytes`' `bytesToImmutable`) and `thawedBytes` (previously +`bytesFromImmutable`). `frozenBytes` wraps a `Uint8Array` view's contents in a +hardened frozen `Uint8Array` backed by an immutable `ArrayBuffer` (a +`'byteArray'` passable); `thawedBytes` copies such a value back out into a fresh +mutable `Uint8Array`. Importing the package's new main entry installs the shim +as a side effect, since `frozenBytes` depends on it; the bare install remains +the separate `@endo/immutable-arraybuffer/shim.js` export. + +Breaking (no backward compatibility is preserved): + +- `@endo/bytes` no longer exports `./to-immutable.js` (`bytesToImmutable`) or + `./from-immutable.js` (`bytesFromImmutable`). Import `frozenBytes` and + `thawedBytes` from `@endo/immutable-arraybuffer` instead. `@endo/bytes` keeps + `./concat-immutables.js` (`concatImmutables`), now implemented on the shared + utilities. +- `@endo/marshal`'s `decodeToJustin` now emits `frozenBytes(decodeHex(...))` + instead of `bytesToImmutable(decodeHex(...))` for byteArray values, so a + Justin evaluation environment must bind `frozenBytes` rather than + `bytesToImmutable`. diff --git a/.changeset/freezable-typedarray-emulation.md b/.changeset/freezable-typedarray-emulation.md new file mode 100644 index 0000000000..9a6846bf7f --- /dev/null +++ b/.changeset/freezable-typedarray-emulation.md @@ -0,0 +1,42 @@ +--- +'@endo/immutable-arraybuffer': minor +'ses': patch +--- + +Add freezable TypedArray emulation for immutable-ArrayBuffer-backed views. + +After loading `@endo/immutable-arraybuffer/shim.js`, constructing a TypedArray +from an emulated immutable `ArrayBuffer` produces an emulated freezable wrapper +whose mutator methods (`copyWithin`, `fill`, `reverse`, `set`, `sort`) throw +`TypeError`, whose `buffer` getter returns the immutable wrapper rather than +the underlying genuine buffer, and which can be frozen via `Object.freeze`. +The wrapper inherits directly from `T.prototype` with no intermediate prototype. + +The genuine-buffer constructor path (passing a regular mutable `ArrayBuffer`) +is unchanged: the result is a normal writable TypedArray view. + +`ses`: the permits walk accepts the shim-installed `%TypedArrayPrototype%` +slots without complaint; no new permit rows are required. + +The shim commits to a single emulated-vs-genuine fidelity loss, +`ArrayBuffer.isView`: an emulated freezable wrapper is a plain ordinary object +and reports `ArrayBuffer.isView === false`, whereas a genuine view (mutable, or +native-immutable) reports `true`. This is the one distinguisher downstream +clients (`@endo/bytes` / `@endo/pass-style`) are entitled to rely on, and it is +pinned by a regression test so an accidental change trips a test rather than +corrupting a consumer. + +The shim also repairs the emulated wrapper's `[Symbol.toStringTag]`: it replaces +the genuine `this`-sensitive `%TypedArrayPrototype%[Symbol.toStringTag]` getter +with a wrapper that amplifies an emulated wrapper to its hidden genuine +TypedArray, so `Object.prototype.toString.call(emulatedView)` now reads +`'[object Uint8Array]'` — matching a genuine view — instead of +`'[object Object]'`. This is a getter-wrapper fix, not a `[Symbol.toStringTag]` +data property (the wrapper still carries no own tag), so the getter and +`Object.prototype.toString` agree. `[Symbol.toStringTag]` is therefore no longer +an emulated-vs-genuine distinguisher; `ArrayBuffer.isView` remains the single +committed distinguisher. A brand check that captures this getter after the shim +installs (e.g. `@endo/harden`'s `isTypedArray`) will classify an emulated wrapper +as a TypedArray and route it through the `freezeTypedArray` path, which is benign +(the wrapper has no own integer-indexed properties, so it freezes without +throwing in either capture order). diff --git a/.changeset/narrow-bytearray-to-uint8.md b/.changeset/narrow-bytearray-to-uint8.md new file mode 100644 index 0000000000..01590a26ad --- /dev/null +++ b/.changeset/narrow-bytearray-to-uint8.md @@ -0,0 +1,102 @@ +--- +'@endo/pass-style': major +'@endo/bytes': major +'@endo/patterns': patch +'@endo/marshal': patch +'@endo/ocapn': patch +'@endo/ocapn-noise': patch +--- + +Narrow the `byteArray` pass style to plain frozen `Uint8Array` only. + +The `byteArray` pass-style brand check previously accepted both raw +immutable `ArrayBuffer` values and plain frozen `Uint8Array` values +backed by an immutable `ArrayBuffer`. It now accepts only the latter +shape: a plain frozen `Uint8Array` whose backing buffer is a plain +frozen immutable `ArrayBuffer`. Raw immutable `ArrayBuffer` values +are no longer recognised as `byteArray`; the `ByteArray` TypeScript +alias is now `Uint8Array` (was `ArrayBuffer`). + +The emulated-vs-genuine distinction the narrowed brand check draws — an +emulated `@endo/immutable-arraybuffer` wrapper versus a genuine +integer-indexed `Uint8Array` view — is committed to a single fidelity +loss, `ArrayBuffer.isView`: an emulated wrapper is a plain object and is +not a view, a genuine view (mutable or native-immutable) is. `@endo/pass-style`'s +`byteArray` brand check discriminates on `ArrayBuffer.isView` (a non-view +must carry zero own indexed properties, a genuine view exactly +`length`-many matching the buffer), which is strictly more precise than +accepting either count unconditionally. `@endo/bytes`'s `compareBytes` +likewise gates on `ArrayBuffer.isView`, indexing a genuine view in place +and copying only a non-view (emulated) wrapper or bare buffer. The +integer-indexed-read (`view[i] === undefined`) and `[Symbol.toStringTag]` +(`'[object Object]'`) behaviors of an emulated wrapper are incidental +consequences of its plain-object shape, not separately committed fidelity +losses. + +`@endo/bytes`: the immutable-byte adapters — consolidated into +`frozenBytes` and `thawedBytes` and re-homed in +`@endo/immutable-arraybuffer` (see the consolidation changeset) — take +on the narrowed shape. `frozenBytes(view)` now wraps the immutable +`ArrayBuffer` produced by `sliceToImmutable` in a fresh frozen +`Uint8Array` before hardening; the return type is now `Uint8Array` +(was `ArrayBuffer`). `thawedBytes` accepts the new shape +(`ArrayBufferView`) in addition to the prior `ArrayBufferLike`. +`concatImmutables` returns a `Uint8Array` rather than an +`ArrayBuffer`, and accepts either shape on input. `bytesEqual` now +gates on `ArrayBuffer.isView` like its `compareBytes` sibling: it +compares a genuine view in place and thaws a non-view (emulated) wrapper +or bare buffer into a mutable `Uint8Array` first. Previously it indexed +its arguments directly, so two distinct equal-length emulated byteArrays +read `undefined` at every position and compared equal, while an +emulated-vs-genuine pair compared unequal. + +`@endo/marshal`: the byteArray rank-compare's `ArrayBuffer.prototype` +dispatch arm becomes dead code and is removed. Values arrive as a frozen +`Uint8Array` backed by an immutable `ArrayBuffer`. On the emulated +`@endo/immutable-arraybuffer` path such a wrapper has no integer-indexed +own properties, so the bytes are read by first copying each wrapper into +a genuine mutable `Uint8Array` (via `slice`, which the shim amplifies) +and then delegating the equal-length lexicographic comparison to +`@endo/bytes`'s `compareBytes`, deduplicating the byte-comparison loop. + +`@endo/patterns`: the `byteArray` matcher's `TypeFromPattern` and +`getMatcherKind` types resolve to `Uint8Array` (was `ArrayBuffer`). + +`@endo/ocapn`: the syrup `writeBytestring` types (and the crypto, +codec, client, cbor, and bytewise-compare byte params throughout the +package) narrow to `Uint8Array`; no function is typed to accept both a +buffer and a buffer view. Where a codec dispatcher still tolerates a raw +`ArrayBuffer` from an older peer, that buffer is normalized to a +`Uint8Array` at the boundary rather than propagated into the callee's +signature. The hub's `hexFromBytes`/`swissnumHex` helpers and +`attachSession`'s `powers.identity` handshake fields +(`sessionId`/`peerPublicKeyQ`/`selfPrivateKeyBytes`) narrow to `Uint8Array` +the same way: `hexFromBytes` gates on `ArrayBuffer.isView` (like +`@endo/bytes`' `toIndexableUint8`), reading a genuine view in place and +copying only an emulated `@endo/immutable-arraybuffer` wrapper — the shape a +`frozenBytes`/`makeSessionId` session id takes — so no handshake or +gift-handoff signature is typed to accept both a buffer and a buffer view. +The byteArray-shaped branded +client types (`SessionId`, `SwissNum`, `PublicKeyId`) change from +`ArrayBufferLike & {_brand}` to `Uint8Array & {_brand}`. Printable +swissnum strings are encoded with canonical `@endo/ascii` before immutable +wrapping. Decoder paths keep non-ASCII swissnums as bytes rather than +coercing them through the WHATWG `ascii` decoder. The CBOR +diagnostic-notation `equals`/`diagnosticEquals` helper's byte comparison +now gates on `ArrayBuffer.isView` as well, thawing an emulated wrapper +before indexing; previously (like the pre-fix `asUint8`) it trusted +`instanceof Uint8Array` and read `undefined` from an emulated wrapper, so +distinct equal-length byteArrays compared equal (latent — diagnostic +notation has no wire consumers). + +`@endo/ocapn-noise`: adapt to the narrowed `byteArray`. Its `asUint8` +helper previously trusted `instanceof Uint8Array` and returned the value +as-is, which broke the peer-key comparison in the crossed-hellos +handshake on the emulated `@endo/immutable-arraybuffer` path: the +decoded public key arrives as a frozen `Uint8Array` wrapper with no +integer-indexed own properties, so `peerBytes[i]` read `undefined` and +every byte compared unequal. It now discriminates on `ArrayBuffer.isView` +(as `@endo/immutable-arraybuffer`'s `thawedBytes` does), copying an emulated +wrapper into a genuine mutable `Uint8Array`. The stale +`OcapnNoiseSession.sessionId` type is updated from `ArrayBufferLike` to +the now-`Uint8Array`-shaped `SessionId`. diff --git a/.changeset/passstyle-typedarray-diagnostic.md b/.changeset/passstyle-typedarray-diagnostic.md new file mode 100644 index 0000000000..be73e1a1d0 --- /dev/null +++ b/.changeset/passstyle-typedarray-diagnostic.md @@ -0,0 +1,16 @@ +--- +'@endo/pass-style': patch +--- + +`passStyleOf` no longer blames mutability when a non-`Uint8Array` typed array +is rejected. The `byteArray` pass style accepts only a whole-buffer +`Uint8Array` over an immutable `ArrayBuffer`; a typed array of any other +element type is rejected for its element type, not its mutability. The +late fall-through guard previously reported every unclaimed genuine +`TypedArray` with the "Cannot pass mutable typed arrays" message, which +misleads for a genuinely frozen non-`Uint8Array` typed array over an +immutable buffer (reachable on a native Immutable-ArrayBuffer engine, and on +the shim leg under unsafe harden taming) — mutability is not the problem +there. That case now reports "Cannot pass typed arrays other than Uint8Array". +A `Uint8Array` still reports the mutable message, since it only reaches that +guard backed by a mutable buffer (an immutable-backed one is always accepted). diff --git a/.changeset/share-to-indexable-uint8.md b/.changeset/share-to-indexable-uint8.md new file mode 100644 index 0000000000..697408563e --- /dev/null +++ b/.changeset/share-to-indexable-uint8.md @@ -0,0 +1,14 @@ +--- +'@endo/bytes': patch +--- + +Deduplicate the byte-order/equality/concat readers' identical +`toIndexableUint8` helper onto one shared `./src/to-indexable-uint8.js` +module. `compareBytes`, `bytesEqual`, and `concatBytes` previously each carried +a byte-for-byte identical copy of the helper that reads a byteArray in place +when it is a genuine `ArrayBuffer.isView` and copies the emulated +`@endo/immutable-arraybuffer` wrapper into a fresh mutable `Uint8Array` +otherwise. No behavior changes. `to-string.js`' `toDecodable` is intentionally +left separate: it keys on `.immutable` (not `isView`) because +`TextDecoder.decode` rejects every immutable-backed view, so it must copy even +a genuine immutable view that the indexed readers read in place. diff --git a/packages/base64/src/encode.js b/packages/base64/src/encode.js index dc9c450ee0..1707509aef 100644 --- a/packages/base64/src/encode.js +++ b/packages/base64/src/encode.js @@ -8,6 +8,36 @@ import { alphabet64, padding } from './common.js'; // primordial, so a tampered `Function.prototype.call` cannot redirect // the dispatched native intrinsic invocation. const { apply } = Reflect; +const { isView } = ArrayBuffer; + +/** + * Normalize a `Uint8Array` to one that supports integer-indexed access + * (`bytes[i]`). + * + * The parameter type is `Uint8Array` because that is the narrowed byteArray + * passable shape (issue #573): a byteArray is always a `Uint8Array`, never a + * bare `ArrayBufferLike` nor some other `ArrayBufferView`. The runtime `isView` + * branch below is *not* type generality — it is tolerance for a single + * emulation infidelity: the emulated frozen byteArray produced by the + * `@endo/immutable-arraybuffer` shim is *typed* `Uint8Array` yet is a plain + * ordinary object that reports `isView === false` and is *not* integer-indexable + * (`wrapper[i]` reads `undefined`), so it must first be copied into a fresh + * mutable `Uint8Array`. This mirrors the identically named helper in + * `@endo/bytes/src/compare.js`. + * + * @param {Uint8Array} input + * @returns {Uint8Array} + */ +const toIndexableUint8 = input => { + if (isView(input)) { + // A genuine `Uint8Array` is indexed in place (zero allocation). + return input; + } + // Not a genuine view: the emulated `@endo/immutable-arraybuffer` wrapper. + // `.slice(0)` yields a fresh mutable array, over which the `Uint8Array` is + // integer-indexable. + return new Uint8Array(/** @type {Uint8Array} */ (input).slice(0)); +}; /** * Pure-JavaScript base64 encoder, exported for benchmarking and for @@ -26,10 +56,18 @@ const { apply } = Reflect; * This function is exported from this *file* for use in benchmarking, * but is not part of the *module*'s public API. * - * @param {Uint8Array} data + * Accepts a `Uint8Array` (the byteArray passable form): a plain mutable one, + * a genuine frozen view over an immutable `ArrayBuffer`, or an emulated + * `@endo/immutable-arraybuffer` wrapper (`ArrayBuffer.isView === false`, so + * `bytes[i]` reads `undefined`). The emulated wrapper is first thawed into a + * mutable `Uint8Array` so the integer-indexed read below sees the real bytes + * rather than silently encoding all zeros. + * + * @param {Uint8Array} input * @returns {string} base64 encoding */ -export const jsEncodeBase64 = data => { +export const jsEncodeBase64 = input => { + const data = toIndexableUint8(input); // A cursory benchmark shows that string concatenation is about 25% faster // than building an array and joining it in v8, in 2020, for strings of about // 100 long. @@ -106,20 +144,43 @@ const xsEncodeBase64 = (() => { return xsEncodeBase64; })(); +// Select the fastest available encoder for genuine, natively-readable +// `Uint8Array` inputs: the TC39 intrinsic first, then the legacy XS binding. +/** @type {typeof jsEncodeBase64 | undefined} */ +const fastEncodeBase64 = (() => { + if (nativeToBase64 !== undefined) return nativeEncodeBase64; + if (xsEncodeBase64 !== undefined) return xsEncodeBase64; + return undefined; +})(); + /** * Encodes bytes into a Base64 string, as specified in * https://tools.ietf.org/html/rfc4648#section-4. * + * Accepts a `Uint8Array` (the byteArray passable form): a plain mutable one, + * a genuine frozen view over an immutable `ArrayBuffer`, or an emulated + * `@endo/immutable-arraybuffer` wrapper. + * * Dispatches to the native `Uint8Array.prototype.toBase64` intrinsic - * when available (stage-4 TC39 proposal-arraybuffer-base64). - * Otherwise falls through to the legacy `globalThis.Base64.encode` XS - * binding, and finally to the pure-JavaScript `jsEncodeBase64`. + * (stage-4 TC39 proposal-arraybuffer-base64), or the legacy + * `globalThis.Base64.encode` XS binding, when one is available *and* the + * input is a genuine `Uint8Array` view (`ArrayBuffer.isView === true`) — + * whose bytes the native intrinsic can read directly, whether the backing + * buffer is mutable or a genuine immutable buffer. An emulated + * `@endo/immutable-arraybuffer` wrapper is a plain object (`isView === false`) + * whose bytes native C++ cannot read through the shim's proxy, so it falls + * through to the pure-JavaScript `jsEncodeBase64`, which thaws it first. + * `isView` is the committed + * genuine-vs-emulated distinguisher (issue #573); consulting it (rather than + * `.buffer.immutable`) keeps the native fast path for genuine immutable views. * * @type {typeof jsEncodeBase64} */ -export const encodeBase64 = (() => { - if (nativeToBase64 !== undefined) return nativeEncodeBase64; - if (xsEncodeBase64 !== undefined) return xsEncodeBase64; - return jsEncodeBase64; -})(); +export const encodeBase64 = + fastEncodeBase64 !== undefined + ? input => + input instanceof Uint8Array && isView(input) + ? fastEncodeBase64(input) + : jsEncodeBase64(input) + : jsEncodeBase64; Object.freeze(encodeBase64); diff --git a/packages/base64/test/forced-polyfill.test.js b/packages/base64/test/forced-polyfill.test.js index 3497ea8b8e..092d48f7e2 100644 --- a/packages/base64/test/forced-polyfill.test.js +++ b/packages/base64/test/forced-polyfill.test.js @@ -74,6 +74,14 @@ test('native-available: dispatched functions match polyfill on clean inputs', t } }); +// The encoders no longer accept a bare `ArrayBuffer`: after the #573 byteArray +// narrowing every value they receive is a `Uint8Array` (plain, genuine +// immutable view, or emulated frozen wrapper), so there is no buffer-vs-view +// type disjunction to normalize. The single remaining runtime distinction — +// a genuine `Uint8Array` view versus an emulated `@endo/immutable-arraybuffer` +// wrapper that reports `ArrayBuffer.isView === false` — is exercised in +// @endo/bytes, whose `toIndexableUint8` this file's normalizer mirrors. + test('jsDecodeBase64 rejects malformed inputs with polyfill-specific messages', t => { t.throws(() => jsDecodeBase64('%'), { message: /Invalid base64 character %/, diff --git a/packages/bytes/README.md b/packages/bytes/README.md index 54bcdfb4c0..f1d6ded665 100644 --- a/packages/bytes/README.md +++ b/packages/bytes/README.md @@ -26,8 +26,8 @@ import { bytesFromText } from '@endo/bytes/from-string.js'; import { bytesToText } from '@endo/bytes/to-string.js'; import { concatBytes } from '@endo/bytes/concat.js'; import { bytesEqual } from '@endo/bytes/equals.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; +// The immutable byte utilities live in @endo/immutable-arraybuffer. +import { frozenBytes, thawedBytes } from '@endo/immutable-arraybuffer'; const a = bytesFromText('Hello, '); const b = bytesFromText('world!'); @@ -36,10 +36,12 @@ bytesToText(greeting); // 'Hello, world!' bytesEqual(bytesFromText('abc'), bytesFromText('abc')); // true -// Wrap a Uint8Array in a passable, immutable ArrayBuffer. -const passable = bytesToImmutable(greeting); -// Recover a working Uint8Array from an immutable buffer received over a vat boundary. -bytesToText(bytesFromImmutable(passable)); // 'Hello, world!' +// Wrap a Uint8Array as a passable, frozen Uint8Array backed by an +// immutable ArrayBuffer. +const passable = frozenBytes(greeting); +// Recover a working mutable Uint8Array from a passable received over a +// vat boundary. +bytesToText(thawedBytes(passable)); // 'Hello, world!' ``` The package is exported as per-symbol subpath modules so that callers @@ -66,24 +68,23 @@ Encodes a string as UTF-8 bytes. Decodes UTF-8 bytes to a string. -### `bytesToImmutable(view) -> ArrayBuffer` +### `concatImmutables(buffers) -> Uint8Array` -Wraps a `Uint8Array` view's contents in an immutable `ArrayBuffer` via -the `ArrayBuffer.prototype.sliceToImmutable` shim -(proposal-immutable-arraybuffer). -The result carries the `'byteArray'` passStyle and is hardened, so it -is safe to share across vat boundaries. -The view's `byteOffset` and `byteLength` are honored, so `subarray` -windows copy only the addressed bytes. +Concatenates a list of byteArray-passable values (or bare +`ArrayBufferLike`s) into a single hardened frozen `Uint8Array` backed +by an immutable `ArrayBuffer`. Equivalent to +`frozenBytes(concatBytes(buffers.map(thawedBytes)))`, provided as a +single-call helper because that composition is common when assembling +protocol records from immutable byte fragments. -### `bytesFromImmutable(buffer) -> Uint8Array` +### `frozenBytes` and `thawedBytes` (in `@endo/immutable-arraybuffer`) -Copies the contents of an immutable `ArrayBuffer` into a fresh, -mutable `Uint8Array`. -Immutable `ArrayBuffer` instances cannot back a `Uint8Array` view -directly and APIs such as `TextDecoder.decode` reject them; this -helper produces a working `Uint8Array` copy that callers can pass to -those APIs. +The immutable byte utilities `frozenBytes` (wrap a `Uint8Array` view's +contents in a hardened frozen `Uint8Array` backed by an immutable +`ArrayBuffer`) and `thawedBytes` (copy such a value back out into a +fresh mutable `Uint8Array`) are exported from +`@endo/immutable-arraybuffer`, alongside the platform shim that remains +its separate `@endo/immutable-arraybuffer/shim.js` export. ## Out of scope diff --git a/packages/bytes/compare.js b/packages/bytes/compare.js new file mode 100644 index 0000000000..74a818de99 --- /dev/null +++ b/packages/bytes/compare.js @@ -0,0 +1,3 @@ +// @ts-check + +export { compareBytes } from './src/compare.js'; diff --git a/packages/bytes/from-immutable.js b/packages/bytes/from-immutable.js deleted file mode 100644 index 7bb9f08b80..0000000000 --- a/packages/bytes/from-immutable.js +++ /dev/null @@ -1,3 +0,0 @@ -// @ts-check - -export { bytesFromImmutable } from './src/from-immutable.js'; diff --git a/packages/bytes/package.json b/packages/bytes/package.json index 6310ac4e38..099b778f5f 100644 --- a/packages/bytes/package.json +++ b/packages/bytes/package.json @@ -21,13 +21,12 @@ }, "type": "module", "exports": { + "./compare.js": "./compare.js", "./equals.js": "./equals.js", "./from-string.js": "./from-string.js", "./to-string.js": "./to-string.js", "./concat.js": "./concat.js", "./concat-immutables.js": "./concat-immutables.js", - "./from-immutable.js": "./from-immutable.js", - "./to-immutable.js": "./to-immutable.js", "./package.json": "./package.json" }, "scripts": { diff --git a/packages/bytes/src/compare.js b/packages/bytes/src/compare.js new file mode 100644 index 0000000000..853e86b9fb --- /dev/null +++ b/packages/bytes/src/compare.js @@ -0,0 +1,48 @@ +// @ts-check + +import harden from '@endo/harden'; +import { toIndexableUint8 } from './to-indexable-uint8.js'; + +/** + * Compare two byte sequences lexicographically. + * + * Accepts a frozen `Uint8Array` backed by an immutable `ArrayBuffer` + * (the byteArray passable form) or a plain mutable `Uint8Array`. A genuine + * view is compared in place; an emulated `@endo/immutable-arraybuffer` wrapper + * (typed `Uint8Array` but `ArrayBuffer.isView === false`, so `bytes[i]` reads + * `undefined`) is first copied into a mutable `Uint8Array` so that + * integer-indexed comparison works correctly. + * + * Returns a negative number when `left` sorts before `right`, `0` when + * the two sequences are byte-for-byte equal, and a positive number when + * `left` sorts after `right`. When neither sequence is empty and the + * shorter is a prefix of the longer, returns the length difference + * (`leftLength - rightLength`). + * + * @param {Uint8Array} left + * @param {Uint8Array} right + * @returns {number} + */ +export const compareBytes = (left, right) => { + const l = toIndexableUint8(left); + const r = toIndexableUint8(right); + const lLen = l.length; + const rLen = r.length; + const minLen = lLen < rLen ? lLen : rLen; + for (let i = 0; i < minLen; i += 1) { + if (l[i] < r[i]) { + return -1; + } + if (l[i] > r[i]) { + return 1; + } + } + // When one sequence is a prefix of the other, return the length difference + // (`leftLength - rightLength`): a left-prefix-of-right sorts first (negative + // result), a right-prefix-of-left sorts last (positive result). + if (lLen !== rLen) { + return lLen - rLen; + } + return 0; +}; +harden(compareBytes); diff --git a/packages/bytes/src/concat-immutables.js b/packages/bytes/src/concat-immutables.js index 23ce9df153..d6c70d7415 100644 --- a/packages/bytes/src/concat-immutables.js +++ b/packages/bytes/src/concat-immutables.js @@ -1,23 +1,27 @@ // @ts-check import harden from '@endo/harden'; +import { frozenBytes, thawedBytes } from '@endo/immutable-arraybuffer'; -import { bytesFromImmutable } from './from-immutable.js'; -import { bytesToImmutable } from './to-immutable.js'; import { concatBytes } from './concat.js'; /** - * Concatenates a list of immutable `ArrayBuffer` values into a single - * hardened immutable `ArrayBuffer`. + * Concatenates a list of byteArray-passable values into a single hardened + * frozen `Uint8Array` backed by an immutable `ArrayBuffer`. * - * Equivalent to - * `bytesToImmutable(concatBytes(buffers.map(bytesFromImmutable)))`, + * Equivalent to `frozenBytes(concatBytes(buffers.map(thawedBytes)))`, * provided as a single-call helper because the composition is common * when assembling protocol records from immutable byte fragments. * - * @param {ReadonlyArray} buffers - * @returns {ArrayBuffer} + * The input element type is `Uint8Array` — the narrowed byteArray passable + * shape (issue #573). Each element is thawed to a fresh mutable `Uint8Array` + * before concatenation, so both a genuine frozen `Uint8Array` and the emulated + * `@endo/immutable-arraybuffer` wrapper (typed `Uint8Array` but + * `isView === false`) are handled by `thawedBytes`. + * + * @param {ReadonlyArray} buffers + * @returns {Uint8Array} */ export const concatImmutables = buffers => - bytesToImmutable(concatBytes(buffers.map(bytesFromImmutable))); + frozenBytes(concatBytes(buffers.map(thawedBytes))); harden(concatImmutables); diff --git a/packages/bytes/src/concat.js b/packages/bytes/src/concat.js index 523f5b3814..012b0bb6a0 100644 --- a/packages/bytes/src/concat.js +++ b/packages/bytes/src/concat.js @@ -1,14 +1,25 @@ import harden from '@endo/harden'; +import { toIndexableUint8 } from './to-indexable-uint8.js'; /** - * Concatenates a list of `Uint8Array` chunks into a single contiguous - * `Uint8Array`. + * Concatenates a list of byte chunks into a single contiguous `Uint8Array`. + * + * Accepts a mix of plain mutable `Uint8Array` chunks and frozen + * `Uint8Array` chunks backed by an immutable `ArrayBuffer` (the + * byteArray passable form). Genuine views (`ArrayBuffer.isView === true`) — + * mutable or genuine-immutable alike — are read in place with no per-chunk + * copy; an emulated `@endo/immutable-arraybuffer` wrapper (typed `Uint8Array` + * but `isView === false`) is thawed once per chunk so + * `Uint8Array.prototype.set`'s native fast path reads its real bytes. Only + * the output allocation is new for genuine-view inputs. + * * Empty input yields an empty `Uint8Array`. * - * @param {readonly Uint8Array[]} chunks + * @param {ReadonlyArray} inputs * @returns {Uint8Array} */ -export const concatBytes = chunks => { +export const concatBytes = inputs => { + const chunks = inputs.map(toIndexableUint8); let totalLength = 0; for (const chunk of chunks) { totalLength += chunk.length; diff --git a/packages/bytes/src/equals.js b/packages/bytes/src/equals.js index d6fc35ae20..1e74f18bda 100644 --- a/packages/bytes/src/equals.js +++ b/packages/bytes/src/equals.js @@ -1,8 +1,18 @@ +// @ts-check + import harden from '@endo/harden'; +import { toIndexableUint8 } from './to-indexable-uint8.js'; /** - * Compares two `Uint8Array` values byte-for-byte. - * Returns `true` when the two arrays have equal length and equal contents. + * Compares two byte sequences byte-for-byte. + * Returns `true` when the two have equal length and equal contents. + * + * Accepts a frozen `Uint8Array` backed by an immutable `ArrayBuffer` + * (the byteArray passable form) or a plain mutable `Uint8Array`. A genuine + * view is compared in place; an emulated `@endo/immutable-arraybuffer` wrapper + * (typed `Uint8Array` but `ArrayBuffer.isView === false`, so `bytes[i]` reads + * `undefined`) is first thawed into a mutable `Uint8Array` so that + * integer-indexed comparison works correctly. * * @param {Uint8Array} a * @param {Uint8Array} b @@ -12,11 +22,13 @@ export const bytesEqual = (a, b) => { if (a === b) { return true; } - if (a.length !== b.length) { + const l = toIndexableUint8(a); + const r = toIndexableUint8(b); + if (l.length !== r.length) { return false; } - for (let i = 0; i < a.length; i += 1) { - if (a[i] !== b[i]) { + for (let i = 0; i < l.length; i += 1) { + if (l[i] !== r[i]) { return false; } } diff --git a/packages/bytes/src/from-immutable.js b/packages/bytes/src/from-immutable.js deleted file mode 100644 index bee65d0743..0000000000 --- a/packages/bytes/src/from-immutable.js +++ /dev/null @@ -1,23 +0,0 @@ -// @ts-check - -import harden from '@endo/harden'; - -/** - * Copies the contents of an immutable `ArrayBuffer` into a fresh - * mutable `Uint8Array`. - * - * Immutable `ArrayBuffer` instances (proposal-immutable-arraybuffer) - * cannot back a `Uint8Array` view directly, and APIs such as - * `TextDecoder.decode` reject them. This helper produces a working - * `Uint8Array` copy that callers can pass to those APIs. - * - * Accepts any `ArrayBufferLike` so callers do not need to narrow the - * argument before invoking. - * - * @param {ArrayBufferLike} buffer - * @returns {Uint8Array} - */ -export const bytesFromImmutable = buffer => { - return new Uint8Array(buffer.slice(0)); -}; -harden(bytesFromImmutable); diff --git a/packages/bytes/src/to-immutable.js b/packages/bytes/src/to-immutable.js deleted file mode 100644 index 42ea9a476c..0000000000 --- a/packages/bytes/src/to-immutable.js +++ /dev/null @@ -1,30 +0,0 @@ -// @ts-check - -import '@endo/immutable-arraybuffer/shim.js'; -import harden from '@endo/harden'; - -/** - * Wraps a `Uint8Array` view's contents in an immutable `ArrayBuffer`. - * - * Calls the `sliceToImmutable` method installed by - * `@endo/immutable-arraybuffer/shim.js` on `ArrayBuffer.prototype`. - * Importing this module triggers the shim install, so the caller does not - * need to arrange for it separately. The resulting buffer carries the - * `'byteArray'` passStyle and is safe to share across vat boundaries. The - * result is hardened so it is passable. - * - * Honors the view's `byteOffset` and `byteLength`, so passing a - * `subarray` copies only that window. - * - * @param {Uint8Array} view - * @returns {ArrayBuffer} A hardened immutable `ArrayBuffer`. - */ -export const bytesToImmutable = view => { - const buffer = /** @type {ArrayBuffer} */ (view.buffer); - const immutable = buffer.sliceToImmutable( - view.byteOffset, - view.byteOffset + view.byteLength, - ); - return harden(immutable); -}; -harden(bytesToImmutable); diff --git a/packages/bytes/src/to-indexable-uint8.js b/packages/bytes/src/to-indexable-uint8.js new file mode 100644 index 0000000000..e97948b673 --- /dev/null +++ b/packages/bytes/src/to-indexable-uint8.js @@ -0,0 +1,50 @@ +// @ts-check + +const { isView } = ArrayBuffer; + +/** + * Normalize a byteArray `Uint8Array` to one whose real bytes can be read in + * place — by integer index (`bytes[i]`) and as a `Uint8Array.prototype.set` + * source. + * + * Shared by `./compare.js`, `./concat.js`, and `./equals.js`, which all consume + * the narrowed byteArray passable shape (issue #573): a byteArray is always a + * whole-buffer-spanning `Uint8Array`, never a bare `ArrayBufferLike` nor some + * other `ArrayBufferView`. The runtime `isView` branch below is *not* type + * generality — it is tolerance for a single emulation infidelity: the emulated + * frozen byteArray produced by the `@endo/immutable-arraybuffer` shim is *typed* + * `Uint8Array` yet is a plain ordinary object that reports `isView === false` + * and is *not* integer-indexable (`wrapper[i]` reads `undefined`, and `set`'s + * native fast path reads zeros through it), so it must first be copied into a + * fresh mutable `Uint8Array`. + * + * `ArrayBuffer.isView` is the single committed genuine-vs-emulated + * distinguisher: a *genuine* `Uint8Array` — whether its backing buffer is + * mutable or a genuine (native, stage-3) immutable buffer — is indexed and read + * in place, so we hand it over without copying (reads and indexed access are + * always permitted on an immutable buffer; only writes are refused). Relying + * exclusively on `isView` is the committed contract: it does *not* consult the + * `ArrayBuffer.prototype.immutable` accessor, which answers a different question + * (immutable-vs-mutable buffer) on which a genuine immutable view and an + * emulated wrapper fall on the *same* side, while `isView` separates them. + * + * This differs from `toDecodable` in `./to-string.js`, which deliberately keys + * on `.immutable` rather than `isView`: `TextDecoder.decode` rejects *every* + * immutable-backed view — genuine-immutable included — so that helper must copy + * a genuine immutable view too, whereas the indexed/`set` readers here read one + * in place. Their triggers and input types differ, so they are not unified. + * + * @param {Uint8Array} input + * @returns {Uint8Array} + */ +export const toIndexableUint8 = input => { + if (isView(input)) { + // A genuine `Uint8Array` (including a genuine view over an immutable + // buffer, and a `Buffer` subclass) is used in place (zero allocation). + return input; + } + // Not a genuine view: the emulated `@endo/immutable-arraybuffer` wrapper, + // whose backing buffer is immutable and non-indexable / unreadable by `set`'s + // fast path. `.slice(0)` copies a fresh mutable array to read from. + return new Uint8Array(/** @type {Uint8Array} */ (input).slice(0)); +}; diff --git a/packages/bytes/src/to-string.js b/packages/bytes/src/to-string.js index 196eb49325..fa5fb4e248 100644 --- a/packages/bytes/src/to-string.js +++ b/packages/bytes/src/to-string.js @@ -10,6 +10,60 @@ import harden from '@endo/harden'; const lenientTextDecoder = new TextDecoder(); const fatalTextDecoder = new TextDecoder('utf-8', { fatal: true }); +const { isView } = ArrayBuffer; + +/** + * Return a `Uint8Array` view or value that `TextDecoder.decode` will + * accept. `TextDecoder.decode` rejects views backed by an immutable + * `ArrayBuffer` (as produced by the `@endo/immutable-arraybuffer` shim + * or a native stage-3 implementation), so we copy into a mutable buffer + * only when necessary. + * + * - Genuine mutable `Uint8Array`: pass through unchanged (zero allocation). + * - Genuine `Uint8Array` view over an immutable `ArrayBuffer`: copy once + * into a fresh mutable buffer before passing to `TextDecoder`. + * - Emulated frozen `Uint8Array` wrapper (the `@endo/immutable-arraybuffer` + * shim, which reports `ArrayBuffer.isView === false`): thaw its contents + * into a fresh mutable buffer. + * + * @param {Uint8Array} input + * @returns {Uint8Array} + */ +const toDecodable = input => { + // Determine the underlying ArrayBuffer and the byte range. `isView` is the + // genuine-vs-emulated distinguisher (issue #573): a genuine view exposes its + // backing buffer directly; an emulated frozen wrapper is a plain object whose + // `.slice` amplifies the immutable backing to a fresh mutable copy. + let buf; + let byteOffset; + let byteLength; + if (isView(input)) { + buf = /** @type {ArrayBuffer} */ (input.buffer); + byteOffset = input.byteOffset; + byteLength = input.byteLength; + } else { + // Not a genuine view: the emulated `@endo/immutable-arraybuffer` wrapper. + // `.slice(0)` reaches the shim's freezable-TypedArray method, amplifying + // the immutable backing into a fresh mutable `Uint8Array` that + // `TextDecoder` accepts. + return new Uint8Array(/** @type {Uint8Array} */ (input).slice(0)); + } + + // `ArrayBuffer.prototype.immutable` is the presence-check for the + // @endo/immutable-arraybuffer shim (and future native stage-3 impl). + // When the accessor reports `true`, TextDecoder will reject the view, + // so we must produce a mutable copy. + if (/** @type {any} */ (buf).immutable === true) { + return new Uint8Array(buf.slice(byteOffset, byteOffset + byteLength)); + } + + // Mutable buffer: return a Uint8Array view (no copy). + if (byteOffset === 0 && byteLength === buf.byteLength) { + return new Uint8Array(buf); + } + return new Uint8Array(buf, byteOffset, byteLength); +}; + /** * @typedef {object} BytesToTextOptions * @property {boolean} [fatal] When `true`, malformed UTF-8 throws instead of @@ -19,18 +73,26 @@ const fatalTextDecoder = new TextDecoder('utf-8', { fatal: true }); /** * Decodes UTF-8 bytes to a string. * + * Accepts a `Uint8Array` — whether a plain mutable one, a genuine frozen + * view over an immutable `ArrayBuffer` (the byteArray passable form), or an + * emulated frozen wrapper from the `@endo/immutable-arraybuffer` shim. + * Callers do not need to produce a mutable copy before calling this + * function. The copy, when required because `TextDecoder.decode` rejects + * immutable backing buffers, is done internally. + * * Pass `{ fatal: true }` for strict UTF-8 decoding that throws on * invalid input. The default lenient mode substitutes the * Unicode replacement character (U+FFFD) for malformed sequences. * - * @param {Uint8Array} view + * @param {Uint8Array} input * @param {BytesToTextOptions} [options] * @returns {string} */ -export const bytesToText = (view, options = undefined) => { +export const bytesToText = (input, options = undefined) => { + const decodable = toDecodable(input); if (options !== undefined && options.fatal) { - return fatalTextDecoder.decode(view); + return fatalTextDecoder.decode(decodable); } - return lenientTextDecoder.decode(view); + return lenientTextDecoder.decode(decodable); }; harden(bytesToText); diff --git a/packages/bytes/test/main.test.js b/packages/bytes/test/main.test.js index 1025c41067..bc6e0d2af6 100644 --- a/packages/bytes/test/main.test.js +++ b/packages/bytes/test/main.test.js @@ -1,13 +1,27 @@ import test from '@endo/ses-ava/test.js'; import { passStyleOf } from '@endo/pass-style'; +import { frozenBytes, thawedBytes } from '@endo/immutable-arraybuffer'; import { bytesEqual } from '../src/equals.js'; import { bytesFromText } from '../src/from-string.js'; import { bytesToText } from '../src/to-string.js'; +import { compareBytes } from '../src/compare.js'; import { concatBytes } from '../src/concat.js'; import { concatImmutables } from '../src/concat-immutables.js'; -import { bytesToImmutable } from '../src/to-immutable.js'; -import { bytesFromImmutable } from '../src/from-immutable.js'; + +// Under a native immutable ArrayBuffer implementation (e.g. current XS), the +// `@endo/immutable-arraybuffer` shim steps aside (stage-3 detect-then-skip), so +// `frozenBytes(...)` yields a GENUINE view: `ArrayBuffer.isView === true` and +// integer-indexable in place. The two emulated-wrapper fidelity assertions +// below (isView false; direct integer-indexed read `undefined`) describe the +// shim path specifically, so they are gated to run under the shim and skip +// under native — rather than baking in the (now obsolete) assumption that no +// engine ships native support. See endojs/endo-but-for-bots#475 (erights +// review). The `compareBytes`/`bytesEqual` byte-value assertions elsewhere in +// this file hold on both paths and stay unguarded. +const emulatedOnlyTest = ArrayBuffer.isView(frozenBytes(new Uint8Array([0]))) + ? test.skip + : test; test('concatBytes: empty input yields empty Uint8Array', t => { const result = concatBytes([]); @@ -146,54 +160,67 @@ test('bytesEqual on bytesFromText output: same input compares equal', t => { t.false(bytesEqual(bytesFromText('abc'), bytesFromText('abd'))); }); -test('bytesToImmutable: returns ArrayBuffer with byteArray passStyle', t => { +test('frozenBytes: returns Uint8Array with byteArray passStyle', t => { const view = new Uint8Array([1, 2, 3, 4, 5]); - const immutable = bytesToImmutable(view); - t.true(immutable instanceof ArrayBuffer); + const immutable = frozenBytes(view); + t.true(immutable instanceof Uint8Array); t.is(immutable.byteLength, 5); - // @ts-expect-error passStyleOf typing infers the wrong type for ArrayBuffer. + // The backing buffer is an immutable ArrayBuffer. + t.true(immutable.buffer instanceof ArrayBuffer); + t.true(/** @type {any} */ (immutable.buffer).immutable); t.is(passStyleOf(immutable), 'byteArray'); }); -test('bytesToImmutable: empty input', t => { - const immutable = bytesToImmutable(new Uint8Array(0)); +test('frozenBytes: empty input', t => { + const immutable = frozenBytes(new Uint8Array(0)); t.is(immutable.byteLength, 0); }); -test('bytesToImmutable: honors subarray byteOffset and byteLength', t => { +test('frozenBytes: honors subarray byteOffset and byteLength', t => { const full = new Uint8Array([0, 1, 2, 3, 4, 5, 6, 7]); const window = full.subarray(2, 6); // [2, 3, 4, 5] - const immutable = bytesToImmutable(window); + const immutable = frozenBytes(window); t.is(immutable.byteLength, 4); - t.deepEqual([...bytesFromImmutable(immutable)], [2, 3, 4, 5]); + t.deepEqual([...thawedBytes(immutable)], [2, 3, 4, 5]); }); -test('bytesFromImmutable: copies bytes into a fresh Uint8Array', t => { +test('thawedBytes: copies bytes into a fresh Uint8Array', t => { const source = new Uint8Array([0, 1, 2, 0xff, 0x80, 0x00, 42, 100]); - const immutable = bytesToImmutable(source); - const result = bytesFromImmutable(immutable); + const immutable = frozenBytes(source); + const result = thawedBytes(immutable); t.true(result instanceof Uint8Array); t.is(result.length, source.length); t.deepEqual([...result], [...source]); }); -test('bytesFromImmutable: empty input', t => { - const immutable = bytesToImmutable(new Uint8Array(0)); - const result = bytesFromImmutable(immutable); +test('thawedBytes: empty input', t => { + const immutable = frozenBytes(new Uint8Array(0)); + const result = thawedBytes(immutable); t.true(result instanceof Uint8Array); t.is(result.length, 0); }); -test('bytesToImmutable + bytesToText composition: UTF-8 round-trip', t => { +test('frozenBytes + bytesToText composition: UTF-8 round-trip', t => { + const original = 'Hello, 你好 \u{1F600}'; + const immutable = frozenBytes(bytesFromText(original)); + t.is(bytesToText(thawedBytes(immutable)), original); +}); + +test('bytesToText: decodes a frozen byteArray directly (no explicit thaw)', t => { const original = 'Hello, 你好 \u{1F600}'; - const immutable = bytesToImmutable(bytesFromText(original)); - t.is(bytesToText(bytesFromImmutable(immutable)), original); + const immutable = frozenBytes(bytesFromText(original)); + // `immutable` is a `Uint8Array` — a genuine immutable view under a native + // stage-3 implementation (`ArrayBuffer.isView === true`), or an emulated + // `@endo/immutable-arraybuffer` wrapper under the shim (`isView === false`). + // bytesToText copies internally as needed for either shape; callers do not + // thaw first. + t.is(bytesToText(immutable), original); }); -test('bytesToImmutable + concatBytes composition: assemble from chunks', t => { +test('frozenBytes + concatBytes composition: assemble from chunks', t => { const parts = ['<', 'test-record', '>'].map(s => bytesFromText(s)); - const combined = bytesToImmutable(concatBytes(parts)); - t.is(bytesToText(bytesFromImmutable(combined)), ''); + const combined = frozenBytes(concatBytes(parts)); + t.is(bytesToText(thawedBytes(combined)), ''); }); test('bytesToText: { fatal: true } accepts valid UTF-8', t => { @@ -222,30 +249,152 @@ test('bytesToText: { fatal: false } also accepts valid UTF-8', t => { t.is(bytesToText(bytes, { fatal: false }), 'plain ASCII'); }); -test('concatImmutables: empty input yields empty immutable buffer', t => { +test('concatImmutables: empty input yields empty immutable Uint8Array', t => { const result = concatImmutables([]); - t.true(result instanceof ArrayBuffer); + t.true(result instanceof Uint8Array); t.is(result.byteLength, 0); - // @ts-expect-error passStyleOf typing infers the wrong type for ArrayBuffer. t.is(passStyleOf(result), 'byteArray'); }); test('concatImmutables: concatenates multiple immutable buffers byte-for-byte', t => { const parts = [ - bytesToImmutable(new Uint8Array([1, 2, 3])), - bytesToImmutable(new Uint8Array([])), - bytesToImmutable(new Uint8Array([4])), - bytesToImmutable(new Uint8Array([5, 6, 7, 8])), + frozenBytes(new Uint8Array([1, 2, 3])), + frozenBytes(new Uint8Array([])), + frozenBytes(new Uint8Array([4])), + frozenBytes(new Uint8Array([5, 6, 7, 8])), ]; const result = concatImmutables(parts); t.is(result.byteLength, 8); - t.deepEqual([...bytesFromImmutable(result)], [1, 2, 3, 4, 5, 6, 7, 8]); - // @ts-expect-error passStyleOf typing infers the wrong type for ArrayBuffer. + t.deepEqual([...thawedBytes(result)], [1, 2, 3, 4, 5, 6, 7, 8]); t.is(passStyleOf(result), 'byteArray'); }); test('concatImmutables: result is hardened', t => { - const parts = [bytesToImmutable(new Uint8Array([42]))]; + const parts = [frozenBytes(new Uint8Array([42]))]; const result = concatImmutables(parts); t.true(Object.isFrozen(result)); }); + +// --------------------------------------------------------------------------- +// The emulated-vs-genuine distinguisher this package depends on: +// `ArrayBuffer.isView`. +// +// An emulated freezable `Uint8Array` produced by `@endo/immutable-arraybuffer` +// is a plain ordinary object, so `ArrayBuffer.isView(wrapper) === false`, +// whereas a genuine `Uint8Array` reports `true`. This is the single committed +// emulated-vs-genuine fidelity loss (see that package's README "The one +// committed fidelity loss: an emulated wrapper is not `ArrayBuffer.isView`"), +// and `compareBytes` gates its copy decision on it: it indexes a genuine view +// in place and copies a non-view (emulated) wrapper into a genuine mutable +// `Uint8Array` first. These tests catch a silent breakage on this (client) +// side: if `compareBytes` ever indexed a non-view wrapper directly it would +// read `undefined` for every position and report all inputs as equal. +// +// `wrapper[i] === undefined` is a real but incidental consequence of the +// wrapper's plain-object shape (the same nature that makes `isView` false), +// not the committed distinguisher; the second test records it as a companion +// observation. The shim-side mirror of the committed contract is pinned in +// `@endo/immutable-arraybuffer`'s `test/shim-typedarray.test.js`. +// --------------------------------------------------------------------------- + +emulatedOnlyTest('emulated byteArray wrapper is not ArrayBuffer.isView; a genuine Uint8Array is', t => { + const wrapper = frozenBytes(new Uint8Array([1, 2, 3])); + // The committed distinguisher `compareBytes` leans on. + t.false(ArrayBuffer.isView(wrapper)); + t.true(ArrayBuffer.isView(new Uint8Array([1, 2, 3]))); +}); + +emulatedOnlyTest('emulated byteArray wrapper: direct integer-indexed read is undefined (incidental)', t => { + const wrapper = frozenBytes(new Uint8Array([1, 2, 3])); + // Not the byte: the wrapper carries no integer-indexed own properties and + // the shim installs no read-through getter. The static type says `number` + // (the narrowed `Uint8Array`), but the emulated wrapper answers `undefined` + // at runtime; cast through `unknown` so the assertion type-checks. + t.is(/** @type {unknown} */ (wrapper[0]), undefined); + t.is(/** @type {unknown} */ (wrapper[2]), undefined); +}); + +test('compareBytes: orders emulated byteArray wrappers by their real bytes', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const b = frozenBytes(new Uint8Array([1, 2, 4])); + const aAgain = frozenBytes(new Uint8Array([1, 2, 3])); + + // A negative/positive/zero triple that is only reachable if `compareBytes` + // reads the real bytes. Were it to index the wrapper directly (reading + // `undefined` everywhere) every comparison would collapse to 0. + t.true(compareBytes(a, b) < 0); + t.true(compareBytes(b, a) > 0); + t.is(compareBytes(a, aAgain), 0); + + // Prefix: shorter sorts before longer. + const abcd = frozenBytes(new Uint8Array([1, 2, 3, 4])); + t.true(compareBytes(a, abcd) < 0); +}); + +test('compareBytes: emulated wrapper against a genuine mutable Uint8Array', t => { + const emulated = frozenBytes(new Uint8Array([1, 2, 3])); + const genuine = new Uint8Array([1, 2, 4]); + t.true(compareBytes(emulated, genuine) < 0); + t.true(compareBytes(genuine, emulated) > 0); +}); + +// The same emulated-vs-genuine hazard for `bytesEqual`. Were it to index a +// non-view wrapper directly it would read `undefined` at every position, so +// two distinct equal-length wrappers would collapse to `undefined !== +// undefined` (false) and compare *equal*, while an emulated-vs-genuine pair +// would compare *unequal*. These tests only pass if `bytesEqual` reads the +// real bytes (thawing the wrapper first). + +test('bytesEqual: distinct emulated byteArray wrappers with different bytes are unequal', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const b = frozenBytes(new Uint8Array([1, 2, 4])); + t.false(bytesEqual(a, b)); +}); + +test('bytesEqual: distinct emulated byteArray wrappers with equal bytes are equal', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const aAgain = frozenBytes(new Uint8Array([1, 2, 3])); + t.true(bytesEqual(a, aAgain)); +}); + +test('bytesEqual: emulated wrapper against an equal genuine mutable Uint8Array', t => { + const emulated = frozenBytes(new Uint8Array([1, 2, 3])); + const genuine = new Uint8Array([1, 2, 3]); + t.true(bytesEqual(emulated, genuine)); + t.true(bytesEqual(genuine, emulated)); +}); + +test('bytesEqual: emulated wrapper against an unequal genuine mutable Uint8Array', t => { + const emulated = frozenBytes(new Uint8Array([1, 2, 3])); + const genuine = new Uint8Array([1, 2, 4]); + t.false(bytesEqual(emulated, genuine)); + t.false(bytesEqual(genuine, emulated)); +}); + +test('bytesEqual: emulated wrappers of different lengths are unequal', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const abcd = frozenBytes(new Uint8Array([1, 2, 3, 4])); + t.false(bytesEqual(a, abcd)); +}); + +// The same emulated-vs-genuine hazard for `concatBytes`. A non-view wrapper +// handed to `Uint8Array.prototype.set` as a source would be read through +// `set`'s native fast path, which sees the wrapper's plain-object shape and +// copies zeros — silently dropping the real bytes. `concatBytes` therefore +// relies on the identical `isView` gate as `compareBytes`/`bytesEqual`, +// thawing a non-view wrapper before assembly. These tests only pass if the +// real bytes survive the concat. + +test('concatBytes: assembles emulated byteArray wrappers by their real bytes', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const b = frozenBytes(new Uint8Array([4, 5])); + const result = concatBytes([a, b]); + t.deepEqual([...result], [1, 2, 3, 4, 5]); +}); + +test('concatBytes: mixes emulated wrappers with genuine mutable chunks', t => { + const emulated = frozenBytes(new Uint8Array([1, 2, 3])); + const genuine = new Uint8Array([4, 5, 6]); + t.deepEqual([...concatBytes([emulated, genuine])], [1, 2, 3, 4, 5, 6]); + t.deepEqual([...concatBytes([genuine, emulated])], [4, 5, 6, 1, 2, 3]); +}); diff --git a/packages/bytes/to-immutable.js b/packages/bytes/to-immutable.js deleted file mode 100644 index a9e7c30e45..0000000000 --- a/packages/bytes/to-immutable.js +++ /dev/null @@ -1,3 +0,0 @@ -// @ts-check - -export { bytesToImmutable } from './src/to-immutable.js'; diff --git a/packages/harden/make-hardener.js b/packages/harden/make-hardener.js index 3fe6ede26a..70f400ff99 100644 --- a/packages/harden/make-hardener.js +++ b/packages/harden/make-hardener.js @@ -263,7 +263,19 @@ assert(getTypedArrayToStringTag); // Exported for tests. /** - * Duplicates packages/marshal/src/helpers/passStyle-helpers.js to avoid a dependency. + * Duplicates packages/pass-style/src/passStyle-helpers.js to avoid a dependency. + * + * Deliberately a genuine TypedArray brand check via the `%TypedArray%` + * `[Symbol.toStringTag]` getter, NOT `ArrayBuffer.isView`. Both are + * unspoofable internal-slot checks, but `isView` is also true for a + * `DataView`, whereas only a TypedArray is an integer-indexed exotic whose + * permanently-writable indexed elements make `Object.freeze` throw. That + * freeze-throw is the sole reason `harden` special-cases here (see + * `freezeTypedArray`); a `DataView` freezes normally and must take the + * ordinary `freeze` path, so the DataView-inclusive `isView` would be the + * wrong, less-precise test. (`byteArray.js` commits to `isView` for a + * different question — emulated-vs-native shape on an already-known + * `Uint8Array` — where DataViews are already excluded.) * * @param {unknown} object */ diff --git a/packages/hex/src/encode.js b/packages/hex/src/encode.js index 1bb3790e67..e7ba981c96 100644 --- a/packages/hex/src/encode.js +++ b/packages/hex/src/encode.js @@ -19,18 +19,32 @@ const hexAlphabet = '0123456789abcdef'; * Emits lowercase hex. Callers that need uppercase can call * `.toUpperCase()` on the result. * - * @param {Uint8Array} bytes + * Accepts a frozen `Uint8Array` backed by an immutable `ArrayBuffer` + * (the byteArray passable form) without an intermediate copy. The + * polyfill uses `for...of` iteration rather than integer-indexed access + * because the `@endo/immutable-arraybuffer` shim wraps frozen TypedArrays + * as plain objects whose `[Symbol.iterator]` delegates to the underlying + * genuine TypedArray (via `amplifyTypedArray`) while integer-indexed + * access (`bytes[i]`) returns `undefined` — the wrapper carries no + * integer-indexed own properties and is not an exotic integer-indexed + * object. + * + * @param {Uint8Array} input * @returns {string} */ -export const jsEncodeHex = bytes => { - // Pre-allocate the output array to avoid quadratic-time string - // concatenation on large inputs. - const chars = new Array(bytes.length * 2); - for (let i = 0; i < bytes.length; i += 1) { - const b = bytes[i]; - const j = i * 2; +export const jsEncodeHex = input => { + // `for...of` reads the bytes whether `input` is a genuine `Uint8Array` + // (native iterator) or an emulated `@endo/immutable-arraybuffer` frozen + // wrapper, whose `[Symbol.iterator]` delegates to the hidden genuine + // TypedArray via `amplifyTypedArray`. Integer-indexed access (`input[i]`) + // is avoided because the emulated wrapper reads `undefined` for it; `.length` + // is delegated correctly by the shim wrapper. + const chars = new Array(input.length * 2); + let j = 0; + for (const b of input) { chars[j] = hexAlphabet[b >>> 4]; chars[j + 1] = hexAlphabet[b & 0x0f]; + j += 2; } return chars.join(''); }; @@ -45,16 +59,43 @@ const nativeToHex = typeof toHex === 'function' ? /** @type {() => string} */ (toHex) : undefined; /** - * Encodes a Uint8Array as a lowercase hex string. + * Encodes bytes as a lowercase hex string. + * + * Accepts a `Uint8Array` (the byteArray passable form): a plain mutable one, + * a genuine frozen view over an immutable `ArrayBuffer`, or an emulated + * `@endo/immutable-arraybuffer` wrapper — without an expensive intermediate + * copy. * * Dispatches to the native `Uint8Array.prototype.toHex` intrinsic when - * available (stage-4 TC39 proposal-arraybuffer-base64). Otherwise - * falls through to the pure-JavaScript polyfill. + * available (stage-4 TC39 proposal-arraybuffer-base64) and the input's + * backing buffer is mutable. For frozen `Uint8Array` values backed by an + * immutable `ArrayBuffer` (byteArray passable form) and for all other + * non-plain-`Uint8Array` inputs, the call falls through to the pure-JavaScript + * polyfill. The polyfill uses `for...of` iteration, which the + * `@endo/immutable-arraybuffer` shim delegates correctly via its + * `[Symbol.iterator]` → `amplifyTypedArray` path, producing the correct + * bytes without an intermediate copy. * - * @type {typeof jsEncodeHex} + * @param {Uint8Array} input + * @returns {string} */ export const encodeHex = nativeToHex !== undefined - ? bytes => apply(nativeToHex, bytes, []) + ? input => { + // Use the native intrinsic only when the backing buffer is mutable. + // For immutable ArrayBuffers (shim or native stage-3), the frozen + // Uint8Array wrapper is a plain object that the native C++ toHex + // cannot handle: it reads via internal TypedArray exotic slots, not + // through the wrapper's delegated accessors. The polyfill's for...of + // path (via the shim's [Symbol.iterator] → amplifyTypedArray delegate) + // works correctly without a copy. + if ( + input instanceof Uint8Array && + /** @type {any} */ (input.buffer).immutable !== true + ) { + return apply(nativeToHex, input, []); + } + return jsEncodeHex(input); + } : jsEncodeHex; harden(encodeHex); diff --git a/packages/immutable-arraybuffer/README.md b/packages/immutable-arraybuffer/README.md index 4a23633569..0361202e1c 100644 --- a/packages/immutable-arraybuffer/README.md +++ b/packages/immutable-arraybuffer/README.md @@ -4,6 +4,27 @@ This `@endo/immutable-arraybuffer` package provides a shim for a proposed new Ja A shim modifies the existing JavaScript primordials as needed to most closely emulate the feature as proposed. Importing `@endo/immutable-arraybuffer/shim.js` will cause these changes. +The package's main entry point additionally exports two byte utilities +that pair with the shim: + +```js +import { frozenBytes, thawedBytes } from '@endo/immutable-arraybuffer'; + +// Wrap a Uint8Array's contents in a hardened frozen Uint8Array backed by +// an immutable ArrayBuffer (a `'byteArray'` passable). Honors the view's +// byteOffset/byteLength. +const passable = frozenBytes(new Uint8Array([1, 2, 3])); + +// Copy such a value (or any view / ArrayBufferLike) back out into a fresh +// mutable Uint8Array so APIs like TextDecoder.decode can consume it. +const mutable = thawedBytes(passable); +``` + +Importing the main entry installs the shim as a side effect, since +`frozenBytes` depends on it. The bare shim install remains available as +the separate `@endo/immutable-arraybuffer/shim.js` export for callers +that want only the platform changes. + Below, we use the term "buffer" to refer informally to an instance of an `ArrayBuffer`, whether immutable or not. ## Background @@ -72,13 +93,167 @@ See [Platform support for `transferToImmutable`](#platform-support-for-transfert Without either, the shim still shims `ArrayBuffer.prototype.sliceToImmutable` but omits `ArrayBuffer.prototype.transferToImmutable`. - The shim's emulated immutable buffers are not real `ArrayBuffer` exotic objects. If they were, the shim would not be able to protect them from being written. -Even though they implement the full proposed `ArrayBuffer` API, they cannot be plug-compatible: they cannot be used as the backing stores of `DataView`s or `TypedArray`s. -Perhaps follow-on shims might modify `DataView` and `TypedArray` to emulate that as well, but that is hard and beyond the ambition of this shim. +Even though they implement the full proposed `ArrayBuffer` API, they cannot be plug-compatible as direct exotic-object arguments to all native APIs. +The freezable `TypedArray` emulation (see below) extends the shim to cover the most common consumer path (`new T(iab)`), but `DataView` construction from an emulated immutable buffer is not yet covered. - Unlike genuine `ArrayBuffer` or `SharedArrayBuffer` exotic objects, the shim's emulated immutable buffers cannot be cloned or transfered between JS threads. - This is a plain *JavaScript* shim, not by itself a *Hardened JavaScript* polyfill/shim. Thus, the objects and function it creates are not hardened by this shim itself. Rather, the ses-shim is expected to import this, and then treat the resulting objects as if they were additional primordials, to be hardened during `lockdown`'s harden phase. +## The Freezable TypedArray Emulation + +The shim also installs a freezable `TypedArray` emulation alongside the `ArrayBuffer`-side install. +After the shim loads, constructing a `TypedArray` from an emulated immutable `ArrayBuffer` produces an emulated freezable wrapper: + +```js +import '@endo/immutable-arraybuffer/shim.js'; + +const ab = new ArrayBuffer(4); +const iab = ab.sliceToImmutable(); + +const view = new Uint8Array(iab); + +view instanceof Uint8Array; // true +Object.getPrototypeOf(view) === Uint8Array.prototype; // true (no intermediate prototype) +view.buffer === iab; // true (returns the immutable wrapper) +view.byteLength; // 4 +view.at(0); // 0 + +view.fill(1); // throws TypeError (mutator blocked) +view.set([1]); // throws TypeError +view.reverse(); // throws TypeError +view.copyWithin(0, 1); // throws TypeError +view.sort(); // throws TypeError + +Object.freeze(view); +Object.isFrozen(view); // true +``` + +The emulation covers all eleven concrete `TypedArray` constructors (`Int8Array`, `Int16Array`, `Int32Array`, `Uint8Array`, `Uint8ClampedArray`, `Uint16Array`, `Uint32Array`, `Float32Array`, `Float64Array`, `BigInt64Array`, `BigUint64Array`). + +Constructing from a genuine mutable `ArrayBuffer` produces a genuine writable `TypedArray` view, unchanged from before: + +```js +const mutableAb = new ArrayBuffer(4); +const view = new Uint8Array(mutableAb); +view.fill(1); // succeeds: genuine writable view +``` + +### Indexed assignment on emulated freezable views + +The emulated wrapper is a plain ordinary object, not a native integer-indexed exotic. +An indexed assignment (`view[0] = 42`) therefore creates an own property on the wrapper rather than writing to the underlying buffer. +The underlying buffer's bytes are never touched. + +On a non-frozen wrapper the own property shadows the prototype's read delegate; `view[0]` reads back `42` while `Uint8Array.prototype.at.call(view, 0)` still reads `0` (the underlying byte). +On a frozen wrapper the assignment throws `TypeError` in strict mode (ES module code is always strict), and the buffer is unchanged. + +This is a known constraint of the TC39 proposal: there is no way to intercept integer-indexed assignments on a plain object via the prototype chain. + +### The one committed fidelity loss: an emulated wrapper is not `ArrayBuffer.isView` + +The shim commits to exactly **one** emulated-vs-genuine fidelity loss, and it is the single distinguisher its clients are entitled to rely on: an emulated freezable wrapper is **not** an `ArrayBuffer` view. + +```js +import '@endo/immutable-arraybuffer/shim.js'; + +const iab = new ArrayBuffer(4).sliceToImmutable(); +const emulated = new Uint8Array(iab); +const genuine = new Uint8Array(4); + +ArrayBuffer.isView(emulated); // false (the committed fidelity loss) +ArrayBuffer.isView(genuine); // true +``` + +An emulated wrapper is a plain ordinary object (`Object.create(Uint8Array.prototype)`) with no `[[ViewedArrayBuffer]]` / `[[TypedArrayName]]` internal slots, so `ArrayBuffer.isView` reports `false` for it. +A genuine `Uint8Array` — whether backed by a mutable buffer or, on a native stage-3 engine, by a genuine immutable buffer — reports `true` and is integer-indexable in place. +This is the axis clients use to tell the two apart, and the one the shim promises to preserve: + +- `@endo/pass-style` recognises the `byteArray` pass style by discriminating on `ArrayBuffer.isView`: a non-view (emulated) wrapper must have **no** own integer-indexed properties, a genuine view exactly `length`-many that match the buffer. +- The `thawedBytes` utility (exported from this package) and `@endo/bytes`' `compareBytes` read a genuine view in place and copy a non-view (emulated) wrapper into a genuine mutable `Uint8Array` first, gating that decision on `ArrayBuffer.isView`. + +The shim **must not** be "improved" in any way that would make an emulated wrapper report `ArrayBuffer.isView === true` (it cannot, short of making the wrapper a genuine exotic) or that would otherwise blur this distinction — for example by materializing own integer-indexed data properties on the wrapper (which would push it out of the zero-own-index emulated shape `@endo/pass-style` requires). +`test/shim-typedarray.test.js` pins `ArrayBuffer.isView(wrapper) === false` (and `true` for a genuine view) as the committed regression, and the client packages mirror it with their own tests, so it is both the shim's responsibility not to break clients and each client's responsibility not to be silently broken. + +The `immutable` accessor on the view's `.buffer` is **not** this distinguisher. +It distinguishes an **immutable** buffer from a **mutable** one; it does *not* distinguish a **genuine** immutable view from an **emulated** one. +Both an emulated wrapper and a (future-native) genuine view backed by an immutable buffer report `.buffer.immutable === true`, so the accessor cannot tell them apart — it answers a question about the *buffer's* mutability, not about the *view's* provenance. +`ArrayBuffer.isView` is the axis that separates them. + +### Integer-indexed reads on emulated freezable views (an incidental consequence) + +Symmetric to the assignment constraint above, an integer-indexed *read* on an emulated freezable wrapper does **not** reach the underlying byte. +On a fresh wrapper `view[i]` evaluates to `undefined`, never the byte stored in the immutable buffer. +The bytes are readable only through the integer-indexed *protocol* — `view.at(i)`, `Uint8Array.prototype.at.call(view, i)`, `for..of`, spread — which the shim redirects to the wrapper's hidden genuine `TypedArray`. +A direct `view[i]` reads an ordinary property off a plain object: the wrapper carries no own indexed properties, its `[[Prototype]]` is `Uint8Array.prototype`, and (per the same TC39-proposal constraint) the shim installs no integer-indexed read accessor on `%TypedArray%.prototype` that could intercept `view[i]`. + +This `view[i] === undefined` behavior is a real but **incidental** consequence of the wrapper being a plain object — the same plain-object nature that makes `ArrayBuffer.isView` report `false`. +It is **not** a separately committed fidelity loss and clients do **not** sniff `view[i]` to tell an emulated wrapper from a genuine view; `ArrayBuffer.isView` (above) is the committed distinguisher for that. +Clients simply route *around* the incidental behavior — `@endo/bytes` copies a non-view wrapper before indexing, so it never observes `view[i]` on one — rather than depending on it as a brand. +The shim should nonetheless keep the wrapper an ordinary plain object with no own integer-indexed slots, because that is what keeps it a non-view with zero own indexed properties (the emulated shape `@endo/pass-style` requires); `test/shim-typedarray.test.js` records `view[i] === undefined` as a companion observation to the committed `isView` pin. + +### `[Symbol.toStringTag]` on emulated views (repaired by a getter wrapper) + +The shim replaces the genuine `%TypedArrayPrototype%[Symbol.toStringTag]` getter with a wrapper around it, so an emulated freezable wrapper reports the same string tag as a genuine view. + +The genuine `%TypedArrayPrototype%[Symbol.toStringTag]` getter is `this`-sensitive: it reads the receiver's `[[TypedArrayName]]` internal slot. +An emulated wrapper is a plain ordinary object (`Object.create(Uint8Array.prototype)`) with no such slot, so the *unmodified* getter would return `undefined` for it and `Object.prototype.toString.call(view)` would read `'[object Object]'`. +The shim's replacement getter closes that gap: on an emulated wrapper it amplifies to the hidden genuine `TypedArray` and reads *its* internal-slot tag; on a genuine `TypedArray` it falls through to the captured genuine getter; on any other receiver it returns `undefined`, exactly as the genuine getter does. + +```js +import '@endo/immutable-arraybuffer/shim.js'; + +const iab = new ArrayBuffer(4).sliceToImmutable(); +const emulated = new Uint8Array(iab); +const genuine = new Uint8Array(4); + +Object.prototype.toString.call(emulated); // '[object Uint8Array]' (repaired) +Object.prototype.toString.call(genuine); // '[object Uint8Array]' +``` + +This is a **getter-wrapper** fix, not a `[Symbol.toStringTag]` **data property** on the wrapper. +A data property would patch only the `Object.prototype.toString` lookup path while leaving the genuine `this`-sensitive getter still reporting `undefined` on a wrapper; the getter wrapper makes the getter and `Object.prototype.toString` agree. +The wrapper therefore still carries **no** own `[Symbol.toStringTag]` — the tag comes from the prototype getter. + +Consequently `[Symbol.toStringTag]` is **no longer** an emulated-vs-genuine distinguisher; the single committed distinguisher remains `ArrayBuffer.isView` (above). +One downstream consequence worth naming: a brand check that captures *this* (shim-installed) getter as an internal-slot probe — as `@endo/harden`'s `isTypedArray` does — will now classify an emulated wrapper as a `TypedArray` *if it captures the getter after the shim installs*. +That reroutes such a wrapper through `harden`'s `freezeTypedArray` branch rather than the ordinary `Object.freeze` branch, which is benign: the wrapper carries no own integer-indexed properties, so `freezeTypedArray` reduces to `preventExtensions` plus a no-op over an empty own-key set, and `harden(wrapper)` succeeds either way. +`test/shim-typedarray-tostringtag.test.js` pins the repaired `'[object Uint8Array]'` reading and the getter-wrapper (not data-property) shape. + +## Function expressions versus declarations + +Throughout `src/lib.js`, exported bindings that hold function values use `const` +with a named function expression rather than `function` declarations. +The reason is JavaScript function-declaration hoisting. + +In the presence of an import cycle, a hoisted `function` declaration's value is +accessible to an importing module before the exporting module finishes +initializing. +An early importer that reads the exported name gets the function value already +present (because hoisting put it there before the module body ran), but any +other module-level state the function closes over may not yet be initialized, +creating a subtle hazard. + +By using `const` instead, the JavaScript standard specifies that an early +importer in a cycle that reads the name before the exporting module's +initializer runs would get a Temporal Dead Zone (TDZ) error, making the hazard +visible at runtime rather than silent. + +Two implementation notes for ses-shim environments: + +- The ses-shim's compiler from JS ESM module code to JS evaluable code does not + correctly implement TDZ. + A cycle hazard of this kind may therefore not be caught at runtime when + running under the ses-shim. +- XS (Moddable's engine) uses native compartment and module support and does + implement TDZ correctly, so the same hazard would be caught at runtime on XS. + +This file introduces no such import cycle; the convention is documented here +so future maintainers understand why function declarations are avoided throughout +endo. + +Source: erights review comment 3439479281 on `src/lib.js` line 578. + ## Platform support for `transferToImmutable` The shim's emulation of `ArrayBuffer.prototype.transferToImmutable` requires the underlying platform to provide either `ArrayBuffer.prototype.transfer` (preferred when present) or the global `structuredClone` (used as a fallback to move the buffer's contents into a new backing store). diff --git a/packages/immutable-arraybuffer/designs/README.md b/packages/immutable-arraybuffer/designs/README.md new file mode 100644 index 0000000000..8081b75005 --- /dev/null +++ b/packages/immutable-arraybuffer/designs/README.md @@ -0,0 +1,40 @@ +# Designs for `@endo/immutable-arraybuffer` + +Composite design documents for the `@endo/immutable-arraybuffer` package. +Each file captures one design topic; new design topics get their own +file rather than extending an existing one. + +## Index + +- [`immutable-arraybuffer.md`](immutable-arraybuffer.md): + the drop-the-pseudo-prototype reshape of the ArrayBuffer-side + emulation. + Establishes the amplifier-with-this-fallthrough pattern, the + lib-as-property-record shape, the consolidated `lib.js` file + topology, and the stage-3 detect-then-skip install policy. + Renamed from the package-rooted `DESIGN.md` on this branch. +- [`freezable-typedarray.md`](freezable-typedarray.md): + the TypedArray-side analog explicitly named in + `immutable-arraybuffer.md` section *Out of scope*. + Brings the same drop-the-pseudo-prototype reshape to the eleven + concrete `TypedArray` constructors so a `Uint8Array` backed by an + emulated immutable `ArrayBuffer` is frozen and immutable at the + JavaScript surface. + Depends on the ArrayBuffer-side reshape having merged first. + +## Conventions + +- File names are kebab-case slugs of the topic. + No `DESIGN-` prefix on the basename; the `designs/` directory carries + that role. +- Each design carries a *Status* table near the top with `Created`, + `Authors`, `Status` (Proposed / Accepted / Implemented / Superseded), + and `Depends` / `Affects` / `Replaces` rows where applicable. +- Cross-references between designs in this directory use relative paths + (for instance, `freezable-typedarray.md` references + `immutable-arraybuffer.md`, not the full + `packages/immutable-arraybuffer/designs/immutable-arraybuffer.md` + path). +- Cross-references from package source (under `src/`, `test/`) to a + design in this directory use the package-rooted path + (`designs/immutable-arraybuffer.md`). diff --git a/packages/immutable-arraybuffer/designs/freezable-typedarray.md b/packages/immutable-arraybuffer/designs/freezable-typedarray.md new file mode 100644 index 0000000000..f763cd7d6f --- /dev/null +++ b/packages/immutable-arraybuffer/designs/freezable-typedarray.md @@ -0,0 +1,1050 @@ +# Freezable TypedArray emulation: drop the pseudo-prototype on the TypedArray side + +This design captures the *delayed freezable TypedArray emulation* +that erights asked for in his 2026-06-17T10:55Z comment on PR #435. +PR #435 is the predecessor that drops the immutable-ArrayBuffer +pseudo-prototype. +This is the TypedArray-side analog explicitly named in PR #435's +`designs/immutable-arraybuffer.md` section *Out of scope* +(quoted: "The TypedArray-side analog (drop +`%FreezableTypedArrayPrototype%` similarly). +Separate PR, separate design."). + +The package keeps its split between a self-contained library layer +(`src/lib.js`) and a shim layer (`src/shim.js`) that installs +emulation onto genuine prototypes at load time. +The TypedArray side mirrors the ArrayBuffer-side amplifier-with-this-fallthrough +shape PR #435 established: every method on `%TypedArrayPrototype%` +discriminates on brand-WeakMap membership, the emulated wrapper +inherits directly from the genuine prototype (no intermediate +pseudo-prototype), and the shim installs the lib's property record +onto the genuine prototype under a stage-3 detect-then-skip policy. + +## Status + +| Field | Value | +| -------- | ------------------------------------------------------------------------------------------------ | +| Created | 2026-06-17 | +| Authors | erights (original framing), kriscendobot (write-up) | +| Status | Proposed | +| Depends | PR #435 (drop-the-pseudo-prototype on the ArrayBuffer side) must merge before the builder fires | +| Affects | `packages/immutable-arraybuffer/`, `packages/ses/src/permits.js` | +| Replaces | The would-be `%FreezableTypedArrayPrototype%` intrinsic that the experiment branch introduced | + +## Problem + +The *Immutable ArrayBuffer* proposal at TC39 (Stage 2.7, advanced +February 2025) carries an explicit guarantee: +*A `DataView` or `TypedArray` using an immutable buffer as its backing +store can be frozen and immutable.* +PR #435 lands the ArrayBuffer-side emulation under the +drop-the-pseudo-prototype shape; this proposal carries the parallel +guarantee for `TypedArray` instances backed by emulated immutable +ArrayBuffers. + +After PR #435 merges, a caller can do this: + +```js +import '@endo/immutable-arraybuffer/shim.js'; + +const ab = new ArrayBuffer(4); +const iab = ab.sliceToImmutable(); // emulated immutable AB +const view = new Uint8Array(iab); // currently throws or + // silently produces a + // wrong-shape view +``` + +Without the freezable-TypedArray emulation, the `new Uint8Array(iab)` +call would fall into one of two unwanted states: + +- **Throws.** + The native `Uint8Array` constructor expects a real `ArrayBuffer` + exotic object as its first argument. + The emulated immutable buffer is a plain object whose `__proto__` + is `ArrayBuffer.prototype`; the spec's internal slot check rejects it. +- **Coerces silently.** + Some engines treat the emulated immutable buffer as a buffer-like + and copy out a degraded view whose `.buffer` is a fresh genuine + ArrayBuffer disconnected from the immutable wrapper. + The view is then mutable, breaking the proposal's + *can-be-frozen-and-immutable* guarantee at the TypedArray surface. + +The proposal's TypedArray guarantee therefore cannot land at the +JavaScript surface without a TypedArray-side shim. +The experiment branch `experiment/no-spackle-immutable-arraybuffer-417` +(prototype, not for merge) demonstrates the pattern; PR #435 fixes +the ArrayBuffer-side surface this design builds on; this PR brings +the TypedArray-side surface to parity. + +## Background + +The freezable-TypedArray design extends the post-#435 lib surface, +not the experiment branch's earlier shape. +A reader meeting this document without having read +`designs/immutable-arraybuffer.md` first needs the following lib-side +topology before the *Implementation outline* section makes sense. + +After PR #435 merges, the lib (`packages/immutable-arraybuffer/src/lib.js`) +owns two internal WeakMaps that the freezable-TypedArray code extends +rather than reintroduces: + +- `hiddenBuffers` maps each emulated immutable ArrayBuffer wrapper to + its backing genuine (mutable) ArrayBuffer. + The lib uses the wrapper as the public-facing identity and the + genuine buffer as the private storage; methods that need to read + bytes (`slice`, `getInt8`, etc.) consult `hiddenBuffers` to recover + the genuine buffer. +- `reverseHiddenBuffers` is the inverse map from genuine backing + buffer to the wrapper. + Methods that need to *return* a buffer (the `view.buffer` getter, + for instance) consult `reverseHiddenBuffers` to hand back the + immutable wrapper rather than the genuine buffer. + +Both WeakMaps live inside the lib's module scope; they are not +exported. +The freezable-TypedArray code adds a third WeakMap (`hiddenTypedArrays`) +keyed on the emulated TypedArray wrappers and reads the two +pre-existing WeakMaps for `view.buffer` lookups. +This is the topology *Implementation outline* section *Lib additions* extends; +that section names `hiddenBuffers` and `reverseHiddenBuffers` without +re-explaining them. + +A reader who wants to see the post-#435 lib in detail (the +amplifier-with-this-fallthrough pattern, the lib-as-property-record +shape, the brand-WeakMap discrimination) should read +`designs/immutable-arraybuffer.md` section *Move 2* first; this design assumes +that surface as a given. + +## API surface + +After this PR merges, the following hold for any concrete TypedArray +constructor `T` in the standard library's eleven concrete TypedArray +constructors (`Int8Array`, `Int16Array`, `Int32Array`, `Uint8Array`, +`Uint8ClampedArray`, `Uint16Array`, `Uint32Array`, `Float32Array`, +`Float64Array`, `BigInt64Array`, `BigUint64Array`): + +```js +import '@endo/immutable-arraybuffer/shim.js'; + +const ab = new ArrayBuffer(4); +const iab = ab.sliceToImmutable(); +const view = new T(iab); +``` + +| Expression | Returns | +| ----------------------------------------- | ------------------------------------------------------------- | +| `view instanceof T` | `true` | +| `Object.getPrototypeOf(view)` | `T.prototype` (no intermediate prototype) | +| `view.buffer` | `iab` (the immutable wrapper, not the underlying genuine AB) | +| `view.byteLength`, `byteOffset`, `length` | correct values, delegated to the hidden genuine TypedArray | +| `view.at(0)`, `slice`, `subarray`, etc. | correct values, delegated to the hidden genuine TypedArray | +| `view.set([1])` | throws `TypeError` (complaining mutator) | +| `view.fill(0)`, `reverse`, `sort`, `copyWithin` | each throws `TypeError` | +| `view[0] = 42; view.at(0)` | `0` (the underlying buffer is never modified; `view.at(0)` delegates to the hidden genuine TypedArray and reads the actual buffer byte; see *Semantics* for the full worked example) | +| `Object.freeze(view); Object.isFrozen(view)` | `true` | + +The non-emulated path (construction from a genuine mutable +ArrayBuffer) is unchanged: + +```js +const realAb = new ArrayBuffer(4); +const view = new T(realAb); + +// view is a genuine TypedArray view. +// Mutators succeed; indexed assignment writes through; .buffer === realAb. +``` + +The pseudo-constructor is a drop-in replacement for `T`: the +emulated-immutable branch is reached only when the first argument is +a hidden buffer (registered in the lib's `hiddenBuffers` WeakMap); +every other call shape falls through to the genuine constructor via +`Reflect.construct(OriginalConstructor, args, new.target)`. + +The constructor surface is symmetric (both `new T(iab)` and +`new T(realAb)` parse and complete without error), but the +*result-of-construction* surface is asymmetric: the resulting views +diverge on mutability. +A reader of a single call site like `new Uint8Array(maybeIab)` cannot +tell from the syntax whether the produced view will throw on +`.set(...)` or write through; only the runtime identity of the +argument decides. +This is the proposal's central trade: the constructor accepts both +shapes uniformly so existing TypedArray-construction code at consumer +sites does not have to branch, and the call site's mutator behavior +is determined by the argument's immutability rather than by a +separate constructor name. + +## Semantics + +Three semantic choices warrant explicit treatment. + +### Mutator methods throw + +The five enumerated mutator methods (`copyWithin`, `fill`, `reverse`, +`set`, `sort`) each `throw TypeError` when invoked on an emulated +freezable TypedArray. +This matches the *Immutable ArrayBuffer* proposal's guarantee that an +immutable-backed view is immutable: a mutator that observably +modifies the contents must be prevented, and a thrown `TypeError` is +the explicit failure mode the proposal text uses. + +The throw is implemented at the lib level via the +amplifier-with-this-fallthrough pattern: each mutator on the lib's +property record checks brand-WeakMap membership and throws on hit; on +miss it delegates to the captured genuine method, which preserves +unchanged behaviour for genuine TypedArrays. + +### Indexed assignment never modifies the underlying buffer + +The proposal does not provide a way to make integer-indexed +assignment to a TypedArray *throw*. +The emulated wrapper's response to `view[0] = 42` is therefore +necessarily different from the genuine integer-indexed exotic's +silent-swallow path; the wrapper is a plain ordinary object whose +`[[Prototype]]` is `T.prototype`, not an integer-indexed exotic +object. +What this design guarantees is the *immutability of the underlying +buffer*: no path through the emulated wrapper can mutate the bytes +the immutable `ArrayBuffer` holds. +What `view[0]` reads back after `view[0] = 42` depends on whether the +wrapper itself has been frozen, and is independent of the buffer's +contents. + +#### Worked example (non-frozen wrapper) + +```js +import '@endo/immutable-arraybuffer/shim.js'; + +const ab = new ArrayBuffer(4); // [0, 0, 0, 0] +const iab = ab.sliceToImmutable(); +const view = new Uint8Array(iab); + +view[0]; // 0 (delegates to the hidden + // genuine TypedArray's read + // of the immutable buffer's + // byte 0) +view[0] = 42; // OrdinarySet on the plain + // wrapper; creates an own + // data property '0' => 42. + // The underlying immutable + // buffer is NOT touched. +view[0]; // 42 (now reads the own + // property, which shadows + // the prototype's indexed + // read delegate) + +Uint8Array.prototype.at.call(view, 0); // 0 (the buffer's actual + // byte 0 is unchanged) +``` + +The own-property creation is a quirk of the plain-object wrapper, not +a security concern: the immutable buffer's bytes are untouched, and +any code that observes the bytes via a non-indexed method (`at`, +`slice`, `subarray`, the DataView accessors, byte enumeration through +`for ... of`) sees the original buffer contents. +The discrepancy is only visible to code that reads through the +wrapper's integer-indexed surface after an integer-indexed write, +and that surface is exactly the surface the proposal cannot prevent +the write to. + +#### Worked example (frozen wrapper) + +```js +const view = new Uint8Array(iab); +Object.freeze(view); +Object.isFrozen(view); // true + +view[0] = 42; // silently swallowed in + // non-strict mode; throws + // TypeError in strict mode + // (own property '0' cannot + // be created on a frozen + // object) +view[0]; // undefined (no own property; + // the prototype has no + // integer-indexed slot) +``` + +After `Object.freeze(view)`, the wrapper rejects new own-property +installation per the ordinary frozen-object semantics; the +integer-indexed write fails to create an own property and the +subsequent read falls through the prototype chain to find no slot, +returning `undefined`. + +The experiment branch carries coverage for the post-freeze case +(the "strengthened indexed-assignment swallow test" in fixup +`740259d2`); this design preserves that coverage and adds the +non-frozen-wrapper worked example as a new test +(`shim: indexed assignment creates a wrapper-local own property on a +non-frozen emulated freezable view; underlying buffer unchanged`). + +This is a known proposal-level constraint, not a shim shortcoming. +The README's *Caveats* section is updated to mention both the +non-frozen and frozen cases. + +### `Object.isFrozen(view)` returns true after `Object.freeze(view)` + +The emulated wrapper has no integer-indexed exotic slots and no +non-configurable own data properties, so `Object.freeze(view)` +succeeds and `Object.isFrozen(view)` returns `true`. +This is the proposal's TypedArray-can-be-frozen guarantee at the +JavaScript surface. + +For a genuine TypedArray on a mutable buffer, `Object.freeze` throws +because the integer-indexed slots are non-configurable accessor-like +slots backed by the buffer; the emulated wrapper has neither of those +properties, so `freeze` is well-defined. + +The spec basis: `Object.freeze` invokes `SetIntegrityLevel` on the +receiver, which iterates the receiver's *own* property keys (via +`[[OwnPropertyKeys]]`) and sets each to non-configurable. +The integer-indexed exotic check that makes genuine TypedArrays +unfreezable lives on the integer-indexed exotic object's +`[[OwnPropertyKeys]]` and `[[DefineOwnProperty]]` internal methods, +which enumerate the integer-indexed slots as own properties. +The emulated wrapper is a plain ordinary object whose `[[Prototype]]` +is `T.prototype`; its own `[[OwnPropertyKeys]]` (the ordinary-object +form) does not enumerate integer-indexed slots because the wrapper +has none. +The freeze walk therefore touches only the wrapper's plain own +properties (none) and completes; the prototype chain's exotic-ness is +not consulted because freeze operates on the receiver. + +The harden phase of SES `lockdown()` reaches every primordial and +freezes it transitively; the emulated wrappers participate normally +in that walk because they are plain objects. + +### `view.buffer` returns the immutable wrapper + +The lib installs `virtualTypedArrayBufferGetter` as the new accessor +for `%TypedArrayPrototype%.buffer`. +The getter checks `hiddenTypedArrays` for the receiver; on hit it +returns the immutable wrapper (`reverseHiddenBuffers.get(genuineAb)`); +on miss it returns the genuine buffer the way the native accessor +would. +This means a caller who does `view.buffer.sliceToImmutable()` on an +emulated freezable view gets the immutable wrapper back, consistent +with the rest of the proposal's surface. + +The same getter therefore serves both genuine TypedArrays and +emulated freezable TypedArrays, so the shim install replaces the +prototype's `buffer` accessor unconditionally (under the stage-3 +detect-then-skip gate). + +### `[Symbol.toStringTag]` + +The shim **replaces** the genuine `%TypedArrayPrototype%[Symbol.toStringTag]` +getter with a wrapper around it, so an emulated freezable TypedArray wrapper +reports the same string tag as a genuine view. +That genuine getter is `this`-sensitive — it reads the receiver's +`[[TypedArrayName]]` internal slot — and an emulated wrapper is a plain +ordinary object (`Object.create(T.prototype)`) with no such slot, so the +*unmodified* getter would return `undefined` for it and +`Object.prototype.toString.call(view)` would read `'[object Object]'` rather +than `'[object Uint8Array]'` (or the concrete flavor's name). +The shim's replacement getter closes that gap using the same +amplifier-with-fallthrough shape as the `buffer` / `byteLength` / +`byteOffset` / `length` accessors: on an emulated wrapper it amplifies to the +hidden genuine TypedArray (`hiddenTypedArrays.get(receiver)`) and reads *its* +internal-slot tag; on a genuine TypedArray it delegates to the captured +genuine getter; on any other receiver it returns `undefined`, exactly as the +genuine getter does. + +This is a **getter-wrapper** fix, not a `[Symbol.toStringTag]` **data +property** on the wrapper. A data property would repair only the +`Object.prototype.toString` lookup path while leaving the genuine +`this`-sensitive getter still reporting `undefined` on a wrapper (a *flawed* +fidelity fix); the getter wrapper makes the getter and +`Object.prototype.toString` agree. The wrapper therefore still carries **no** +own `[Symbol.toStringTag]` — the tag comes from the prototype getter. + +Consequently `[Symbol.toStringTag]` is **no longer** an emulated-vs-genuine +distinguisher; the single committed distinguisher remains `ArrayBuffer.isView`. +One downstream consequence: a brand check that captures *this* (shim-installed) +getter as an internal-slot probe — as `@endo/harden`'s `isTypedArray` does — +will classify an emulated wrapper as a `TypedArray` when it captures the getter +after the shim installs, rerouting the wrapper through `harden`'s +`freezeTypedArray` branch. That reroute is benign: the wrapper carries no own +integer-indexed properties, so `freezeTypedArray` reduces to +`preventExtensions` plus a no-op over an empty own-key set, and +`harden(wrapper)` succeeds in either capture order (verified empirically). +`test/shim-typedarray-tostringtag.test.js` pins the repaired +`'[object Uint8Array]'` reading and the getter-wrapper (not data-property) +shape. + +**Reversal of the earlier "defer to the genuine tag" decision.** An earlier +revision of this design deliberately did *not* replace the getter, following +erights's call on +[PR #449's open question 3](https://github.com/endojs/endo-but-for-bots/issues/comments/4735477238) +(*"(b) is best. … I'm happy not to add complexity to avoid it until we find out +if it is an actual problem"*) and treating the `'[object Object]'` reading as an +incidental, un-depended-upon fidelity loss. erights subsequently asked for the +receiver-aware getter as a higher-fidelity fix and to land it as a separately +reviewable commit to see what it does and does not break +([#475 review comments 3817252816 / 3817264546](https://github.com/endojs/endo-but-for-bots/pull/475)). +This section records that reversal; the getter wrapper is that commit. +It parallels PR #435's ArrayBuffer-side post-departure recovery (which installed +`'ImmutableArrayBuffer'` as an own-property tag on each emulated immutable +buffer), differing in mechanism — a receiver-aware getter rather than an own +data property, which is why the emulated view's tag stays faithful to *this* +receiver rather than being pinned to a single literal. + +The experiment branch set `[Symbol.toStringTag] = 'FreezableTypedArray'` on the +would-be intermediate prototype; under the drop-the-pseudo-prototype shape there +is no intermediate prototype to hang a tag on, and this fix supplies the flavor- +faithful tag through the prototype getter instead. + +## Implementation outline + +The implementation is the post-#435 reshape of the experiment branch's +freezable-TypedArray commits (`721c68a3`, `2097641c`, `cfe99f7e`, +`e02ec0d0`, `1ef6c174`, plus four review-response fixups). + +### Files added or modified + +| File | Action | Notes | +| ------------------------------------------------------------------- | ------- | --------------------------------------------------------------------- | +| `packages/immutable-arraybuffer/src/lib.js` | EDIT | extend with the freezable-TypedArray surface (see *Lib additions*) | +| `packages/immutable-arraybuffer/src/shim.js` | EDIT | extend the shim to also install the freezable-TypedArray surface | +| `packages/immutable-arraybuffer/test/lib-typedarray.test.js` | NEW | lib-level unit tests (translated from `freezable-typedarray-pony.test.js`) | +| `packages/immutable-arraybuffer/test/shim-typedarray.test.js` | NEW | shim-level integration tests (translated from `freezable-typedarray-shim.test.js`) | +| `packages/immutable-arraybuffer/test/shim-typedarray-per-flavor.test.js` | NEW | per-flavor parameterized coverage across all eleven concrete TypedArray constructors | +| `packages/immutable-arraybuffer/README.md` | EDIT | new section "The Freezable TypedArray Emulation"; retire the "follow-on shims might modify `DataView` and `TypedArray`" caveat | +| `packages/immutable-arraybuffer/designs/freezable-typedarray.md` | NEW | this file | +| `packages/ses/src/permits.js` | EDIT | see *permits.js delta* sub-section below | +| `packages/ses/test/immutable-arraybuffer.test.js` | EDIT | extend to cover the freezable-TypedArray case (a `Uint8Array` constructed from an immutable AB is frozen / immutable after lockdown) | +| `.changeset/freezable-typedarray-emulation.md` | NEW | minor on `@endo/immutable-arraybuffer`; patch on `ses` | + +#### permits.js delta + +The current `%TypedArrayPrototype%` entry in `packages/ses/src/permits.js` (on +`master` at `4a04d078b`) already contains a `buffer: getter` permit: + +```js +'%TypedArrayPrototype%': { + buffer: getter, + byteLength: getter, + byteOffset: getter, + constructor: '%TypedArray%', + copyWithin: fn, + // ... (fill, filter, find, findIndex, forEach, includes, indexOf, + // join, keys, lastIndexOf, length, map, reduce, reduceRight, + // reverse, set, slice, some, sort, subarray, toLocaleString, + // toString, values, @@iterator, @@toStringTag, at, findLast, + // findLastIndex, toReversed, toSorted, with) +}, +``` + +The shim replaces the native `%TypedArrayPrototype%.buffer` accessor with +`virtualTypedArrayBufferGetter` (the discriminating accessor the lib +installs). +Because the shim's replacement accessor is itself a getter, the SES +permits walk does not see a new property kind: the slot was `getter` +before and remains `getter` after the shim's install. +No new permit row is required; the existing `buffer: getter` entry +covers the shim-installed replacement without modification. + +The five mutator methods (`copyWithin`, `fill`, `reverse`, `set`, `sort`) +already appear as `fn` entries in the same `%TypedArrayPrototype%` +entry and remain `fn` after the shim installs the amplifier-with-this- +fallthrough property record. +Their permit shape does not change. + +Therefore the only edit to `permits.js` this design requires is none of +the kind the critic's question anticipated (no new row, no row type +change). +The EDIT action in the table above is present because the test +`packages/ses/test/immutable-arraybuffer.test.js` (a sibling edit) will +exercise the permits walk against the shim-installed slots; if that test +surfaces an unexpected gap the builder patches the permits entry at that +time. +The expected outcome is that no gap surfaces: the existing `getter` and +`fn` entries cover the shim's replacements. + +This design does **not** introduce a new ses-side intrinsic. +Under the drop-the-pseudo-prototype shape the emulated wrappers +inherit directly from the genuine `T.prototype`, so +`get-anonymous-intrinsics.js` does not need a new sample. +This is the parallel to PR #435's deletion of the +`%ImmutableArrayBufferPrototype%` sample. + +### Lib additions + +The lib gains four exported bindings (in addition to whatever PR #435 +leaves as the post-merge exports): + +```js +// In src/lib.js, after the existing ArrayBuffer-side property record: + +const hiddenTypedArrays = new WeakMap(); + +export const amplifyTypedArray = typedArray => + apply(weakmapGet, hiddenTypedArrays, [typedArray]) || typedArray; + +export const virtualTypedArrayBufferGetter = /* getter that consults + hiddenTypedArrays first, then walks via FERAL_GET_ARRAY_BUFFER and + reverseHiddenBuffers to return the immutable wrapper for hidden + cases and the genuine buffer for fallthrough */; + +export const makePseudoTypedArrayConstructor = OriginalConstructor => + /* returns a constructor that delegates to OriginalConstructor on + non-hidden-buffer args, and produces an emulated wrapper on + hidden-buffer args */; + +export const freezableTypedArrayLibProperties = /* property record + the shim copies onto %TypedArrayPrototype%; contains the mutator + throw / read delegate methods, plus the `buffer` accessor + replacement */; +``` + +The `freezableTypedArrayLibProperties` record bundles two +semantically distinct concerns under one install loop for shim-side +simplicity, not because they are the same kind of property: + +- The mutator-throws descriptors (`copyWithin`, `fill`, `reverse`, + `set`, `sort`): discriminate on `hiddenTypedArrays` brand + membership and throw on hit; on miss, delegate to the captured + genuine method (the *amplifier-with-this-fallthrough* shape). +- The `buffer` accessor replacement: discriminate on the same brand + WeakMap but with a different fallthrough semantic. + On hit, return the immutable wrapper via `reverseHiddenBuffers`; + on miss, return the genuine buffer the native accessor would have + returned. + +The two share the brand WeakMap and the install loop but answer +different questions (throw-versus-delegate for mutators, redirect- +versus-passthrough for the buffer getter). +The bundling is an install-loop economy, not a category claim. + +The internal `hiddenBuffers` and `reverseHiddenBuffers` WeakMaps +(see *Background* above) remain owned by the immutable-ArrayBuffer +side of the post-#435 lib; +the freezable-TypedArray code reads them from the lib's existing +module-internal scope. +On post-#435 master, the immutable-ArrayBuffer side already lives +inside the consolidated `lib.js` (the experiment branch's separate +`immutable-arraybuffer-pony-internal.js` file does not survive the +merge), so this design extends a single `lib.js` and does not +reintroduce an internal file split. + +The experiment branch carries an `internal-heir.js` helper (a 100+ +line "intermediate prototype with redirect + complain semantics" +builder) that does not exist on post-#435 master. +Under the drop-the-pseudo-prototype shape there is no intermediate +prototype to build; the helper's role is taken by the property +record copied onto `T.prototype`. +The builder therefore does not port the helper; the design needs no +property-record-building utility beyond what `lib.js` exports +directly. + +### Shim additions + +`src/shim.js` extends the existing detect-then-skip install body: + +```js +// In src/shim.js, inside the existing detect-then-skip block: + +if (!('sliceToImmutable' in arrayBufferPrototype)) { + // ... existing ArrayBuffer-side install from PR #435 ... + + // New: freezable-TypedArray install. + const TypedArray = getPrototypeOf(Uint8Array); + const { prototype: typedArrayPrototype } = TypedArray; + + defineProperties( + typedArrayPrototype, + getOwnPropertyDescriptors(freezableTypedArrayLibProperties), + ); + + // Replace each of the eleven concrete global TypedArray constructors + // with the pseudo-constructor produced from the lib. + for (const { name, Ctor } of concreteTypedArrayCtors) { + defineProperty(globalThis, name, { + value: makePseudoTypedArrayConstructor(Ctor), + writable: true, + enumerable: false, + configurable: true, + }); + } +} +``` + +The stage-3 detect-then-skip gate is shared: if a prior shim or a +native implementation has already installed the ArrayBuffer-side +`sliceToImmutable`, this shim's entire install body is skipped, +including the TypedArray-side additions. +The two sides ship as a unit because they are part of the same TC39 +proposal. + +### Diagram + +```mermaid +flowchart LR + subgraph "After this PR" + direction TB + EW[Emulated freezable TypedArray wrapper] -->|__proto__| TP[T.prototype] + LIB[Lib property record - freezableTypedArrayLibProperties] -.->|copied by shim| TPP["%TypedArrayPrototype%"] + TPP -->|methods discriminate via| WM[hiddenTypedArrays brand WeakMap] + EW -.->|registered in| WM + BG[virtualTypedArrayBufferGetter] -.->|replaces| BUF["%TypedArrayPrototype%.buffer accessor"] + BG -->|consults| WM + BG -->|on hit, consults| RHB[reverseHiddenBuffers WeakMap] + RHB -->|owned by| ABLIB[ArrayBuffer-side lib from #435] + end +``` + +## Test plan + +The implementation must pass three test layers. + +### Lib-level (the property-record + pseudo-constructor in isolation) + +`packages/immutable-arraybuffer/test/lib-typedarray.test.js` +(translation of the experiment branch's +`freezable-typedarray-pony.test.js`, four tests): + +- `makePseudoTypedArrayConstructor wraps an immutable ArrayBuffer`: + the brand-check WeakMap registration succeeds; the + `virtualTypedArrayBufferGetter` recovers the immutable wrapper. +- `makePseudoTypedArrayConstructor forwards a non-immutable first arg`: + the fallthrough branch via `Reflect.construct(OriginalConstructor, + args, new.target)` produces a genuine TypedArray view. +- `virtualTypedArrayBufferGetter returns the real buffer for a + genuine TypedArray`: the fallthrough returns the genuine buffer. +- `virtualTypedArrayBufferGetter redirects to the immutable wrapper + when present`: the redirect via `reverseHiddenBuffers` works. + +### Shim-level (after `import '../src/shim.js'`) + +`packages/immutable-arraybuffer/test/shim-typedarray.test.js` +(translation of the experiment branch's +`freezable-typedarray-shim.test.js`, eight tests): + +- `shim: global Uint8Array on an immutable ArrayBuffer wraps as + emulated freezable`. +- `shim: global Uint8Array on a regular ArrayBuffer forwards to the + OriginalConstructor`. +- `shim: virtual buffer getter returns the real buffer for a genuine + TypedArray`. +- `shim: virtual buffer getter redirects to the immutable wrapper + when present`. +- `shim: emulated freezable byteLength and at redirect via + amplifyTypedArray`. +- `shim: emulated freezable mutators complain` (each of the five + enumerated mutators throws). +- `shim: emulated freezable subarray returns a view whose buffer is + the immutable wrapper`. +- `shim: detect-then-skip is idempotent under re-import` (parallel + to PR #435's gate behaviour). + +Additional tests this PR introduces beyond the experiment branch: + +- `shim: indexed assignment on a non-frozen emulated freezable view + creates a wrapper-local own property; the underlying immutable + buffer is unchanged` and `shim: indexed assignment on a frozen + emulated freezable view is silently swallowed; the underlying + immutable buffer is unchanged` (cover both halves of the + proposal-level constraint named in *Semantics* section + *Indexed assignment never modifies the underlying buffer*). +- `shim: Object.freeze(view); Object.isFrozen(view) === true` on an + emulated freezable view (the proposal's + TypedArray-can-be-frozen guarantee). +- `shim: Object.getPrototypeOf(view) === Uint8Array.prototype` on an + emulated freezable view (documents that no intermediate prototype + exists, parallel to PR #435's analogous assertion). + +### Per-flavor parameterized coverage + +`packages/immutable-arraybuffer/test/shim-typedarray-per-flavor.test.js` +runs a parameterized matrix over all eleven concrete TypedArray +constructors (`Int8Array`, `Int16Array`, `Int32Array`, `Uint8Array`, +`Uint8ClampedArray`, `Uint16Array`, `Uint32Array`, `Float32Array`, +`Float64Array`, `BigInt64Array`, `BigUint64Array`). + +The matrix carries a per-flavor *sample value* for each row. +For the nine non-BigInt flavors the sample is `1`; for the two +BigInt flavors (`BigInt64Array`, `BigUint64Array`) the sample is `1n`. +The mutator and `with` calls must use the per-flavor sample because +the native operations throw `TypeError` on a type mismatch *before* +reaching the brand check, which would mask the test's intent. +Specifically: + +- `view.with(0, sample)` requires `sample === 1n` for the two BigInt + flavors and `sample === 1` for the nine non-BigInt flavors. + `view.with(0, 1)` on a `BigInt64Array` throws `TypeError` + ("Cannot convert a Number value to a BigInt") before the + emulation's mutator-throws path is reached. +- `view.fill(sample)` and `view.set([sample])` carry the same + per-flavor constraint. + Both are *expected* to throw `TypeError` on the emulated freezable + view (the mutator-throws contract), but the test must construct + the argument with the flavor-correct type so that the throw the + test observes is the brand-check throw and not a type-mismatch + throw at the native call site. + +For each flavor, the matrix asserts (with the per-flavor sample +substituted into the parenthesized positions): + +- Construction from an immutable buffer succeeds and yields a + freezable wrapper whose `__proto__` is `T.prototype`. +- Each of the five mutator methods throws `TypeError`: + `view.copyWithin(0, 1)`, `view.fill(sample)`, `view.reverse()`, + `view.set([sample])`, `view.sort()`. +- Indexed assignment does not modify the underlying buffer. + On a non-frozen wrapper: `view[0] = sample; t.is(view[0], sample)` + reads back the own property; the buffer's byte 0 remains the + per-flavor zero (`0` for non-BigInt flavors, `0n` for BigInt + flavors), confirmed via `T.prototype.at.call(view, 0)`. + On a frozen wrapper: `Object.freeze(view); view[0] = sample; + t.is(view[0], undefined)` confirms the silent swallow and the + buffer's byte 0 remains the per-flavor zero. +- `view.byteLength`, `view.byteOffset`, `view.length`, `view.buffer` + all return correct values. +- `view.slice(...)`, `view.subarray(...)`, `view.at(0)`, + `view.with(0, sample)`, `view.toReversed()`, `view.toSorted()` + return correct values. +- `Object.freeze(view); Object.isFrozen(view)` returns `true`. +- The fallthrough constructor (`new T(genuineMutableBuffer)`) still + produces a genuine writable view. + +The eleven-flavor table catches regressions that a `Uint8Array`-only +test suite would miss (the experiment branch covers only +`Uint8Array`). +Naming the per-flavor sample shape explicitly here lets the builder +write the right matrix on the first try rather than rediscovering +the BigInt distinction in a CI run. + +### ses-side integration + +`packages/ses/test/immutable-arraybuffer.test.js` extends to cover: + +- After `lockdown()`, an emulated freezable `Uint8Array` is hardened + and `Object.isFrozen(view) === true`. +- After `lockdown()`, the permits walk does not complain about the + `%TypedArrayPrototype%` slots the shim installs. +- After `lockdown()`, an emulated freezable view's mutator methods + still throw (the harden phase does not break the lib's + discrimination logic). + +### Cross-package consumer touchpoints + +The freezable-TypedArray emulation surfaces an explicit cross-package +risk against `packages/pass-style/src/byteArray.js`. +On post-#435 master, `byteArray.js`'s `confirmCanBeValid` requires +`candidate instanceof ArrayBuffer && candidate.immutable` and +`assertRestValid` requires `getPrototypeOf(candidate) === ArrayBuffer.prototype`. +A `Uint8Array` (genuine or emulated freezable) therefore does **not** +pass the current byte-array brand check. +This is a pre-existing condition of the post-#435 lib, not a +regression this design introduces. + +A separate revision to `byteArray.js` is required for the +freezable-TypedArray emulation to be useful at the pass-style brand +boundary. +Per erights's +[inline comment on this design](https://github.com/endojs/endo-but-for-bots/pull/449#discussion_r3431570369) +(2026-06-17T21:26Z): +*"Also need to revise `packages/pass-style/src/byteArray.js` to use a +frozen Uint8Array rather than a frozen immutable ArrayBuffer as a +byteArray."* +And: +*"Perhaps packages/bytes need a similar revision."* +*"I'll leave that to @kriskowal."* +The `byteArray.js` revision is **out of scope for this PR** (the +design's scope is the immutable-arraybuffer package's freezable- +TypedArray emulation, not pass-style's brand check) and is left to a +follow-up that the maintainer files separately. + +#### Adapter consolidation and withdrawal from `@endo/bytes` + +Per kriskowal's inline comment on this design +(discussion `r3431584143`, 2026-06-17T21:29Z): +*"I believe we will be able to withdraw adapters for frozen Uint8 +arrays backed by frozen immutable ArrayBuffer from `@endo/bytes` as +the shim presents as sufficiently ergonomic without utility +functions. +This does not need to be engaged in the same builder PR."* + +That consolidation has since landed in this PR (see the +`consolidate-immutable-byte-utilities` changeset). `@endo/bytes` +previously carried two adapter functions that bridged between frozen +`Uint8Array` instances backed by frozen immutable `ArrayBuffer`s and +the broader bytes-handling surface; both have been withdrawn from +`@endo/bytes`, consolidated onto the shim's shared implementation, and +renamed: + +- `frozenBytes(view)` (now exported from `@endo/immutable-arraybuffer`; + formerly `bytesToImmutable` at `packages/bytes/src/to-immutable.js`): + wraps a `Uint8Array` view's byte window into a hardened frozen + `Uint8Array` backed by an immutable `ArrayBuffer`. + With this design's shim installed it is the direct-construction + pattern `Object.freeze(new Uint8Array(ab.sliceToImmutable()))` + behind a single name. +- `thawedBytes(buffer)` (now exported from `@endo/immutable-arraybuffer`; + formerly `bytesFromImmutable` at `packages/bytes/src/from-immutable.js`): + copies an immutable `ArrayBuffer`'s contents into a fresh mutable + `Uint8Array` so downstream APIs (such as `TextDecoder.decode`) that + reject immutable buffers can consume the bytes. + It is the fresh-copy pattern `new Uint8Array(immutableAb.slice(0))` + behind a single name. + +Because the shim makes that adapter shape ergonomic at the language +surface, the `@endo/bytes` sub-path exports `./to-immutable.js` and +`./from-immutable.js` were removed rather than re-homed under new +names in the same package: a consumer that wants a frozen `Uint8Array` +backed by an immutable `ArrayBuffer` either constructs it directly via +`new Uint8Array(ab.sliceToImmutable())` and freezes the wrapper, or +reaches for the shared `frozenBytes` / `thawedBytes` exported from +`@endo/immutable-arraybuffer`. + +This consolidation rode the same cross-package consumer sweep this +design describes, confirming no regressions in `@endo/pass-style` or +`@endo/marshal`. + +The implementation PR's consumer sweep therefore expects the +following: + +- `yarn workspace @endo/pass-style test` after the implementation + lands: passes unchanged. + An emulated freezable `Uint8Array` does **not** pass the existing + brand check; pass-style tests do not exercise the freezable-Uint8Array + path and are unaffected. +- `yarn workspace @endo/marshal test` after the implementation lands: + passes unchanged for the same reason. + Marshal's byte-array codec routes through pass-style's + `byteArray` style; without the `byteArray.js` revision, no marshal + test exercises a freezable-Uint8Array round-trip. + +The named regression signals the builder watches for (kinds of CI +failure that would indicate a real regression rather than the +expected no-op): + +- Any `concordance`-routed `Buffer.from` `TypeError` on an emulated + freezable `Uint8Array` (the parallel to PR #435's 13 ocapn-codec + failures named in *Notes from the field*, 2026-06-09). +- Any pass-style brand-check mis-classification on an emulated + freezable `Uint8Array` (the check correctly returns "not a + byteArray"; a different routing is a regression). +- Any marshal codec test failing on a byte-array encode/decode + round-trip whose input is constructed from an emulated freezable + `Uint8Array` (this should not happen because no marshal test + constructs such an input; if one does, the test should be updated + to use the genuine `byteArray` shape rather than the regression + being absorbed silently). + +If the cross-package sweep surfaces any of these named signals, the +builder escalates back to the maintainer rather than installing a +workaround in the freezable-TypedArray PR; the underlying remediation +is the separate `byteArray.js` revision erights describes. + +Per the *Notes from the field* entry in `roles/designer/AGENT.md` +2026-06-09: PR #435's `[Symbol.toStringTag]` decision killed 13 ocapn +codec tests because `concordance` routed through `Buffer.from` on the +`'[object ArrayBuffer]'` tag. +The parallel risk on the TypedArray side is acknowledged by erights's +*"It does have the hazard you mention, but I'm happy not to add +complexity to avoid it until we find out if it is an actual problem"* +on PR #449's open question 3 (resolution recorded in *Decisions* section 3); +the builder runs the same downstream consumer sweep before opening +the implementation PR and, if the sweep surfaces a regression, +escalates back to the maintainer rather than installing the tag +unilaterally. + +## Scope + +### In scope + +- Adding the freezable-TypedArray emulation to `@endo/immutable-arraybuffer` + as a new section of the lib's property record and as a new portion + of the shim install body. +- Replacing each of the eleven concrete global TypedArray + constructors with a pseudo-constructor that handles the + emulated-immutable branch and falls through to the genuine + constructor otherwise. +- Replacing `%TypedArrayPrototype%.buffer` with a getter that + redirects emulated freezable views to the immutable wrapper. +- Extending the SES permits entry for `%TypedArrayPrototype%` to + cover the shim-installed slots. +- Adding lib-level, shim-level, and per-flavor tests. +- Updating the package README to document the new surface and retire + the "follow-on shims might modify `DataView` and `TypedArray`" + caveat. + +### Dependency: PR #435 must merge first + +The builder dispatch for this design **must not fire before PR #435 +merges**. +PR #435 establishes the amplifier-with-this-fallthrough pattern, the +lib-as-property-record shape, the stage-3 detect-then-skip install +policy, and the consolidated `lib.js` file that this design extends. +Building on top of pre-#435 master would either (a) fork the pattern, +producing a TypedArray-side shape that does not match the +ArrayBuffer-side, or (b) require a substantive rebase that rewrites +this PR's contents after #435 merges. + +The designer's dispatch (this document) can fire before #435 merges; +the design is independent of the implementation's exact diff. +The builder's dispatch waits. + +As of this draft's authoring (2026-06-17T20Z), PR #435 has merged +(merge commit `855a8f7bc`); the builder can fire as soon as the +project's frozen-base branch is updated. + +### Out of scope + +- `DataView` emulation. + A parallel "freezable DataView" surface is implied by the same + proposal text ("A `DataView` or `TypedArray` using an immutable + buffer ..."), but it is not part of this design. + DataView's surface is much smaller (one constructor, a handful of + typed accessors), and the same drop-the-pseudo-prototype shape + applies; a separate follow-up PR can land it after this one + validates the pattern on the richer TypedArray surface. +- Subclass support. + The pseudo-constructor throws if `new.target !== PseudoTypedArray` + on the emulated-immutable branch (per the experiment branch's + current code); subclassing an emulated freezable TypedArray is not + supported. + The fallthrough branch supports the standard subclass story for + genuine TypedArrays. +- Cross-realm support. + The lib's WeakMaps are realm-local; an emulated freezable + TypedArray from one realm is not recognised in another realm. + This matches the proposal text (TC39 proposals are realm-local by + default) and the ArrayBuffer-side behaviour. +- A new `%FreezableTypedArrayPrototype%` SES intrinsic. + Explicitly excluded: under the drop-the-pseudo-prototype shape, + emulated wrappers inherit directly from `T.prototype` and no new + intrinsic exists. + The experiment branch's `cfe99f7e` "fixup: partial progress" commit + introduced a 48-line `%FreezableTypedArrayPrototype%` permits + entry; that entry is dropped under this design. +- A `view.freeze()` or `view.toImmutable()` API. + Freezable-TypedArray-ness in this design (and in the proposal) is + constructor-time-determined by the backing buffer's immutability. + There is no API to "freeze later"; the only way to obtain an + emulated freezable TypedArray is to construct it from an emulated + immutable ArrayBuffer. +- Native engine work. + This is a shim layer; native engines must implement the proposal + separately at TC39 Stage 3 advance. + The stage-3 detect-then-skip gate ensures this shim steps aside + when a native implementation is present. + +## Decisions + +Three framing questions on the original draft were resolved by +erights on PR #449 +([issuecomment-4735477238](https://github.com/endojs/endo-but-for-bots/issues/comments/4735477238), +2026-06-17). +This section records the resolutions so a future reader does not have +to reconstruct them from PR thread history. + +### Decision 1: "Delayed" means sequencing of PRs (confirmed) + +erights confirmed the researcher's hypothesis: the "delayed freezable +TypedArray emulation" phrasing is a *sequencing* word, not a +runtime-lazy semantic. +A follow-up PR that *follows* PR #435's merge and *delays* the +TypedArray-side work to its own design and builder cycle is what was +asked for. +Freezable-TypedArray-ness is *constructor-time-determined by the +backing buffer's immutability*; there is no `view.freeze()` or +`view.toImmutable()` API and no runtime detection that flips the +view's mode after construction. +The two alternative readings the researcher ruled out (a lazy +`view.freeze()` / `view.toImmutable()` API; a "delayed install" / +detect-then-skip framing already decided by PR #435) are accordingly +out of scope. + +### Decision 2: Two design files with parallel naming (confirmed) + +erights confirmed the sibling-files shape and asked for parallel +naming. +Both designs now sit at: + +- `packages/immutable-arraybuffer/designs/immutable-arraybuffer.md` + (PR #435's design; renamed from the generic `DESIGN.md` on this + PR's branch as part of this resolution). +- `packages/immutable-arraybuffer/designs/freezable-typedarray.md` + (this design). + +The rename uses `git mv` so the immutable-arraybuffer design's file +history is preserved. +The alternative shape (extending PR #435's `DESIGN.md` with a +*"Phase 2: TypedArray-side"* section) is ruled out: keeping the two +designs in separate files avoids merge conflicts on future +amendments and keeps each document within the *Length: aim for 1 to +3 screens* guideline in `roles/designer/AGENT.md` section *Operating norms*. + +### Decision 3: `[Symbol.toStringTag]`: defer to the genuine tag (confirmed) + +erights chose option (b): *"(b) is best. +It does have the hazard you mention, but I'm happy not to add +complexity to avoid it until we find out if it is an actual +problem."* + +The shim therefore does **not** install +`[Symbol.toStringTag] = 'FreezableTypedArray'` on the emulated +wrapper, and does not replace the genuine `this`-sensitive +`%TypedArrayPrototype%[Symbol.toStringTag]` getter. +Because that getter reads the receiver's `[[TypedArrayName]]` slot, +which the plain-object wrapper lacks, it returns `undefined` for the +wrapper, so `Object.prototype.toString.call(view)` reads as +`'[object Object]'` — a fidelity loss relative to a genuine view's +`'[object Uint8Array]'`. See section *`[Symbol.toStringTag]`* above for +the client-contract consequences and the pin test that guards them. +This deliberately diverges from PR #435's ArrayBuffer-side +post-departure recovery (which installed +`'ImmutableArrayBuffer'` as an own-property on each emulated +immutable buffer). + +The risk acknowledged in erights's reply (a downstream consumer like +`concordance` routing on `'[object Uint8Array]'` and treating it as a +license to mutate or to call `Buffer.from`) is real but not blocking. +The builder runs the same cross-package consumer sweep PR #435 used +(per *Test plan* section *Cross-package consumer touchpoints*, against +`@endo/pass-style` and `@endo/marshal`). +If the sweep surfaces a concrete regression, the builder escalates +back to the maintainer rather than installing the tag unilaterally; +adding the own-property tag is a small, reversible follow-up if it +ever becomes necessary. + +The experiment branch's original shape installs the tag on the +would-be intermediate prototype; that install is dropped during the +post-#435 translation. + +## References + +- [erights's "delayed freezable TypedArray emulation" comment on PR #435](https://github.com/endojs/endo-but-for-bots/pull/435) + (2026-06-17T10:55Z): the framing this document expands. +- [PR #435 `designs/immutable-arraybuffer.md`](https://github.com/endojs/endo-but-for-bots/pull/435/files) + (renamed from `DESIGN.md` on this PR's branch per *Decisions* section 2): + the drop-the-pseudo-prototype shape this design adopts on the + TypedArray side. + Specifically the section *Out of scope* names the work this PR + does (quoting the "TypedArray-side analog (drop + `%FreezableTypedArrayPrototype%` similarly), separate PR, separate + design" clause). +- [erights's resolution of open questions 1, 2, 3 on PR #449](https://github.com/endojs/endo-but-for-bots/issues/comments/4735477238) + (2026-06-17): the comment that pinned the sibling-files shape, the + parallel-naming convention, and option (b) on the + `[Symbol.toStringTag]` decision. + Recorded in *Decisions* above. +- The experiment branch + `experiment/no-spackle-immutable-arraybuffer-417` + (origin remote, head `1ef6c174d` plus four review-response + fixups): the working prototype this PR translates. + Foundational commits: `721c68a3` (initial freezable-typedarray pony + scaffolding), `e02ec0d0` (shim install body), `1ef6c174` + (shim-level tests). +- [TC39 *Immutable ArrayBuffer* proposal](https://github.com/tc39/proposal-immutable-arraybuffer) + (Stage 2.7 as of 2026-06): the proposal text that includes the + "A `DataView` or `TypedArray` using an immutable buffer as its + backing store can be frozen and immutable" guarantee this PR + realises at the shim layer. +- README.md *Caveats* section: the existing caveat "Perhaps follow-on + shims might modify `DataView` and `TypedArray` to emulate that as + well, but that is hard and beyond the ambition of this ponyfill + + shim" is the readme-side anchor; this PR rewrites that caveat to + point at the new section. diff --git a/packages/immutable-arraybuffer/DESIGN.md b/packages/immutable-arraybuffer/designs/immutable-arraybuffer.md similarity index 100% rename from packages/immutable-arraybuffer/DESIGN.md rename to packages/immutable-arraybuffer/designs/immutable-arraybuffer.md diff --git a/packages/immutable-arraybuffer/index.js b/packages/immutable-arraybuffer/index.js new file mode 100644 index 0000000000..56e9bc9c2a --- /dev/null +++ b/packages/immutable-arraybuffer/index.js @@ -0,0 +1,3 @@ +// @ts-check + +export { frozenBytes, thawedBytes } from './src/bytes.js'; diff --git a/packages/immutable-arraybuffer/package.json b/packages/immutable-arraybuffer/package.json index 7c0ea08f85..ff7cb92c90 100644 --- a/packages/immutable-arraybuffer/package.json +++ b/packages/immutable-arraybuffer/package.json @@ -20,7 +20,13 @@ "url": "https://github.com/endojs/endo/issues" }, "type": "module", + "dependencies": { + "@endo/harden": "workspace:^" + }, "exports": { + ".": { + "default": "./index.js" + }, "./shim.js": { "types": "./shim.types.d.ts", "default": "./shim.js" diff --git a/packages/immutable-arraybuffer/src/bytes.js b/packages/immutable-arraybuffer/src/bytes.js new file mode 100644 index 0000000000..6e2f772e3d --- /dev/null +++ b/packages/immutable-arraybuffer/src/bytes.js @@ -0,0 +1,95 @@ +// @ts-check + +import '../shim.js'; +import harden from '@endo/harden'; +import { sliceBufferToImmutable } from './lib.js'; + +const { + ArrayBuffer, + Uint8Array, + // eslint-disable-next-line no-restricted-globals +} = globalThis; + +const { isView } = ArrayBuffer; + +/** + * The shared byte-conversion utilities that pair with the immutable + * `ArrayBuffer` shim: `frozenBytes` produces a hardened frozen + * `Uint8Array` backed by an immutable `ArrayBuffer`, and `thawedBytes` + * copies such a value's contents back out into a fresh mutable + * `Uint8Array`. They are inverses over the byte contents. + * + * Importing this module triggers the `@endo/immutable-arraybuffer/shim.js` + * install (so the emulated freezable `Uint8Array` constructor is present), + * the same side effect the shim export provides on its own. The shim + * remains available as the separate `@endo/immutable-arraybuffer/shim.js` + * export for callers that only want the platform install. + */ + +/** + * Wraps a `Uint8Array` view's contents in a hardened frozen `Uint8Array` + * backed by an immutable `ArrayBuffer`. + * + * Slices the view's window into a fresh immutable `ArrayBuffer`, then wraps + * that in a fresh `Uint8Array` and hardens the wrapper. The resulting + * wrapper carries the `'byteArray'` passStyle and is safe to share across + * vat boundaries. Hardening the wrapper also hardens the underlying + * immutable buffer. + * + * Honors the view's `byteOffset` and `byteLength`, so passing a + * `subarray` copies only that window. + * + * @param {Uint8Array} view + * @returns {Uint8Array} A hardened frozen `Uint8Array` backed by an + * immutable `ArrayBuffer`. + */ +export const frozenBytes = view => { + const immutable = sliceBufferToImmutable( + /** @type {ArrayBuffer} */ (view.buffer), + view.byteOffset, + view.byteOffset + view.byteLength, + ); + return harden(new Uint8Array(immutable)); +}; +harden(frozenBytes); + +/** + * Copies the contents of a `Uint8Array` into a fresh mutable `Uint8Array`. + * + * The hardened frozen `Uint8Array` produced by `frozenBytes` cannot itself + * be written through, and consumers such as `TextDecoder.decode` reject + * views over immutable buffers. This helper produces a working mutable + * `Uint8Array` copy that callers can pass to those APIs. + * + * The parameter type is `Uint8Array` — the narrowed byteArray shape (issue + * #573), never a bare `ArrayBufferLike` nor some other `ArrayBufferView`. The + * runtime `isView` branch is *not* type generality: it distinguishes a genuine + * `Uint8Array` view (including a genuine view over an immutable buffer) from an + * emulated frozen wrapper produced by the shim, which is *typed* `Uint8Array` + * yet reports `isView === false`. The result is always a fresh, mutable + * `Uint8Array`. + * + * @param {Uint8Array} buffer + * @returns {Uint8Array} + */ +export const thawedBytes = buffer => { + if (isView(buffer)) { + // A genuine view (including a genuine view over an immutable buffer): + // copy its window out of the backing buffer. `.slice` here reaches the + // `ArrayBuffer` method — the shim's replacement, which amplifies an + // immutable backing buffer to its mutable copy. + return new Uint8Array( + // eslint-disable-next-line @endo/no-polymorphic-call + buffer.buffer.slice( + buffer.byteOffset, + buffer.byteOffset + buffer.byteLength, + ), + ); + } + // An emulated frozen `Uint8Array` wrapper (typed `Uint8Array` but not + // `ArrayBuffer.isView`), whose `.slice` is the freezable-TypedArray method + // installed by the shim, amplifying the immutable backing to a mutable copy. + // eslint-disable-next-line @endo/no-polymorphic-call + return new Uint8Array(/** @type {Uint8Array} */ (buffer).slice(0)); +}; +harden(thawedBytes); diff --git a/packages/immutable-arraybuffer/src/lib.js b/packages/immutable-arraybuffer/src/lib.js index f3892028d2..147a5b5d2a 100644 --- a/packages/immutable-arraybuffer/src/lib.js +++ b/packages/immutable-arraybuffer/src/lib.js @@ -8,12 +8,28 @@ const { WeakMap, // Capture structuredClone before it can be scuttled. structuredClone: optStructuredClone, + Int8Array, + Int16Array, + Int32Array, + Uint8ClampedArray, + Uint16Array, + Uint32Array, + Float32Array, + Float64Array, + BigInt64Array, + BigUint64Array, // eslint-disable-next-line no-restricted-globals } = globalThis; -const { freeze, defineProperty, getOwnPropertyDescriptor, getPrototypeOf } = - Object; -const { apply, ownKeys } = Reflect; +const { + freeze, + defineProperty, + getOwnPropertyDescriptor, + getPrototypeOf, + setPrototypeOf, + create, +} = Object; +const { apply, ownKeys, construct } = Reflect; // Capture the WeakMap prototype methods up front so we can use them with // `apply` below, without exposing the `buffers` WeakMap to post-hoc @@ -52,14 +68,85 @@ const optArrayBufferMaxByteLength = getOwnPropertyDescriptor( 'maxByteLength', )?.get; +// %TypedArray% is the abstract superclass of all TypedArray constructors. +// `getPrototypeOf(Uint8Array)` reaches it via the constructor's prototype chain. +// Captured before the shim can shadow any of the global TypedArray constructors. +const TypedArray = getPrototypeOf(Uint8Array); + const typedArrayPrototype = getPrototypeOf(Uint8Array.prototype); const { set: uint8ArraySet } = typedArrayPrototype; // @ts-expect-error TS doesn't know it'll be there -const { get: uint8ArrayBuffer } = getOwnPropertyDescriptor( +const { get: typedArrayBufferGetter } = getOwnPropertyDescriptor( typedArrayPrototype, 'buffer', ); +// Capture all %TypedArrayPrototype% methods and accessors before the shim can +// shadow them. The five mutator methods are used for the brand-check throw / +// delegate pattern; all read-only methods are used for the amplifier-delegate +// pattern (plain wrappers delegate to the hidden genuine TypedArray). +const { + copyWithin: typedArrayCopyWithin, + entries: typedArrayEntries, + every: typedArrayEvery, + fill: typedArrayFill, + filter: typedArrayFilter, + find: typedArrayFind, + findIndex: typedArrayFindIndex, + findLast: typedArrayFindLast, + findLastIndex: typedArrayFindLastIndex, + forEach: typedArrayForEach, + includes: typedArrayIncludes, + indexOf: typedArrayIndexOf, + join: typedArrayJoin, + keys: typedArrayKeys, + lastIndexOf: typedArrayLastIndexOf, + map: typedArrayMap, + reduce: typedArrayReduce, + reduceRight: typedArrayReduceRight, + reverse: typedArrayReverse, + set: typedArraySet, + slice: typedArraySlice, + some: typedArraySome, + sort: typedArraySort, + subarray: typedArraySubarray, + toLocaleString: typedArrayToLocaleString, + toString: typedArrayToString, + values: typedArrayValues, + at: typedArrayAt, + toReversed: typedArrayToReversed, + toSorted: typedArrayToSorted, + with: typedArrayWith, +} = typedArrayPrototype; + +// Capture read-accessor getters for byteLength, byteOffset, and length. +// @ts-expect-error TS doesn't know they'll be there +const { get: typedArrayByteLengthGetter } = getOwnPropertyDescriptor( + typedArrayPrototype, + 'byteLength', +); +// @ts-expect-error TS doesn't know they'll be there +const { get: typedArrayByteOffsetGetter } = getOwnPropertyDescriptor( + typedArrayPrototype, + 'byteOffset', +); +// @ts-expect-error TS doesn't know they'll be there +const { get: typedArrayLengthGetter } = getOwnPropertyDescriptor( + typedArrayPrototype, + 'length', +); +// Capture the genuine, `this`-sensitive `[Symbol.toStringTag]` getter. It reads +// the receiver's `[[TypedArrayName]]` internal slot, returning the flavor name +// (`'Uint8Array'`, …) for a genuine TypedArray and `undefined` for anything +// without the slot. The shim's replacement getter (below) delegates to this via +// the amplifier so an emulated wrapper resolves to its hidden genuine TypedArray +// first — closing the `Object.prototype.toString` fidelity gap. +// @ts-expect-error TS doesn't know it'll be there +const { get: typedArrayToStringTagGetter } = getOwnPropertyDescriptor( + typedArrayPrototype, + Symbol.toStringTag, +); + /** * Copy a range of values from a genuine ArrayBuffer exotic object into a new * ArrayBuffer. @@ -294,7 +381,10 @@ for (const key of ownKeys(immutableArrayBufferLibProperties)) { enumerable: false, }); } -freeze(immutableArrayBufferLibProperties); +// Do not freeze: the shim passes these descriptors directly to +// `defineProperties`, and frozen descriptors (configurable: false, +// writable: false) would conflict with SES's later tamings. Leave the +// record unfrozen so descriptors are directly usable. // Internal-test export. The helper itself is load-bearing for every // method on `immutableArrayBufferLibProperties`, but the package's @@ -328,7 +418,7 @@ const makeImmutableArrayBufferInternal = realBuffer => { // property of each emulated immutable buffer (not on the shared prototype, // which must retain the genuine `'ArrayBuffer'` tag so genuine instances // continue to read as `[object ArrayBuffer]`). This is the minimum - // departure from DESIGN.md Move 2 paragraph 7 needed to keep + // departure from designs/immutable-arraybuffer.md Move 2 paragraph 7 needed to keep // `concordance` (and any downstream consumer that sniffs the toStringTag) // from misrouting an emulated immutable through `Buffer.from`, which // throws because the emulated immutable is not a genuine exotic. With the @@ -416,7 +506,7 @@ if (optArrayBufferTransfer) { const oldTA = new Uint8Array(buffer); const newTA = new Uint8Array(newLength); apply(uint8ArraySet, newTA, [oldTA]); - buffer = apply(uint8ArrayBuffer, newTA, []); + buffer = apply(typedArrayBufferGetter, newTA, []); } } const result = makeImmutableArrayBufferInternal(buffer); @@ -427,3 +517,784 @@ if (optArrayBufferTransfer) { } export const optTransferBufferToImmutable = transferBufferToImmutable; + +// --------------------------------------------------------------------------- +// Freezable TypedArray emulation +// --------------------------------------------------------------------------- +// +// The design document is at: +// packages/immutable-arraybuffer/designs/freezable-typedarray.md +// +// This section extends the immutable-ArrayBuffer lib surface with two exported bindings: +// - makePseudoTypedArrayConstructor (export; factory for per-flavor pseudo-constructors) +// - freezableTypedArrayLibProperties (export; property record the shim copies onto +// %TypedArrayPrototype%) +// +// The following are module-internal: +// - hiddenTypedArrays (brand WeakMap) +// - amplifyTypedArray (returns the hidden genuine TypedArray or the receiver on fallthrough) +// - virtualTypedArrayBufferGetter (getter for %TypedArrayPrototype%.buffer) +// +// The internal `buffers` and `reverseBuffers` WeakMaps from the ArrayBuffer +// side are reused for `view.buffer` redirections. + +/** + * Inverse map: genuine backing ArrayBuffer -> emulated immutable wrapper. + * The ArrayBuffer-side lib owns `buffers` (wrapper -> genuine); this map is + * the reverse direction, used by `virtualTypedArrayBufferGetter` to hand back + * the immutable wrapper when a view's buffer is looked up. + * + * @type {WeakMap} + */ +const reverseBuffers = new WeakMap(); + +/** + * Brand WeakMap for emulated freezable TypedArray wrappers. Maps each wrapper + * to its hidden genuine TypedArray (the one constructed from the actual + * underlying ArrayBuffer). The genuine TypedArray is the storage delegate; + * the wrapper is the public-facing object. + * + * @type {WeakMap} + */ +const hiddenTypedArrays = new WeakMap(); + +/** + * Amplifier-with-this-fallthrough for freezable TypedArrays. Returns the + * hidden genuine TypedArray when `typedArray` is an emulated freezable wrapper + * (present in the brand WeakMap), and returns `typedArray` itself otherwise. + * This lets the methods on `%TypedArrayPrototype%` (after the shim install) + * work as drop-in replacements for genuine TypedArrays. + * + * @param {TypedArray} typedArray + * @returns {TypedArray} + */ +const amplifyTypedArray = typedArray => { + const result = apply(weakmapGet, hiddenTypedArrays, [typedArray]); + if (result !== undefined) { + return result; + } + return typedArray; +}; + +/** + * Getter that replaces `%TypedArrayPrototype%.buffer`. + * When `this` is an emulated freezable wrapper (registered in `hiddenTypedArrays`), + * it returns the immutable ArrayBuffer wrapper via `reverseBuffers`. Otherwise + * it delegates to the captured genuine `%TypedArrayPrototype%.buffer` getter. + * + * Declared via concise method syntax (inside a temporary object literal) rather + * than a `function` declaration or `function`-keyword expression. A + * `function`-keyword function has both `[[Construct]]` and `[[Call]]` behaviors + * (callable with `new`) and an irrelevant `prototype` property pointing at an + * extra object, so `freeze` of such a function is not equivalent to `harden` + * and leaves behind hazardous mutability. An arrow function cannot be used + * here because this getter must be `this`-sensitive. Concise method syntax + * avoids both problems: the method has only `[[Call]]`, no `prototype` + * property, and no `[[Construct]]`. + * + * The surrounding `const` binding avoids JavaScript function-declaration + * hoisting. In the presence of an import cycle a hoisted function + * declaration's value is available to an importing module before the exporting + * module finishes initializing, which can expose uninitialized state. A `const` + * binding produces a Temporal Dead Zone (TDZ) error for the early importer + * instead. + * Note: the ses-shim's compiler from JS ESM module code to JS evaluable code + * does not implement TDZ correctly, so this cycle hazard may not be caught at + * runtime under ses-shim. + * XS uses native compartment and module support and does implement TDZ, so the + * hazard would be caught there. + * This particular PR introduces no such import cycle; the note is for future + * maintainers. + * See the README section "Function expressions versus declarations" for full + * context (erights review comments 3439479281, 3439500526). + */ +const taGetters = { + /** @type {(this: object) => ArrayBuffer} */ + get buffer() { + const genuineTA = apply(weakmapGet, hiddenTypedArrays, [this]); + if (genuineTA !== undefined) { + // The hidden genuine TypedArray's buffer is the genuine backing buffer. + const genuineAB = apply(typedArrayBufferGetter, genuineTA, []); + // Return the immutable wrapper (reverseBuffers maps genuine -> wrapper). + const immutableWrapper = apply(weakmapGet, reverseBuffers, [genuineAB]); + if (immutableWrapper !== undefined) { + return immutableWrapper; + } + return genuineAB; + } + // Fallthrough: delegate to the genuine getter. + return apply(typedArrayBufferGetter, this, []); + }, +}; + +// `getOwnPropertyDescriptor` returns `PropertyDescriptor | undefined`, but +// the `buffer` accessor was just defined on `taGetters` so the descriptor is +// always present. The `PropertyDescriptor` cast informs TypeScript of this. +const virtualTypedArrayBufferGetter = + /** @type {(this: object) => ArrayBuffer} */ ( + /** @type {PropertyDescriptor} */ ( + getOwnPropertyDescriptor(taGetters, 'buffer') + ).get + ); + +/** + * Factory for per-flavor pseudo-constructors. Each pseudo-constructor replaces + * the corresponding global TypedArray constructor (for example `Uint8Array`). + * When called with an emulated immutable ArrayBuffer as the first argument, it + * produces an emulated freezable TypedArray wrapper. For all other call shapes + * it falls through to the genuine constructor via `Reflect.construct`. + * + * The wrapper is a plain ordinary object whose `[[Prototype]]` is + * `OriginalConstructor.prototype`. This is the "drop-the-pseudo-prototype" + * shape: no intermediate prototype exists between the wrapper and the genuine + * prototype. + * + * @param {Function} OriginalConstructor - The genuine TypedArray constructor to wrap. + * @returns {Function} A pseudo-constructor with the same `.name` and `.prototype`. + */ +export const makePseudoTypedArrayConstructor = OriginalConstructor => { + /** + * @param {...any} args + * @returns {object} + */ + function PseudoTypedArray(...args) { + // Determine whether the first argument is an emulated immutable + // ArrayBuffer. + const [firstArg] = args; + const isHidden = + firstArg !== undefined && apply(weakmapHas, buffers, [firstArg]); + + if (!isHidden) { + // Fallthrough: delegate to the genuine constructor. + return construct( + OriginalConstructor, + args, + new.target ?? OriginalConstructor, + ); + } + + // Emulated-immutable branch. + // Retrieve the genuine backing ArrayBuffer from the `buffers` WeakMap. + const genuineAB = apply(weakmapGet, buffers, [firstArg]); + + // Build the remaining constructor arguments using the genuine buffer. + const [, ...restArgs] = args; + const genuineTA = construct(OriginalConstructor, [genuineAB, ...restArgs]); + + // Create the emulated freezable wrapper as a plain object whose prototype + // is OriginalConstructor.prototype (no intermediate prototype). + const wrapper = create(OriginalConstructor.prototype); + + // Register the wrapper in the brand WeakMap (wrapper -> genuine TypedArray). + apply(weakmapSet, hiddenTypedArrays, [wrapper, genuineTA]); + + // Register the reverse mapping so `view.buffer` can reconstruct the + // immutable wrapper from the genuine backing buffer. + apply(weakmapSet, reverseBuffers, [genuineAB, firstArg]); + + return wrapper; + } + + // Preserve the constructor name for debugging and instanceof checks. + defineProperty(PseudoTypedArray, 'name', { + value: OriginalConstructor.name, + writable: false, + enumerable: false, + configurable: true, + }); + + // The `prototype` property must be the genuine prototype so that + // `instanceof T` and `Object.getPrototypeOf(wrapper) === T.prototype` + // both hold. We share the genuine prototype rather than creating a new one. + PseudoTypedArray.prototype = OriginalConstructor.prototype; + + // Set the `prototype.constructor` to the pseudo-constructor so that SES's + // intrinsic walk finds consistency: after we install PseudoTypedArray as + // `globalThis.BigInt64Array` (for example), SES samples `BigInt64Array` and + // resolves it to PseudoTypedArray. It then walks the permit graph and checks + // that `intrinsics.%BigInt64ArrayPrototype%.constructor === intrinsics.BigInt64Array`. + // If `prototype.constructor` still points to the original constructor, + // that check fails. Updating `prototype.constructor` to PseudoTypedArray + // ensures both pointers agree with the intrinsics map. + // + // This does NOT affect genuine TypedArray construction: + // `new OriginalConstructor(realAb)` delegates to the captured genuine + // constructor via `Reflect.construct`, which ignores `prototype.constructor`. + defineProperty(OriginalConstructor.prototype, 'constructor', { + value: PseudoTypedArray, + writable: true, + enumerable: false, + configurable: true, + }); + + // Set the pseudo-constructor's `[[Prototype]]` to `%TypedArray%` (the + // abstract TypedArray superclass). SES's intrinsic walk validates that + // each concrete TypedArray constructor inherits from `%TypedArray%` via the + // constructor chain (`BigInt64Array.__proto__ === TypedArray`). A plain + // function's default `Function.prototype` prototype would fail that check. + setPrototypeOf(PseudoTypedArray, TypedArray); + + // Copy the `BYTES_PER_ELEMENT` static property from the original constructor. + // Callers such as `packages/captp/src/atomics.js` read this constant + // directly off the constructor (`BigUint64Array.BYTES_PER_ELEMENT`, + // `Int32Array.BYTES_PER_ELEMENT`). The shim replaces the global binding with + // `PseudoTypedArray`, so the property must be present on the replacement or + // those reads return `undefined`, making arithmetic expressions produce NaN. + // + // `BYTES_PER_ELEMENT` is not inherited through the prototype chain on + // TypedArray constructors; each concrete constructor carries its own own- + // property value (8 for BigUint64Array, 4 for Int32Array, etc.). + defineProperty(PseudoTypedArray, 'BYTES_PER_ELEMENT', { + // @ts-expect-error TS2339: BYTES_PER_ELEMENT exists on TypedArray constructors but not on Function + value: OriginalConstructor.BYTES_PER_ELEMENT, + writable: false, + enumerable: false, + configurable: true, + }); + + // Do NOT freeze here. SES's `hardenIntrinsics` will freeze all + // primordials (including the pseudo-constructors installed on globalThis) + // as part of `lockdown()`. Pre-freezing would cause SES's pre-lockdown + // consistency check ("all intrinsics must be unfrozen before repairIntrinsics") + // to fail. The function is effectively immutable at runtime because the + // only callers are the shim install path (which has already run) and + // callers of the returned constructor. + return PseudoTypedArray; +}; + +/** + * Property record the shim copies onto `%TypedArrayPrototype%`. Contains: + * + * - The `buffer`, `byteLength`, `byteOffset`, and `length` accessor + * replacements. Each discriminates on `hiddenTypedArrays` brand membership: + * on hit it delegates to the hidden genuine TypedArray (amplifier pattern); + * on miss it delegates to the captured genuine accessor (fallthrough). + * + * - Mutator-throws descriptors for the five mutator methods: `copyWithin`, + * `fill`, `reverse`, `set`, `sort`. On emulated freezable wrappers each + * throws `TypeError`; on genuine TypedArrays each delegates to the captured + * genuine method (the amplifier-with-this-fallthrough shape). + * + * - Amplifier-delegate wrappers for all remaining read-only `%TypedArrayPrototype%` + * methods. Plain ordinary wrappers cannot be passed as `this` to any native + * TypedArray method that checks for integer-indexed exotic internal slots. + * The amplifier resolves the wrapper to its hidden genuine TypedArray first, + * so the captured genuine method receives a valid `this`. + * + * The record's properties are made non-enumerable below, matching the shape + * of the genuine `%TypedArrayPrototype%`. + */ +export const freezableTypedArrayLibProperties = { + __proto__: null, + + // ------------------------------------------------------------------------- + // Accessors: `buffer`, `byteLength`, `byteOffset`, `length` + // ------------------------------------------------------------------------- + + /** + * @this {object} + * @returns {ArrayBuffer} + */ + get buffer() { + return apply(virtualTypedArrayBufferGetter, this, []); + }, + /** + * @this {object} + * @returns {number} + */ + get byteLength() { + return apply(typedArrayByteLengthGetter, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @returns {number} + */ + get byteOffset() { + return apply(typedArrayByteOffsetGetter, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @returns {number} + */ + get length() { + return apply(typedArrayLengthGetter, amplifyTypedArray(this), []); + }, + + // ------------------------------------------------------------------------- + // `[Symbol.toStringTag]`: amplify then read the genuine internal-slot tag + // ------------------------------------------------------------------------- + + /** + * Replaces the genuine, `this`-sensitive `%TypedArrayPrototype%` + * `[Symbol.toStringTag]` getter with a wrapper around it. On an emulated + * freezable wrapper (brand-WeakMap member) it amplifies to the hidden genuine + * TypedArray and reads *its* tag, so `Object.prototype.toString.call(wrapper)` + * reads `'[object Uint8Array]'` — matching a genuine view. On a genuine + * TypedArray it falls through to the genuine getter (amplifier returns `this` + * unchanged); on any other receiver the genuine getter returns `undefined`, + * exactly as before. + * + * This is the getter-wrapper fidelity fix from erights's review of + * endojs/endo-but-for-bots#475 (comments 3817252816 / 3817264546): a + * higher-fidelity repair than installing a `[Symbol.toStringTag]` *data* + * property, which would patch only the `Object.prototype.toString` lookup + * path and leave this getter still reporting `undefined` on a wrapper. + * Because the getter and `Object.prototype.toString` now agree, any brand + * check that captures *this* (shim-installed) getter — e.g. + * `@endo/harden`'s `isTypedArray` if it captures after the shim installs — + * observes an emulated wrapper as a TypedArray. + * + * @this {object} + * @returns {string | undefined} + */ + get [Symbol.toStringTag]() { + return apply(typedArrayToStringTagGetter, amplifyTypedArray(this), []); + }, + + // ------------------------------------------------------------------------- + // Mutator methods: throw on emulated freezable wrappers + // ------------------------------------------------------------------------- + + /** + * @this {object} + * @param {number} [target] + * @param {number} [start] + * @param {number} [end] + * @returns {object} + */ + copyWithin(target = undefined, start = undefined, end = undefined) { + if (apply(weakmapHas, hiddenTypedArrays, [this])) { + throw TypeError( + 'Cannot copyWithin on a freezable TypedArray backed by an immutable ArrayBuffer', + ); + } + return apply(typedArrayCopyWithin, this, [target, start, end]); + }, + /** + * @this {object} + * @param {any} [value] + * @param {number} [start] + * @param {number} [end] + * @returns {object} + */ + fill(value = undefined, start = undefined, end = undefined) { + if (apply(weakmapHas, hiddenTypedArrays, [this])) { + throw TypeError( + 'Cannot fill a freezable TypedArray backed by an immutable ArrayBuffer', + ); + } + return apply(typedArrayFill, this, [value, start, end]); + }, + /** + * @this {object} + * @returns {object} + */ + reverse() { + if (apply(weakmapHas, hiddenTypedArrays, [this])) { + throw TypeError( + 'Cannot reverse a freezable TypedArray backed by an immutable ArrayBuffer', + ); + } + return apply(typedArrayReverse, this, []); + }, + /** + * @this {object} + * @param {any} array + * @param {number} [offset] + * @returns {void} + */ + set(array = undefined, offset = undefined) { + if (apply(weakmapHas, hiddenTypedArrays, [this])) { + throw TypeError( + 'Cannot set on a freezable TypedArray backed by an immutable ArrayBuffer', + ); + } + return apply(typedArraySet, this, [array, offset]); + }, + /** + * @this {object} + * @param {Function} [compareFn] + * @returns {object} + */ + sort(compareFn = undefined) { + if (apply(weakmapHas, hiddenTypedArrays, [this])) { + throw TypeError( + 'Cannot sort a freezable TypedArray backed by an immutable ArrayBuffer', + ); + } + return apply(typedArraySort, this, [compareFn]); + }, + + // ------------------------------------------------------------------------- + // Read-only method delegates: amplify then call the captured genuine method + // ------------------------------------------------------------------------- + + /** + * @this {object} + * @param {number} [index] + * @returns {any} + */ + at(index = undefined) { + return apply(typedArrayAt, amplifyTypedArray(this), [index]); + }, + /** + * @this {object} + * @returns {Iterator} + */ + entries() { + return apply(typedArrayEntries, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {boolean} + */ + every(predicate = undefined, thisArg = undefined) { + return apply(typedArrayEvery, amplifyTypedArray(this), [ + predicate, + thisArg, + ]); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {object} + */ + filter(predicate = undefined, thisArg = undefined) { + return apply(typedArrayFilter, amplifyTypedArray(this), [ + predicate, + thisArg, + ]); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {any} + */ + find(predicate = undefined, thisArg = undefined) { + return apply(typedArrayFind, amplifyTypedArray(this), [predicate, thisArg]); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {number} + */ + findIndex(predicate = undefined, thisArg = undefined) { + return apply(typedArrayFindIndex, amplifyTypedArray(this), [ + predicate, + thisArg, + ]); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {any} + */ + findLast(predicate = undefined, thisArg = undefined) { + if (typedArrayFindLast === undefined) { + throw TypeError( + 'TypedArray.prototype.findLast is not available on this platform', + ); + } + return apply(typedArrayFindLast, amplifyTypedArray(this), [ + predicate, + thisArg, + ]); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {number} + */ + findLastIndex(predicate = undefined, thisArg = undefined) { + if (typedArrayFindLastIndex === undefined) { + throw TypeError( + 'TypedArray.prototype.findLastIndex is not available on this platform', + ); + } + return apply(typedArrayFindLastIndex, amplifyTypedArray(this), [ + predicate, + thisArg, + ]); + }, + /** + * @this {object} + * @param {Function} [callback] + * @param {any} [thisArg] + * @returns {void} + */ + forEach(callback = undefined, thisArg = undefined) { + return apply(typedArrayForEach, amplifyTypedArray(this), [ + callback, + thisArg, + ]); + }, + /** + * @this {object} + * @param {any} searchElement + * @param {number} [fromIndex] + * @returns {boolean} + */ + includes(searchElement = undefined, fromIndex = undefined) { + return apply(typedArrayIncludes, amplifyTypedArray(this), [ + searchElement, + fromIndex, + ]); + }, + /** + * @this {object} + * @param {any} searchElement + * @param {number} [fromIndex] + * @returns {number} + */ + indexOf(searchElement = undefined, fromIndex = undefined) { + return apply(typedArrayIndexOf, amplifyTypedArray(this), [ + searchElement, + fromIndex, + ]); + }, + /** + * @this {object} + * @param {string} [separator] + * @returns {string} + */ + join(separator = undefined) { + return apply(typedArrayJoin, amplifyTypedArray(this), [separator]); + }, + /** + * @this {object} + * @returns {Iterator} + */ + keys() { + return apply(typedArrayKeys, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @param {any} searchElement + * @param {number} [fromIndex] + * @returns {number} + */ + lastIndexOf(searchElement = undefined, fromIndex = undefined) { + return apply(typedArrayLastIndexOf, amplifyTypedArray(this), [ + searchElement, + fromIndex, + ]); + }, + /** + * @this {object} + * @param {Function} [callback] + * @param {any} [thisArg] + * @returns {object} + */ + map(callback = undefined, thisArg = undefined) { + return apply(typedArrayMap, amplifyTypedArray(this), [callback, thisArg]); + }, + /** + * @this {object} + * @param {Function} [callback] + * @param {any} [initialValue] + * @returns {any} + */ + reduce(callback = undefined, initialValue = undefined) { + return apply( + typedArrayReduce, + amplifyTypedArray(this), + arguments.length > 1 ? [callback, initialValue] : [callback], + ); + }, + /** + * @this {object} + * @param {Function} [callback] + * @param {any} [initialValue] + * @returns {any} + */ + reduceRight(callback = undefined, initialValue = undefined) { + return apply( + typedArrayReduceRight, + amplifyTypedArray(this), + arguments.length > 1 ? [callback, initialValue] : [callback], + ); + }, + /** + * @this {object} + * @param {number} [start] + * @param {number} [end] + * @returns {object} + */ + slice(start = undefined, end = undefined) { + return apply(typedArraySlice, amplifyTypedArray(this), [start, end]); + }, + /** + * @this {object} + * @param {Function} [predicate] + * @param {any} [thisArg] + * @returns {boolean} + */ + some(predicate = undefined, thisArg = undefined) { + return apply(typedArraySome, amplifyTypedArray(this), [predicate, thisArg]); + }, + /** + * @this {object} + * @param {number} [begin] + * @param {number} [end] + * @returns {object} + */ + subarray(begin = undefined, end = undefined) { + const genuineTA = apply(weakmapGet, hiddenTypedArrays, [this]); + if (genuineTA !== undefined) { + // `this` is an emulated freezable wrapper. Delegate to the hidden genuine + // TypedArray to get the sub-view genuine TypedArray, then wrap it in a new + // emulated freezable wrapper so the safety contract (`sub.buffer === iab`) + // holds for sub-views. + const genuineSub = apply(typedArraySubarray, genuineTA, [begin, end]); + // `create(getPrototypeOf(this))` is sufficient rather than calling the + // pseudo-constructor because the sub-view wrapper needs exactly three + // things the parent wrapper already provides: + // + // 1. The right prototype. `getPrototypeOf(this)` is + // `OriginalConstructor.prototype`, the same prototype the + // pseudo-constructor would set via `create(OriginalConstructor.prototype)`. + // The shim's freezable behaviors live on that prototype (installed onto + // `%TypedArrayPrototype%`), so they are already inherited. + // + // 2. A `hiddenTypedArrays` registration. The line below registers + // `subWrapper -> genuineSub` in the brand WeakMap, which is what + // `amplifyTypedArray` and every method that discriminates on brand + // membership require. No other per-instance state is needed. + // + // 3. A `reverseBuffers` entry for `view.buffer` redirection. A sub-array + // shares its backing buffer with the parent; `typedArraySubarray` does + // not allocate a new buffer. The pseudo-constructor's `reverseBuffers` + // registration maps the genuine backing buffer to the immutable wrapper, + // and that entry was already written when the parent was constructed. + // The sub-view's genuine buffer is the same genuine buffer, so no new + // `reverseBuffers` entry is needed. + // + // Static properties (`BYTES_PER_ELEMENT`) live on + // `OriginalConstructor.prototype.constructor`, not on the instance, so + // they are also already present via the prototype chain. There is no + // instance state that the pseudo-constructor would add that `create` does + // not already provide. + const subWrapper = create(getPrototypeOf(this)); + apply(weakmapSet, hiddenTypedArrays, [subWrapper, genuineSub]); + // `reverseBuffers` already maps the genuine backing buffer to the immutable + // wrapper from when the parent wrapper was constructed; no new entry needed. + return subWrapper; + } + return apply(typedArraySubarray, this, [begin, end]); + }, + /** + * @this {object} + * @param {...any} args + * @returns {string} + */ + toLocaleString(...args) { + return apply(typedArrayToLocaleString, amplifyTypedArray(this), args); + }, + /** + * @this {object} + * @returns {string} + */ + toString() { + return apply(typedArrayToString, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @returns {Iterator} + */ + values() { + return apply(typedArrayValues, amplifyTypedArray(this), []); + }, + /** + * The default iterator for TypedArrays is `%TypedArrayPrototype%.values`. + * In the baseline runtime `%TypedArrayPrototype%[Symbol.iterator]` is the + * same function object as `%TypedArrayPrototype%.values`. After the shim + * installs a new `values` wrapper, `Symbol.iterator` would still point at + * the original genuine `values` function unless we re-install it here as + * well. Without this entry, `for...of` loops and spread syntax on emulated + * freezable wrappers throw `TypeError: this is not a typed array.` + * + * @this {object} + * @returns {Iterator} + */ + [Symbol.iterator]() { + return apply(typedArrayValues, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @returns {object} + */ + toReversed() { + if (typedArrayToReversed === undefined) { + throw TypeError( + 'TypedArray.prototype.toReversed is not available on this platform', + ); + } + return apply(typedArrayToReversed, amplifyTypedArray(this), []); + }, + /** + * @this {object} + * @param {Function} [compareFn] + * @returns {object} + */ + toSorted(compareFn = undefined) { + if (typedArrayToSorted === undefined) { + throw TypeError( + 'TypedArray.prototype.toSorted is not available on this platform', + ); + } + return apply(typedArrayToSorted, amplifyTypedArray(this), [compareFn]); + }, + /** + * @this {object} + * @param {number} [index] + * @param {any} [value] + * @returns {object} + */ + with(index = undefined, value = undefined) { + if (typedArrayWith === undefined) { + throw TypeError( + 'TypedArray.prototype.with is not available on this platform', + ); + } + return apply(typedArrayWith, amplifyTypedArray(this), [index, value]); + }, +}; + +// Make all properties non-enumerable, matching %TypedArrayPrototype%'s shape. +for (const key of ownKeys(freezableTypedArrayLibProperties)) { + defineProperty(freezableTypedArrayLibProperties, key, { + enumerable: false, + }); +} +// Do not freeze: the shim passes these descriptors directly to +// `defineProperties`, and frozen descriptors (configurable: false, +// writable: false) would prevent SES's `tameLocaleMethods` from later +// replacing `toLocaleString`. Leave the record unfrozen so descriptors +// are directly usable. + +/** + * The eleven concrete TypedArray constructors that share `%TypedArrayPrototype%`. + * The shim replaces each with a pseudo-constructor from `makePseudoTypedArrayConstructor`. + * + * @type {Array<{name: string, Ctor: Function}>} + */ +export const concreteTypedArrayCtors = [ + { name: 'Int8Array', Ctor: Int8Array }, + { name: 'Int16Array', Ctor: Int16Array }, + { name: 'Int32Array', Ctor: Int32Array }, + { name: 'Uint8Array', Ctor: Uint8Array }, + { name: 'Uint8ClampedArray', Ctor: Uint8ClampedArray }, + { name: 'Uint16Array', Ctor: Uint16Array }, + { name: 'Uint32Array', Ctor: Uint32Array }, + { name: 'Float32Array', Ctor: Float32Array }, + { name: 'Float64Array', Ctor: Float64Array }, + { name: 'BigInt64Array', Ctor: BigInt64Array }, + { name: 'BigUint64Array', Ctor: BigUint64Array }, +]; diff --git a/packages/immutable-arraybuffer/src/shim.js b/packages/immutable-arraybuffer/src/shim.js index dd3dbea79f..f91229686e 100644 --- a/packages/immutable-arraybuffer/src/shim.js +++ b/packages/immutable-arraybuffer/src/shim.js @@ -1,9 +1,19 @@ -import { immutableArrayBufferLibProperties } from './lib.js'; +import { + immutableArrayBufferLibProperties, + freezableTypedArrayLibProperties, + makePseudoTypedArrayConstructor, + concreteTypedArrayCtors, +} from './lib.js'; // eslint-disable-next-line no-restricted-globals const { ArrayBuffer, Object } = globalThis; -const { getOwnPropertyDescriptors, defineProperties } = Object; +const { + getOwnPropertyDescriptors, + defineProperties, + defineProperty, + getPrototypeOf, +} = Object; const { prototype: arrayBufferPrototype } = ArrayBuffer; // Stage-3 install policy: detect-then-skip. @@ -27,8 +37,51 @@ const { prototype: arrayBufferPrototype } = ArrayBuffer; // divergent platform implementations. The Immutable ArrayBuffer proposal // is past that threshold. if (!('sliceToImmutable' in arrayBufferPrototype)) { + // ArrayBuffer-side install (immutable ArrayBuffer shim). defineProperties( arrayBufferPrototype, getOwnPropertyDescriptors(immutableArrayBufferLibProperties), ); + + // Freezable TypedArray install. + // + // The %TypedArrayPrototype% is the shared abstract superclass prototype + // that all eleven concrete TypedArray constructors (Int8Array, Uint8Array, + // etc.) inherit through their own `.prototype`. Installing the property + // record once on %TypedArrayPrototype% covers all eleven flavors. + // + // `getPrototypeOf(Uint8Array.prototype)` is the standard way to reach + // %TypedArrayPrototype% without a dedicated + // intrinsic name. + const typedArrayPrototype = getPrototypeOf( + // eslint-disable-next-line no-restricted-globals + globalThis.Uint8Array.prototype, + ); + + // Install the lib property record onto %TypedArrayPrototype%. + // + // `freezableTypedArrayLibProperties` is an unfrozen record whose + // descriptors are configurable and writable (matching the shape of the + // native %TypedArrayPrototype% methods), so we can pass them directly + // to `defineProperties` without reopening. + defineProperties( + typedArrayPrototype, + getOwnPropertyDescriptors(freezableTypedArrayLibProperties), + ); + + // Replace each of the eleven concrete global TypedArray constructors with + // the pseudo-constructor produced by the lib. The pseudo-constructor + // discriminates on `buffers` brand membership and falls through to + // the genuine constructor for all other call shapes. + for (const { name, Ctor } of concreteTypedArrayCtors) { + const PseudoCtor = makePseudoTypedArrayConstructor(Ctor); + defineProperty( + // eslint-disable-next-line no-restricted-globals + globalThis, + name, + { + value: PseudoCtor, + }, + ); + } } diff --git a/packages/immutable-arraybuffer/test/_emulated-only.js b/packages/immutable-arraybuffer/test/_emulated-only.js new file mode 100644 index 0000000000..63d3aa1e5a --- /dev/null +++ b/packages/immutable-arraybuffer/test/_emulated-only.js @@ -0,0 +1,41 @@ +// Test helper: detect whether the freezable-TypedArray emulation is active. +// +// The shim installs under a stage-3 detect-then-skip policy (see +// `src/shim.js`): on an engine that already ships a native Immutable +// ArrayBuffer implementation — current XS does — `sliceToImmutable` is already +// present, the shim steps aside, and `new Uint8Array(iab)` is a GENUINE +// integer-indexed view rather than the emulated plain-object wrapper. +// +// On such a native engine the emulated-wrapper fidelity observations do NOT +// hold, because there is no emulated wrapper to observe: +// - `ArrayBuffer.isView(view)` is `true`, not `false`; +// - `view[i]` reads the underlying byte, not `undefined`; +// - `Object.prototype.toString.call(view)` reads `'[object Uint8Array]'`, +// and an immutable buffer reads `'[object ArrayBuffer]'`, not the shim's +// `'[object Object]'` / `'[object ImmutableArrayBuffer]'` departures; +// - an indexed assignment does not create an own OrdinarySet shadow. +// +// Tests that assert those emulated-only shapes therefore describe the shim +// path specifically and have nothing to check on a native engine. Gate them +// with `emulatedOnlyTest` so they run under the shim and skip under native, +// rather than baking in the (now obsolete) assumption that no engine ships +// native support. See endojs/endo-but-for-bots#475 (erights review). +// +// A file that uses this helper must `import '../src/shim.js'` first, so +// `sliceToImmutable` is guaranteed present (native or shim) before detection. +import test from 'ava'; + +/** + * True when `@endo/immutable-arraybuffer` is providing the emulated + * freezable-TypedArray wrapper (i.e. no native implementation is present). + */ +export const emulationActive = !ArrayBuffer.isView( + new Uint8Array(new ArrayBuffer(0).sliceToImmutable()), +); + +/** + * `test` when the emulation is active, `test.skip` otherwise. Use for + * assertions that describe the emulated plain-object wrapper specifically and + * so have nothing to observe on a native immutable-ArrayBuffer engine. + */ +export const emulatedOnlyTest = emulationActive ? test : test.skip; diff --git a/packages/immutable-arraybuffer/test/_lib-setup.md b/packages/immutable-arraybuffer/test/_lib-setup.md index 02bc9d1849..4c16965675 100644 --- a/packages/immutable-arraybuffer/test/_lib-setup.md +++ b/packages/immutable-arraybuffer/test/_lib-setup.md @@ -19,9 +19,6 @@ shim has run. The lib's free-function helpers (`sliceBufferToImmutable`, `optTransferBufferToImmutable`) remain importable from the lib module for tests that want to exercise the free-function call shape directly. -They are also (today) re-exported from `index.js` for pre-shim callers; -the premise-2 follow-up PR will retire the free-function exports from -the package's module surface but not from the lib module itself. The `shim-amplifier.test.js` and `shim-slice.test.js` / `shim-transfer.test.js` test files have always installed the shim at module top (their purpose diff --git a/packages/immutable-arraybuffer/test/bytes.test.js b/packages/immutable-arraybuffer/test/bytes.test.js new file mode 100644 index 0000000000..8e734e1ff3 --- /dev/null +++ b/packages/immutable-arraybuffer/test/bytes.test.js @@ -0,0 +1,67 @@ +// @ts-nocheck +// Tests for the `frozenBytes` / `thawedBytes` byte utilities exported from +// the package's main entry point. They pair with the shim (which the module +// installs as a side effect of importing it). +import test from 'ava'; +import { frozenBytes, thawedBytes } from '../index.js'; + +const { isFrozen } = Object; + +test('frozenBytes: wraps a view in a frozen Uint8Array on an immutable buffer', t => { + const view = new Uint8Array([1, 2, 3, 4, 5]); + const frozen = frozenBytes(view); + t.true(frozen instanceof Uint8Array); + t.is(frozen.byteLength, 5); + t.true(frozen.buffer instanceof ArrayBuffer); + t.true(frozen.buffer.immutable); + t.true(isFrozen(frozen)); + t.deepEqual([...thawedBytes(frozen)], [1, 2, 3, 4, 5]); +}); + +test('frozenBytes: honors subarray byteOffset and byteLength', t => { + const full = new Uint8Array([0, 1, 2, 3, 4, 5, 6, 7]); + const window = full.subarray(2, 6); // [2, 3, 4, 5] + const frozen = frozenBytes(window); + t.is(frozen.byteLength, 4); + // The frozen wrapper spans its whole backing buffer one-to-one. + t.is(frozen.byteOffset, 0); + t.is(frozen.byteLength, frozen.buffer.byteLength); + t.deepEqual([...thawedBytes(frozen)], [2, 3, 4, 5]); +}); + +test('frozenBytes: empty input', t => { + const frozen = frozenBytes(new Uint8Array(0)); + t.is(frozen.byteLength, 0); + t.true(frozen.buffer.immutable); +}); + +test('thawedBytes: returns a fresh mutable copy of a frozen value', t => { + const source = new Uint8Array([0, 1, 2, 0xff, 0x80, 0x00, 42, 100]); + const frozen = frozenBytes(source); + const mutable = thawedBytes(frozen); + t.true(mutable instanceof Uint8Array); + t.false(isFrozen(mutable)); + t.deepEqual([...mutable], [...source]); + // Mutating the copy does not affect the frozen original. + mutable[0] = 99; + t.deepEqual([...thawedBytes(frozen)], [...source]); +}); + +test('thawedBytes: accepts a genuine mutable Uint8Array view', t => { + const view = new Uint8Array([9, 8, 7]); + const copy = thawedBytes(view); + t.not(copy, view); + t.deepEqual([...copy], [9, 8, 7]); +}); + +test('thawedBytes: accepts a bare ArrayBufferLike', t => { + const buffer = new Uint8Array([5, 6, 7, 8]).buffer; + const copy = thawedBytes(buffer); + t.true(copy instanceof Uint8Array); + t.deepEqual([...copy], [5, 6, 7, 8]); +}); + +test('frozenBytes and thawedBytes are hardened', t => { + t.true(isFrozen(frozenBytes)); + t.true(isFrozen(thawedBytes)); +}); diff --git a/packages/immutable-arraybuffer/test/lib-slice.test.js b/packages/immutable-arraybuffer/test/lib-slice.test.js index d8487fe3f2..97d035edc4 100644 --- a/packages/immutable-arraybuffer/test/lib-slice.test.js +++ b/packages/immutable-arraybuffer/test/lib-slice.test.js @@ -105,24 +105,27 @@ test('Standard TypedArray behavior baseline', t => { // This could have been written as a test.failing as compared to // the immutable ArrayBuffer we'll propose. However, I'd rather test what // the shim purposely does instead. -test('TypedArray on Immutable ArrayBuffer lib limitations', t => { +test('TypedArray on Immutable ArrayBuffer: freezable-TypedArray emulation now supported', t => { + // As of the freezable-TypedArray shim (PR implementing + // designs/freezable-typedarray.md), calling a TypedArray constructor with an + // emulated immutable ArrayBuffer produces an emulated freezable wrapper + // whose byteLength matches the underlying buffer, whose mutator methods + // throw TypeError, and whose buffer accessor returns the immutable wrapper. + // The old limitation (producing a 0-byte TypedArray) no longer applies. const ab1 = new ArrayBuffer(2); - const dv1 = new DataView(ab1); - t.is(dv1.buffer, ab1); - t.is(dv1.byteLength, 2); const ta1 = new Uint8Array(ab1); ta1[0] = 3; ta1[1] = 4; - t.is(ta1.byteLength, 2); const iab = sliceBufferToImmutable(ab1); - // Unfortunately, unlike the immutable ArrayBuffer to be proposed, - // calling a TypedArray constructor with the shim implementation of - // an immutable ArrayBuffer as argument treats it as an unrecognized object, - // rather than throwing an error or acting as a non-changeable TypedArray. t.is(iab.byteLength, 2); const ta3 = new Uint8Array(iab); - t.is(ta3.byteLength, 0); + // The emulated freezable wrapper covers the full 2 bytes. + t.is(ta3.byteLength, 2); + // Mutators throw on the emulated freezable wrapper. + t.throws(() => ta3.set([0, 0]), { instanceOf: TypeError }); + // The buffer accessor returns the immutable wrapper. + t.is(ta3.buffer, iab); }); const testTransfer = t => { diff --git a/packages/immutable-arraybuffer/test/lib-transfer.test.js b/packages/immutable-arraybuffer/test/lib-transfer.test.js index 4ae689b5a1..c1008fbf4b 100644 --- a/packages/immutable-arraybuffer/test/lib-transfer.test.js +++ b/packages/immutable-arraybuffer/test/lib-transfer.test.js @@ -107,27 +107,33 @@ test('Standard TypedArray behavior baseline', t => { t.is(ta2.byteLength, 0); }); -// This could have been written as a test.failing as compared to -// the immutable ArrayBuffer we'll propose. However, I'd rather test what // the shim purposely does instead. -test('TypedArray on Immutable ArrayBuffer lib limitations', t => { +test('TypedArray on Immutable ArrayBuffer: freezable-TypedArray emulation now supported', t => { + // As of the freezable-TypedArray shim (PR implementing + // designs/freezable-typedarray.md), calling a TypedArray constructor with an + // emulated immutable ArrayBuffer produced by transferToImmutable produces an + // emulated freezable wrapper whose byteLength matches the underlying buffer. + // The old limitation (producing a 0-byte TypedArray) no longer applies. + if (optTransferBufferToImmutable === undefined) { + t.pass( + 'Platform lacks transfer or structuredClone; skip transferToImmutable coverage', + ); + return; + } const ab1 = new ArrayBuffer(2); - const dv1 = new DataView(ab1); - t.is(dv1.buffer, ab1); - t.is(dv1.byteLength, 2); const ta1 = new Uint8Array(ab1); ta1[0] = 3; ta1[1] = 4; - t.is(ta1.byteLength, 2); const iab = optTransferBufferToImmutable(ab1); - // Unfortunately, unlike the immutable ArrayBuffer to be proposed, - // calling a TypedArray constructor with the shim implementation of - // an immutable ArrayBuffer as argument treats it as an unrecognized object, - // rather than throwing an error or acting as a non-changeable TypedArray. t.is(iab.byteLength, 2); const ta3 = new Uint8Array(iab); - t.is(ta3.byteLength, 0); + // The emulated freezable wrapper covers the full 2 bytes. + t.is(ta3.byteLength, 2); + // Mutators throw on the emulated freezable wrapper. + t.throws(() => ta3.set([0, 0]), { instanceOf: TypeError }); + // The buffer accessor returns the immutable wrapper. + t.is(ta3.buffer, iab); }); const testTransfer = t => { diff --git a/packages/immutable-arraybuffer/test/lib-typedarray.test.js b/packages/immutable-arraybuffer/test/lib-typedarray.test.js new file mode 100644 index 0000000000..ce1c4e9cf7 --- /dev/null +++ b/packages/immutable-arraybuffer/test/lib-typedarray.test.js @@ -0,0 +1,122 @@ +// @ts-nocheck +// Lib-level unit tests for the freezable-TypedArray emulation. +// These tests exercise the property-record and pseudo-constructor machinery in +// isolation (with the ArrayBuffer-side shim installed so that +// `sliceBufferToImmutable` and friends are available via the prototype). +import '../src/shim.js'; +import test from 'ava'; +import { + sliceBufferToImmutable, + makePseudoTypedArrayConstructor, +} from '../src/lib.js'; + +const { getPrototypeOf } = Object; + +// --------------------------------------------------------------------------- +// makePseudoTypedArrayConstructor - wrapping an immutable ArrayBuffer +// --------------------------------------------------------------------------- + +test('makePseudoTypedArrayConstructor wraps an immutable ArrayBuffer', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([1, 2, 3, 4]); + const iab = sliceBufferToImmutable(ab); + + const PseudoUint8Array = makePseudoTypedArrayConstructor(Uint8Array); + const view = new PseudoUint8Array(iab); + + // The wrapper's prototype is Uint8Array.prototype (no intermediate prototype). + t.is(getPrototypeOf(view), Uint8Array.prototype); + + // The amplifier (via .buffer getter) returns the immutable wrapper, not the + // genuine backing buffer. + // `virtualTypedArrayBufferGetter` is exercised internally; we observe its + // effect through the installed `view.buffer` accessor. + t.is(view.buffer, iab); + t.true(view.buffer.immutable); +}); + +// --------------------------------------------------------------------------- +// makePseudoTypedArrayConstructor - forwarding a non-immutable first arg +// --------------------------------------------------------------------------- + +test('makePseudoTypedArrayConstructor forwards a non-immutable first arg', t => { + const realAb = new ArrayBuffer(4); + new Uint8Array(realAb).set([10, 20, 30, 40]); + + const PseudoUint8Array = makePseudoTypedArrayConstructor(Uint8Array); + const view = new PseudoUint8Array(realAb); + + // Fallthrough path: the result is a genuine TypedArray, not a wrapper. + t.is(getPrototypeOf(view), Uint8Array.prototype); + + // `view.buffer` returns the real buffer (amplifyTypedArray falls through to + // the receiver itself for a genuine TypedArray). + t.is(view.buffer, realAb); + + // Mutators work normally on the genuine view. + view[0] = 99; + t.is(view[0], 99); +}); + +// --------------------------------------------------------------------------- +// buffer getter - returns genuine buffer for a genuine TypedArray (fallthrough) +// --------------------------------------------------------------------------- + +test('buffer getter returns the real buffer for a genuine TypedArray', t => { + const realAb = new ArrayBuffer(4); + const view = new Uint8Array(realAb); + + // `virtualTypedArrayBufferGetter` is installed on %TypedArrayPrototype%; + // `view.buffer` exercises the fallthrough path. + t.is(view.buffer, realAb); + t.false(view.buffer.immutable); +}); + +// --------------------------------------------------------------------------- +// buffer getter - redirects to the immutable wrapper when the TypedArray is +// an emulated freezable +// --------------------------------------------------------------------------- + +test('buffer getter redirects to the immutable wrapper when present', t => { + const ab = new ArrayBuffer(4); + const iab = sliceBufferToImmutable(ab); + + const PseudoUint8Array = makePseudoTypedArrayConstructor(Uint8Array); + const view = new PseudoUint8Array(iab); + + // `virtualTypedArrayBufferGetter` is installed on %TypedArrayPrototype%; + // `view.buffer` exercises the emulated-wrapper path. + t.is(view.buffer, iab); + t.true(view.buffer.immutable); +}); + +// --------------------------------------------------------------------------- +// amplifyTypedArray - brand-WeakMap amplifier (observed through read delegates) +// --------------------------------------------------------------------------- + +test('amplifyTypedArray delegates reads from the hidden genuine TypedArray for a wrapper', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([5, 6, 7, 8]); + const iab = sliceBufferToImmutable(ab); + + const PseudoUint8Array = makePseudoTypedArrayConstructor(Uint8Array); + const view = new PseudoUint8Array(iab); + + // `amplifyTypedArray` is called by the `byteLength`, `length`, and `at` + // property descriptors installed on %TypedArrayPrototype%. The values + // match the underlying bytes. + t.is(view.byteLength, 4); + t.is(view.length, 4); + t.is(view.at(0), 5); + t.is(view.at(3), 8); +}); + +test('amplifyTypedArray falls through for a genuine TypedArray', t => { + // A genuine TypedArray is not in `hiddenTypedArrays`; `amplifyTypedArray` + // returns the receiver itself. + // We observe this by verifying that `byteLength` reads from the view + // directly (the genuine buffer's length, not a wrapper's). + const view = new Uint8Array(new ArrayBuffer(4)); + t.is(view.byteLength, 4); + t.is(view.length, 4); +}); diff --git a/packages/immutable-arraybuffer/test/shim-amplifier.test.js b/packages/immutable-arraybuffer/test/shim-amplifier.test.js index 382131b5f3..156e01f3f2 100644 --- a/packages/immutable-arraybuffer/test/shim-amplifier.test.js +++ b/packages/immutable-arraybuffer/test/shim-amplifier.test.js @@ -13,6 +13,7 @@ import { isBufferImmutable, _amplifyArrayBufferForTests as amplifyArrayBuffer, } from '../src/lib.js'; +import { emulatedOnlyTest } from './_emulated-only.js'; const { getPrototypeOf } = Object; @@ -23,9 +24,9 @@ test('emulated immutable inherits directly from ArrayBuffer.prototype', t => { t.true(iab.immutable); }); -test('Object.prototype.toString.call(immuAB) reads as ImmutableArrayBuffer', t => { +emulatedOnlyTest('Object.prototype.toString.call(immuAB) reads as ImmutableArrayBuffer', t => { const iab = new ArrayBuffer(2).sliceToImmutable(); - // Per DESIGN.md § Move 2 paragraph 7 (as amended for the design-departure + // Per designs/immutable-arraybuffer.md section Move 2 paragraph 7 (as amended for the design-departure // recorded in the same paragraph), the `[Symbol.toStringTag]` slot is // installed as an own property on each emulated immutable buffer (not on // the shared ArrayBuffer.prototype). Genuine ArrayBuffers continue to diff --git a/packages/immutable-arraybuffer/test/shim-slice.test.js b/packages/immutable-arraybuffer/test/shim-slice.test.js index bc5fc26fcb..1b50c4870d 100644 --- a/packages/immutable-arraybuffer/test/shim-slice.test.js +++ b/packages/immutable-arraybuffer/test/shim-slice.test.js @@ -100,27 +100,26 @@ test('Standard TypedArray behavior baseline', t => { t.is(ta2.byteLength, 0); }); -// This could have been written as a test.failing as compared to -// the immutable ArrayBuffer we'll propose. However, I'd rather test what // the shim purposely does instead. -test('TypedArray on Immutable ArrayBuffer shim limitations', t => { +test('TypedArray on Immutable ArrayBuffer: freezable-TypedArray emulation now supported', t => { + // As of the freezable-TypedArray shim (PR implementing + // designs/freezable-typedarray.md), calling a TypedArray constructor with + // `sliceToImmutable()`'s result produces an emulated freezable wrapper. + // The old limitation (producing a 0-byte TypedArray) no longer applies. const ab1 = new ArrayBuffer(2); - const dv1 = new DataView(ab1); - t.is(dv1.buffer, ab1); - t.is(dv1.byteLength, 2); const ta1 = new Uint8Array(ab1); ta1[0] = 3; ta1[1] = 4; - t.is(ta1.byteLength, 2); const iab = ab1.sliceToImmutable(); - // Unfortunately, unlike the immutable ArrayBuffer to be proposed, - // calling a TypedArray constructor with the shim implementation of - // an immutable ArrayBuffer as argument treats it as an unrecognized object, - // rather than throwing an error or acting as a non-changeable TypedArray. t.is(iab.byteLength, 2); const ta3 = new Uint8Array(iab); - t.is(ta3.byteLength, 0); + // The emulated freezable wrapper covers the full 2 bytes. + t.is(ta3.byteLength, 2); + // Mutators throw on the emulated freezable wrapper. + t.throws(() => ta3.set([0, 0]), { instanceOf: TypeError }); + // The buffer accessor returns the immutable wrapper. + t.is(ta3.buffer, iab); }); const testTransfer = t => { diff --git a/packages/immutable-arraybuffer/test/shim-transfer.test.js b/packages/immutable-arraybuffer/test/shim-transfer.test.js index 906d943e78..27bbbb1c24 100644 --- a/packages/immutable-arraybuffer/test/shim-transfer.test.js +++ b/packages/immutable-arraybuffer/test/shim-transfer.test.js @@ -100,27 +100,32 @@ test('Standard TypedArray behavior baseline', t => { t.is(ta2.byteLength, 0); }); -// This could have been written as a test.failing as compared to -// the immutable ArrayBuffer we'll propose. However, I'd rather test what // the shim purposely does instead. -test('TypedArray on Immutable ArrayBuffer shim limitations', t => { +test('TypedArray on Immutable ArrayBuffer: freezable-TypedArray emulation now supported', t => { + // As of the freezable-TypedArray shim (PR implementing + // designs/freezable-typedarray.md), calling a TypedArray constructor with + // `transferToImmutable()`'s result produces an emulated freezable wrapper. + // The old limitation (producing a 0-byte TypedArray) no longer applies. + if (!('transferToImmutable' in ArrayBuffer.prototype)) { + t.pass( + 'Platform lacks transferToImmutable; skip transferToImmutable coverage', + ); + return; + } const ab1 = new ArrayBuffer(2); - const dv1 = new DataView(ab1); - t.is(dv1.buffer, ab1); - t.is(dv1.byteLength, 2); const ta1 = new Uint8Array(ab1); ta1[0] = 3; ta1[1] = 4; - t.is(ta1.byteLength, 2); const iab = ab1.transferToImmutable(); - // Unfortunately, unlike the immutable ArrayBuffer to be proposed, - // calling a TypedArray constructor with the shim implementation of - // an immutable ArrayBuffer as argument treats it as an unrecognized object, - // rather than throwing an error or acting as a non-changeable TypedArray. t.is(iab.byteLength, 2); const ta3 = new Uint8Array(iab); - t.is(ta3.byteLength, 0); + // The emulated freezable wrapper covers the full 2 bytes. + t.is(ta3.byteLength, 2); + // Mutators throw on the emulated freezable wrapper. + t.throws(() => ta3.set([0, 0]), { instanceOf: TypeError }); + // The buffer accessor returns the immutable wrapper. + t.is(ta3.buffer, iab); }); const testTransfer = t => { diff --git a/packages/immutable-arraybuffer/test/shim-typedarray-per-flavor.test.js b/packages/immutable-arraybuffer/test/shim-typedarray-per-flavor.test.js new file mode 100644 index 0000000000..3d36f5c35a --- /dev/null +++ b/packages/immutable-arraybuffer/test/shim-typedarray-per-flavor.test.js @@ -0,0 +1,220 @@ +// @ts-nocheck +// Per-flavor parameterized coverage for the freezable-TypedArray emulation. +// Runs a matrix of assertions over all eleven concrete TypedArray constructors. +// +// Per-flavor sample values: +// - Non-BigInt flavors: 1 (numeric) +// - BigInt flavors (BigInt64Array, BigUint64Array): 1n (BigInt) +// +// The BigInt distinction matters because the native TypedArray operations throw +// a TypeError on a type-mismatch *before* reaching the brand check, which +// would mask the mutator-throws path the tests are verifying. +import '../src/shim.js'; +import test from 'ava'; +import { emulatedOnlyTest } from './_emulated-only.js'; + +const { getPrototypeOf, freeze, isFrozen } = Object; + +/** + * @type {Array<{name: string, Ctor: Function, sample: number|bigint, zero: number|bigint}>} + */ +const flavors = [ + { name: 'Int8Array', Ctor: Int8Array, sample: 1, zero: 0 }, + { name: 'Int16Array', Ctor: Int16Array, sample: 1, zero: 0 }, + { name: 'Int32Array', Ctor: Int32Array, sample: 1, zero: 0 }, + { name: 'Uint8Array', Ctor: Uint8Array, sample: 1, zero: 0 }, + { + name: 'Uint8ClampedArray', + Ctor: Uint8ClampedArray, + sample: 1, + zero: 0, + }, + { name: 'Uint16Array', Ctor: Uint16Array, sample: 1, zero: 0 }, + { name: 'Uint32Array', Ctor: Uint32Array, sample: 1, zero: 0 }, + { name: 'Float32Array', Ctor: Float32Array, sample: 1, zero: 0 }, + { name: 'Float64Array', Ctor: Float64Array, sample: 1, zero: 0 }, + { name: 'BigInt64Array', Ctor: BigInt64Array, sample: 1n, zero: 0n }, + { name: 'BigUint64Array', Ctor: BigUint64Array, sample: 1n, zero: 0n }, +]; + +for (const { name, Ctor, sample, zero } of flavors) { + const tName = label => `${name}: ${label}`; + + // ------------------------------------------------------------------------- + // Construction from an immutable buffer + // ------------------------------------------------------------------------- + + test( + tName( + 'construction from an immutable buffer succeeds; __proto__ is T.prototype', + ), + t => { + const ab = new ArrayBuffer(16); + const iab = ab.sliceToImmutable(); + const view = new Ctor(iab); + t.is(getPrototypeOf(view), Ctor.prototype); + t.true(view instanceof Ctor); + }, + ); + + // ------------------------------------------------------------------------- + // Mutator methods throw TypeError on frozen wrappers + // ------------------------------------------------------------------------- + + test(tName('copyWithin throws TypeError on emulated freezable view'), t => { + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + t.throws(() => view.copyWithin(0, 1), { instanceOf: TypeError }); + }); + + test(tName('fill throws TypeError on emulated freezable view'), t => { + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + t.throws(() => view.fill(sample), { instanceOf: TypeError }); + }); + + test(tName('reverse throws TypeError on emulated freezable view'), t => { + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + t.throws(() => view.reverse(), { instanceOf: TypeError }); + }); + + test(tName('set throws TypeError on emulated freezable view'), t => { + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + t.throws(() => view.set([sample]), { instanceOf: TypeError }); + }); + + test(tName('sort throws TypeError on emulated freezable view'), t => { + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + t.throws(() => view.sort(), { instanceOf: TypeError }); + }); + + // ------------------------------------------------------------------------- + // Indexed assignment does not modify the underlying buffer + // ------------------------------------------------------------------------- + + emulatedOnlyTest( + tName( + 'indexed assignment on non-frozen wrapper creates own property; buffer unchanged', + ), + t => { + const ab = new ArrayBuffer(16); + const iab = ab.sliceToImmutable(); + const view = new Ctor(iab); + + // Byte 0 is the per-flavor zero before assignment. + t.is(Ctor.prototype.at.call(view, 0), zero); + + // Assign — creates an own property on the plain wrapper. + view[0] = sample; + + // The own property reads back. + t.is(view[0], sample); + + // The underlying buffer's byte 0 is still zero. + t.is(Ctor.prototype.at.call(view, 0), zero); + }, + ); + + emulatedOnlyTest( + tName( + 'indexed assignment on frozen wrapper throws in strict mode; buffer unchanged', + ), + t => { + // ES modules run in strict mode. In strict mode, assigning to a frozen + // ordinary object throws TypeError. In non-strict mode the assignment + // would be silently swallowed. Either way the underlying buffer is + // unchanged. See designs/freezable-typedarray.md, frozen-wrapper example. + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + freeze(view); + + // In strict mode (ES module), assigning to a frozen object throws. + t.throws( + () => { + view[0] = sample; + }, + { instanceOf: TypeError }, + ); + + // The underlying buffer's byte 0 is still zero. + t.is(Ctor.prototype.at.call(view, 0), zero); + }, + ); + + // ------------------------------------------------------------------------- + // Read-only surface + // ------------------------------------------------------------------------- + + test(tName('byteLength, byteOffset, length return correct values'), t => { + const ab = new ArrayBuffer(16); + const iab = ab.sliceToImmutable(); + const view = new Ctor(iab); + // byteLength and length are flavor-dependent; byteOffset is always 0 here. + t.is(view.byteOffset, 0); + t.is(view.byteLength, 16); + t.is(view.buffer, iab); + }); + + test(tName('at(0) returns correct value'), t => { + const ab = new ArrayBuffer(16); + const iab = ab.sliceToImmutable(); + const view = new Ctor(iab); + t.is(view.at(0), zero); + }); + + test( + tName('with(0, sample), toReversed, toSorted return correct values'), + t => { + const ab = new ArrayBuffer(16); + const iab = ab.sliceToImmutable(); + const view = new Ctor(iab); + + // `with` is a non-mutating method that returns a new TypedArray. + // On an emulated freezable wrapper that has no indexed slots, `view.with(0, sample)` + // delegates via the amplifier to the hidden genuine TypedArray. + const withResult = view.with(0, sample); + // The result is a new TypedArray (not the wrapper itself). + t.not(withResult, view); + + // `toReversed` and `toSorted` are non-mutating; they return new TypedArrays. + const reversed = view.toReversed(); + t.not(reversed, view); + + const sorted = view.toSorted(); + t.not(sorted, view); + }, + ); + + // ------------------------------------------------------------------------- + // Object.freeze + // ------------------------------------------------------------------------- + + test(tName('Object.freeze(view); Object.isFrozen(view) === true'), t => { + const iab = new ArrayBuffer(16).sliceToImmutable(); + const view = new Ctor(iab); + freeze(view); + t.true(isFrozen(view)); + }); + + // ------------------------------------------------------------------------- + // Fallthrough constructor (genuine mutable buffer) + // ------------------------------------------------------------------------- + + test( + tName( + 'fallthrough constructor on genuine mutable buffer produces genuine writable view', + ), + t => { + const realAb = new ArrayBuffer(16); + const view = new Ctor(realAb); + t.is(getPrototypeOf(view), Ctor.prototype); + t.is(view.buffer, realAb); + // Write succeeds. + view[0] = sample; + t.is(view[0], sample); + }, + ); +} diff --git a/packages/immutable-arraybuffer/test/shim-typedarray-tostringtag.test.js b/packages/immutable-arraybuffer/test/shim-typedarray-tostringtag.test.js new file mode 100644 index 0000000000..49ac89a7e6 --- /dev/null +++ b/packages/immutable-arraybuffer/test/shim-typedarray-tostringtag.test.js @@ -0,0 +1,109 @@ +// @ts-nocheck +// Shim-level regression tests pinning the `[Symbol.toStringTag]` fidelity +// contract of the freezable-TypedArray emulation. +// +// The shim replaces the genuine, `this`-sensitive +// `%TypedArrayPrototype%[Symbol.toStringTag]` getter with a wrapper around it. +// On an emulated freezable wrapper (a plain ordinary object created with +// `Object.create(Uint8Array.prototype)`, which has no `[[TypedArrayName]]` +// internal slot) the wrapper getter amplifies to the hidden genuine TypedArray +// and reads *its* tag, so `Object.prototype.toString.call(wrapper)` reads +// `'[object Uint8Array]'`, matching a genuine view. On a genuine TypedArray the +// wrapper falls through to the genuine getter; on any other receiver the genuine +// getter returns `undefined`, exactly as before. +// +// This is the getter-wrapper fidelity fix requested in erights's review of +// endojs/endo-but-for-bots#475 (review comments 3817252816 / 3817264546). It is +// a higher-fidelity repair than installing a `[Symbol.toStringTag]` *data* +// property, which would patch only the `Object.prototype.toString` lookup path +// and leave the getter itself still reporting `undefined` on a wrapper — the +// getter and `Object.prototype.toString` now agree instead. +// +// Consequently `[Symbol.toStringTag]` is NO LONGER an emulated-vs-genuine +// distinguisher. The single committed distinguisher remains `ArrayBuffer.isView` +// (pinned in `shim-typedarray.test.js`); downstream clients (`@endo/bytes` / +// `@endo/pass-style`) tell an emulated wrapper apart from a genuine `Uint8Array` +// via `isView`, never by sniffing `toStringTag`. +import '../src/shim.js'; +import test from 'ava'; + +const { getPrototypeOf, getOwnPropertyDescriptor } = Object; +const { apply } = Reflect; + +// After the shim installs, `%TypedArrayPrototype%[Symbol.toStringTag]` is the +// shim's wrapper getter (still an accessor with a getter function, not a data +// property). +const typedArrayPrototype = getPrototypeOf(Uint8Array.prototype); +const tagGetterDesc = getOwnPropertyDescriptor( + typedArrayPrototype, + Symbol.toStringTag, +); +const shimTagGetter = tagGetterDesc.get; + +const makeEmulatedWrapper = length => { + const ab = new ArrayBuffer(length); + // NOTE: `Array.from(arrayLike, mapFn)` relies on the map (relation) function. + // There is a known XS defect where `Array.from` does not recognize the map + // function argument; were this helper exercised under test262 on an affected + // XS build, the fill would not run. It is inconsequential here (the fill only + // seeds distinct bytes for identity assertions), but flagged so a future + // test262 run of these shim-path cases is not surprised by it. + new Uint8Array(ab).set(Array.from({ length }, (_, i) => i + 1)); + const iab = ab.sliceToImmutable(); + return new Uint8Array(iab); +}; + +test('shim installs a getter (not a data property) for %TypedArrayPrototype% toStringTag', t => { + // The replacement remains an accessor with a getter function — the + // getter-wrapper fix, NOT a `[Symbol.toStringTag]` data property (which would + // be the flawed, lower-fidelity repair). It still reports the genuine tag for + // a genuine view. + t.is(typeof shimTagGetter, 'function'); + t.is(tagGetterDesc.set, undefined); + t.false('value' in tagGetterDesc); + t.is(apply(shimTagGetter, new Uint8Array(3), []), 'Uint8Array'); +}); + +test('emulated freezable wrapper carries no own [Symbol.toStringTag]', t => { + // The tag is supplied by the prototype's wrapper getter, not by an own data + // property on the wrapper — the distinction between the getter-wrapper fix and + // the flawed data-property fix. + const wrapper = makeEmulatedWrapper(4); + t.is(getOwnPropertyDescriptor(wrapper, Symbol.toStringTag), undefined); +}); + +test('shim toStringTag getter reports Uint8Array for an emulated wrapper', t => { + // The wrapper getter amplifies the emulated wrapper to its hidden genuine + // TypedArray and reads that TypedArray's internal-slot tag, so the getter now + // reports the flavor name instead of `undefined`. + const wrapper = makeEmulatedWrapper(4); + t.is(apply(shimTagGetter, wrapper, []), 'Uint8Array'); + t.is(apply(shimTagGetter, new Uint8Array(4), []), 'Uint8Array'); +}); + +test('shim toStringTag getter returns undefined for a non-TypedArray receiver', t => { + // Fallthrough is preserved: on a receiver that is neither an emulated wrapper + // nor a genuine TypedArray, the amplifier returns the receiver unchanged and + // the genuine getter returns `undefined`. + t.is(apply(shimTagGetter, {}, []), undefined); + t.is(apply(shimTagGetter, [], []), undefined); +}); + +test('Object.prototype.toString reads emulated wrapper as its TypedArray flavor', t => { + const wrapper = makeEmulatedWrapper(4); + // The emulated wrapper now reads as a Uint8Array, matching a genuine view — + // the toStringTag fidelity gap is closed. (The committed distinguisher is + // `ArrayBuffer.isView`, not this.) + t.is(Object.prototype.toString.call(wrapper), '[object Uint8Array]'); + t.is( + Object.prototype.toString.call(new Uint8Array(4)), + '[object Uint8Array]', + ); +}); + +test('the toStringTag fidelity holds after freezing the emulated wrapper', t => { + const wrapper = Object.freeze(makeEmulatedWrapper(4)); + t.true(Object.isFrozen(wrapper)); + t.is(apply(shimTagGetter, wrapper, []), 'Uint8Array'); + t.is(Object.prototype.toString.call(wrapper), '[object Uint8Array]'); +}); diff --git a/packages/immutable-arraybuffer/test/shim-typedarray.test.js b/packages/immutable-arraybuffer/test/shim-typedarray.test.js new file mode 100644 index 0000000000..ab7a4ee0fe --- /dev/null +++ b/packages/immutable-arraybuffer/test/shim-typedarray.test.js @@ -0,0 +1,337 @@ +// @ts-nocheck +// Shim-level integration tests for the freezable-TypedArray emulation. +// These tests exercise the shim-installed pseudo-constructors and +// %TypedArrayPrototype% property record after the full shim install. +import '../src/shim.js'; +import test from 'ava'; +import { emulatedOnlyTest } from './_emulated-only.js'; + +const { getPrototypeOf, freeze, isFrozen, keys, getOwnPropertyDescriptor } = + Object; + +// --------------------------------------------------------------------------- +// Basic construction +// --------------------------------------------------------------------------- + +test('shim: global Uint8Array on an immutable ArrayBuffer wraps as emulated freezable', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([1, 2, 3, 4]); + const iab = ab.sliceToImmutable(); + + const view = new Uint8Array(iab); + + // The wrapper's prototype is Uint8Array.prototype (no intermediate prototype). + t.is(getPrototypeOf(view), Uint8Array.prototype); + t.true(view instanceof Uint8Array); + + // `view.buffer` returns the immutable wrapper, not the genuine backing buffer. + t.is(view.buffer, iab); + t.true(view.buffer.immutable); +}); + +test('shim: global Uint8Array on a regular ArrayBuffer forwards to the OriginalConstructor', t => { + const realAb = new ArrayBuffer(4); + new Uint8Array(realAb).set([10, 20, 30, 40]); + + const view = new Uint8Array(realAb); + + // Fallthrough path: genuine TypedArray. + t.is(view.buffer, realAb); + t.false(view.buffer.immutable); + + // Mutators succeed on the genuine view. + view[0] = 99; + t.is(view[0], 99); +}); + +// --------------------------------------------------------------------------- +// `view.buffer` getter +// --------------------------------------------------------------------------- + +test('shim: virtual buffer getter returns the real buffer for a genuine TypedArray', t => { + const realAb = new ArrayBuffer(4); + const view = new Uint8Array(realAb); + t.is(view.buffer, realAb); +}); + +test('shim: virtual buffer getter redirects to the immutable wrapper when present', t => { + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + t.is(view.buffer, iab); + t.true(view.buffer.immutable); +}); + +// --------------------------------------------------------------------------- +// Mutators throw on emulated freezable views +// --------------------------------------------------------------------------- + +test('shim: emulated freezable mutators complain', t => { + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + + t.throws(() => view.copyWithin(0, 1), { instanceOf: TypeError }); + t.throws(() => view.fill(0), { instanceOf: TypeError }); + t.throws(() => view.reverse(), { instanceOf: TypeError }); + t.throws(() => view.set([0]), { instanceOf: TypeError }); + t.throws(() => view.sort(), { instanceOf: TypeError }); +}); + +// --------------------------------------------------------------------------- +// Read-only delegations (`byteLength`, `at`, `length`, `byteOffset`) +// --------------------------------------------------------------------------- + +test('shim: emulated freezable byteLength and at redirect via amplifyTypedArray', t => { + const ab = new ArrayBuffer(8); + new Uint8Array(ab).set([10, 20, 30, 40, 50, 60, 70, 80]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + t.is(view.byteLength, 8); + t.is(view.length, 8); + t.is(view.byteOffset, 0); + t.is(view.at(0), 10); + t.is(view.at(7), 80); +}); + +// --------------------------------------------------------------------------- +// `subarray` returns a view whose `buffer` is the immutable wrapper +// --------------------------------------------------------------------------- + +test('shim: emulated freezable subarray returns a wrapped view whose buffer is the immutable wrapper', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([1, 2, 3, 4]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + const sub = view.subarray(1, 3); + // `subarray` on an emulated wrapper now returns a new emulated wrapper + // backed by the sub-view of the hidden genuine TypedArray. The safety + // contract (`sub.buffer === iab`) is preserved: the sub-view's `.buffer` + // redirects to the same immutable ArrayBuffer wrapper as the parent view. + t.is(sub.byteLength, 2); + t.is(sub.byteOffset, 1); + // Indexed element access uses `at()` (the amplifier-delegate path) rather + // than `sub[0]` (which would read an own property on the plain wrapper object, + // returning `undefined` for unset indices, per the wrapper semantics). + t.is(sub.at(0), 2); + t.is(sub.at(1), 3); + // Core safety-contract assertion: the sub-view's buffer is the immutable wrapper. + t.is(sub.buffer, iab); + t.true(sub.buffer.immutable); + // Chained subarray must also preserve the immutable buffer reference. + t.is(view.subarray(0, 2).subarray(0, 1).buffer, iab); +}); + +// --------------------------------------------------------------------------- +// Symbol.iterator: for...of and spread work on emulated freezable wrappers +// --------------------------------------------------------------------------- + +test('shim: for...of loop works on an emulated freezable wrapper', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([10, 20, 30, 40]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + const collected = []; + for (const v of view) { + collected.push(v); + } + t.deepEqual(collected, [10, 20, 30, 40]); +}); + +test('shim: spread syntax works on an emulated freezable wrapper', t => { + const ab = new ArrayBuffer(3); + new Uint8Array(ab).set([7, 8, 9]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + t.deepEqual([...view], [7, 8, 9]); +}); + +test('shim: Symbol.iterator on %TypedArrayPrototype% matches the values wrapper after shim install', t => { + // After the shim installs a `values` wrapper on %TypedArrayPrototype%, the + // `Symbol.iterator` slot must point at the same (or equivalent) wrapper, not + // the original genuine `values` function. This regression test pins the fix: + // if `Symbol.iterator` is left pointing at the original genuine function, + // `for...of` on a freezable wrapper throws `TypeError: this is not a typed array.` + const ab = new ArrayBuffer(2); + new Uint8Array(ab).set([1, 2]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + // Both iteration protocols must work on an emulated freezable wrapper. + t.deepEqual([...view.values()], [1, 2]); + const iterResult = []; + for (const v of view) { + iterResult.push(v); + } + t.deepEqual(iterResult, [1, 2]); +}); + +// --------------------------------------------------------------------------- +// detect-then-skip is idempotent under re-import +// --------------------------------------------------------------------------- + +test('shim: detect-then-skip is idempotent under re-import', async t => { + // The gate is keyed on `'sliceToImmutable' in ArrayBuffer.prototype`. + // A second import of the shim must not overwrite the already-installed surface. + const sliceFnBefore = ArrayBuffer.prototype.sliceToImmutable; + + // Dynamic re-import exercises the gate from a fresh module invocation. + await import('../src/shim.js'); + + t.is( + ArrayBuffer.prototype.sliceToImmutable, + sliceFnBefore, + 'second shim import did not replace the already-installed sliceToImmutable', + ); +}); + +// --------------------------------------------------------------------------- +// Indexed assignment semantics (proposal-level constraint) +// --------------------------------------------------------------------------- + +emulatedOnlyTest('shim: indexed assignment on a non-frozen emulated freezable view creates a wrapper-local own property; the underlying immutable buffer is unchanged', t => { + const ab = new ArrayBuffer(4); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + // The underlying buffer's byte 0 is 0. + t.is(Uint8Array.prototype.at.call(view, 0), 0); + + // Indexed assignment performs OrdinarySet on the plain wrapper, creating an + // own data property '0' => 42. The underlying buffer is not touched. + view[0] = 42; + + // `view[0]` now reads the own property. + t.is(view[0], 42); + + // But the underlying buffer's byte 0 is still 0. + t.is(Uint8Array.prototype.at.call(view, 0), 0); +}); + +emulatedOnlyTest('shim: indexed assignment on a frozen emulated freezable view throws in strict mode; the underlying immutable buffer is unchanged', t => { + // ES modules are implicitly strict. In strict mode, an indexed assignment + // to a frozen ordinary object throws TypeError ("Cannot add property 0, + // object is not extensible"). In non-strict mode the same assignment would + // be silently swallowed. Both behaviors leave the underlying immutable + // buffer unchanged; the proposal's buffer-immutability guarantee holds + // regardless of mode. See designs/freezable-typedarray.md section + // "Indexed assignment never modifies the underlying buffer", frozen example. + const ab = new ArrayBuffer(4); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + freeze(view); + t.true(isFrozen(view)); + + // In strict mode (ES module), assigning to a frozen object throws. + t.throws( + () => { + view[0] = 42; + }, + { instanceOf: TypeError }, + ); + + // The underlying buffer's byte 0 is still 0 (unchanged regardless of mode). + t.is(Uint8Array.prototype.at.call(view, 0), 0); +}); + +// --------------------------------------------------------------------------- +// The one committed emulated-vs-genuine fidelity loss: `ArrayBuffer.isView` +// +// An emulated freezable wrapper is a plain ordinary object with no +// `[[ViewedArrayBuffer]]` / `[[TypedArrayName]]` internal slots, so +// `ArrayBuffer.isView(wrapper)` is `false`, whereas a genuine `Uint8Array` +// (mutable, or native-immutable) reports `true`. This is the single +// distinguisher `@endo/bytes` and `@endo/pass-style` are entitled to rely on +// to tell an emulated wrapper apart from a genuine integer-indexed view, and +// the one the shim commits to preserve. This test fails first if a future +// change ever made an emulated wrapper report `isView === true`. See README +// "The one committed fidelity loss: an emulated wrapper is not +// `ArrayBuffer.isView`". +// --------------------------------------------------------------------------- + +emulatedOnlyTest('shim: emulated freezable wrapper is not ArrayBuffer.isView; a genuine view is (the committed fidelity loss)', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([10, 20, 30, 40]); + const iab = ab.sliceToImmutable(); + const emulated = new Uint8Array(iab); + + // The committed distinguisher: the emulated wrapper is not a view. + t.false(ArrayBuffer.isView(emulated)); + // A genuine mutable view is a view. + t.true(ArrayBuffer.isView(new Uint8Array(4))); + // The distinction survives freezing the wrapper. + t.false(ArrayBuffer.isView(freeze(emulated))); +}); + +// --------------------------------------------------------------------------- +// Indexed read semantics (an incidental consequence of the plain-object shape) +// +// Symmetric to the indexed-assignment constraint above: an integer-indexed +// *read* `view[i]` on a fresh emulated freezable wrapper returns `undefined`, +// never the underlying byte. The wrapper is a plain ordinary object whose +// prototype is `Uint8Array.prototype`; it carries no own indexed properties, +// and the shim installs no integer-indexed read accessor on +// %TypedArrayPrototype% that could intercept `view[i]` (the TC39 proposal +// offers no way to do so through the prototype chain). Bytes are readable only +// through the integer-indexed protocol (`view.at(i)`, `for..of`, spread). +// +// This `view[i] === undefined` behavior is a real but INCIDENTAL consequence +// of the wrapper being a plain object — the same plain-object nature that +// makes `ArrayBuffer.isView` report `false` (pinned above). It is NOT the +// committed distinguisher: `@endo/bytes` and `@endo/pass-style` discriminate +// via `ArrayBuffer.isView`, not by sniffing `view[i]`. This test records the +// companion observation (and the zero-own-index shape `@endo/pass-style` +// requires of an emulated wrapper). See README "Integer-indexed reads on +// emulated freezable views (an incidental consequence)". +// --------------------------------------------------------------------------- + +emulatedOnlyTest('shim: integer-indexed read on a fresh emulated freezable wrapper is undefined, not the underlying byte (incidental consequence)', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([10, 20, 30, 40]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + + // The bytes are readable through the integer-indexed protocol, which the + // shim redirects to the hidden genuine TypedArray. + t.is(view.at(0), 10); + t.is(view.at(3), 40); + + // But a direct integer-indexed read yields `undefined`, never the byte. + // Non-zero source bytes make this unambiguous: a coincidental 0 cannot + // masquerade as the "no such property" answer. + t.is(view[0], undefined); + t.is(view[1], undefined); + t.is(view[2], undefined); + t.is(view[3], undefined); + + // The wrapper carries no own indexed properties at all: the shape + // `@endo/pass-style` requires of an emulated (non-view) `byteArray` wrapper. + t.deepEqual(keys(view), []); + t.is(getOwnPropertyDescriptor(view, 0), undefined); +}); + +// --------------------------------------------------------------------------- +// Object.freeze + Object.isFrozen (the proposal's TypedArray-can-be-frozen +// guarantee) +// --------------------------------------------------------------------------- + +test('shim: Object.freeze(view); Object.isFrozen(view) === true', t => { + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + + freeze(view); + t.true(isFrozen(view)); +}); + +// --------------------------------------------------------------------------- +// No intermediate prototype +// --------------------------------------------------------------------------- + +test('shim: Object.getPrototypeOf(view) === Uint8Array.prototype on an emulated freezable view', t => { + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + t.is(getPrototypeOf(view), Uint8Array.prototype); +}); diff --git a/packages/immutable-arraybuffer/tsconfig.composite.json b/packages/immutable-arraybuffer/tsconfig.composite.json index d57b08c653..5628aa177c 100644 --- a/packages/immutable-arraybuffer/tsconfig.composite.json +++ b/packages/immutable-arraybuffer/tsconfig.composite.json @@ -4,5 +4,9 @@ "compilerOptions": { "composite": true }, - "references": [] + "references": [ + { + "path": "../harden/tsconfig.composite.json" + } + ] } diff --git a/packages/marshal/docs/smallcaps-cheatsheet.md b/packages/marshal/docs/smallcaps-cheatsheet.md index b00f736b90..e82b7725d2 100644 --- a/packages/marshal/docs/smallcaps-cheatsheet.md +++ b/packages/marshal/docs/smallcaps-cheatsheet.md @@ -10,7 +10,7 @@ An example-based summary of the Smallcaps encoding of the OCapN [Abstract Syntax | bigint | Integer | `7n`
`-7n` | `"+7"`
`"-7"` | | number | Float64 | `Infinity`
`-Infinity`
`NaN`
`-0`
`7.1` | `"#Infinity"`
`"#-Infinity"`
`"#NaN"`
`"#-0"` // unimplemented
`7.1` | | string | String | `'#foo'`
`'foo'` | `"!#foo"` // special strings
`"foo"` // other strings | -| byteArray | ByteArray | `buf.toImmutable()` | // undecided & unimplemented | +| byteArray | ByteArray | `frozenBytes(bytes)` | `"*b0b5cafe"` // after `*`, hex encoding | | passable symbols | Symbol | `passableSymbolForName('foo')` | `"%foo"` // in transition | | copyArray | List | `[a,b]` | `[,]` | | copyRecord | Struct | `{foo:a,'#foo':b}` | `{"!#foo":,"foo":}` // keys sorted | @@ -28,7 +28,6 @@ An example-based summary of the Smallcaps encoding of the OCapN [Abstract Syntax * Structs [can only have string-named properties](https://github.com/endojs/endo/blob/master/packages/pass-style/doc/copyRecord-guarantees.md). * Errors can also carry an optional `errorId` string property. * We expect to expand the optional error properties over time. -* The ByteArray encoding is not yet designed or implemented. ## Readability Invariants diff --git a/packages/marshal/package.json b/packages/marshal/package.json index 79757172e1..f38a9486c1 100644 --- a/packages/marshal/package.json +++ b/packages/marshal/package.json @@ -43,11 +43,14 @@ }, "homepage": "https://github.com/endojs/endo#readme", "dependencies": { + "@endo/bytes": "workspace:^", "@endo/common": "workspace:^", "@endo/env-options": "workspace:^", "@endo/errors": "workspace:^", "@endo/eventual-send": "workspace:^", "@endo/harden": "workspace:^", + "@endo/hex": "workspace:^", + "@endo/immutable-arraybuffer": "workspace:^", "@endo/nat": "workspace:^", "@endo/pass-style": "workspace:^" }, diff --git a/packages/marshal/src/encodePassable.js b/packages/marshal/src/encodePassable.js index 645d0820d5..cf75de47f7 100644 --- a/packages/marshal/src/encodePassable.js +++ b/packages/marshal/src/encodePassable.js @@ -11,6 +11,8 @@ import { nameForPassableSymbol, passableSymbolForName, } from '@endo/pass-style'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; +import { encodeHex, decodeHex } from '@endo/hex'; /** * @import {CopyRecord, PassStyle, Passable, RemotableObject, ByteArray} from '@endo/pass-style' @@ -473,14 +475,50 @@ const decodeLegacyArray = (encoded, decodePassable, skip = 0) => { }; /** + * Encodes a ByteArray as `a<:>`, where `` is the + * `encodeBigInt` of `byteLength` (a non-negative bigint, which already + * encodes in shortlex-numerical order), followed by the explicit + * separator `:` and the lowercase hex of the bytes. + * + * Shortlex is inherited in two stages: + * 1. `encodeBigInt` on the length preserves numerical order, so shorter + * byteArrays sort before longer ones. + * 2. At equal length, the length encodings are identical and ordering + * falls through to the byte-lex-preserving hex body. + * + * Every character used here — `a`, `p`/`n`/`~`/`#`, decimal digits, `:`, + * and hex digits `[0-9a-f]` — is outside the escape-reserved sets of + * both the `legacyOrdered` and `compactOrdered` array framings, so no + * escaping is needed. + * * @param {ByteArray} byteArray * @param {(byteArray: ByteArray) => string} _encodePassable * @returns {string} */ const encodeByteArray = (byteArray, _encodePassable) => { - // TODO implement - Fail`encodePassable(byteArray) not yet implemented: ${byteArray}`; - return ''; // Just for the type + const lenEnc = encodeBigInt(BigInt(byteArray.byteLength)); + return `a${lenEnc}:${encodeHex(thawedBytes(byteArray))}`; +}; + +// `byteLength` is non-negative, so the Elias-delta length prefix from +// `encodeBigInt` always starts with `p` (the non-negative bigint sigil), +// never `n`. +const rByteArrayPayload = /^(p[~]*[0-9]+:[0-9]+):([0-9a-f]*)$/; + +/** + * Inverse of {@link encodeByteArray}. + * + * @param {string} encoded The body after the leading `'a'` prefix char. + * @returns {ByteArray} + */ +const decodeByteArray = encoded => { + const match = encoded.match(rByteArrayPayload); + match || Fail`Encoded byteArray expected: ${encoded}`; + const [, lenEnc, hex] = /** @type {RegExpMatchArray} */ (match); + const byteLength = Number(decodeBigInt(lenEnc)); + hex.length === byteLength * 2 || + Fail`byteArray length mismatch: header ${q(byteLength)} vs body ${q(hex.length / 2)}`; + return frozenBytes(decodeHex(hex, 'encodePassable byteArray')); }; const encodeRecord = (record, encodeArray, encodePassable) => { @@ -749,6 +787,9 @@ const makeInnerDecode = (decodeStringSuffix, decodeArray, options) => { case ':': { return decodeTagged(encoded, decodeArray, innerDecode, skip); } + case 'a': { + return decodeByteArray(getSuffix(encoded, skip + 1)); + } default: { throw Fail`invalid database key: ${getSuffix(encoded, skip)}`; } diff --git a/packages/marshal/src/encodeToCapData.js b/packages/marshal/src/encodeToCapData.js index 6dd14e9c27..a5f7dbb336 100644 --- a/packages/marshal/src/encodeToCapData.js +++ b/packages/marshal/src/encodeToCapData.js @@ -15,6 +15,8 @@ import { nameForPassableSymbol, passableSymbolForName, } from '@endo/pass-style'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; +import { encodeHex, decodeHex } from '@endo/hex'; import { X, Fail, q } from '@endo/errors'; /** @import {Passable, RemotableObject} from '@endo/pass-style' */ @@ -194,8 +196,10 @@ export const makeEncodeToCapData = (encodeOptions = {}) => { return passable.map(encodeToCapDataRecur); } case 'byteArray': { - // TODO implement - throw Fail`marsal of byteArray not yet implemented: ${passable}`; + return { + [QCLASS]: 'byteArray', + data: encodeHex(thawedBytes(passable)), + }; } case 'tagged': { return { @@ -367,6 +371,12 @@ export const makeDecodeFromCapData = (decodeOptions = {}) => { const { tag, payload } = jsonEncoded; return makeTagged(tag, decodeFromCapData(payload)); } + case 'byteArray': { + const { data } = jsonEncoded; + typeof data === 'string' || + Fail`invalid byteArray data typeof ${q(typeof data)}`; + return frozenBytes(decodeHex(data, 'capData byteArray')); + } case 'slot': { // See note above about how the current encoding cannot reliably // distinguish which we should call, so in the non-default case @@ -441,3 +451,4 @@ export const makeDecodeFromCapData = (decodeOptions = {}) => { }; return harden(decodeFromCapData); }; +harden(makeDecodeFromCapData); diff --git a/packages/marshal/src/encodeToSmallcaps.js b/packages/marshal/src/encodeToSmallcaps.js index 3421c29ca5..2e40045d4c 100644 --- a/packages/marshal/src/encodeToSmallcaps.js +++ b/packages/marshal/src/encodeToSmallcaps.js @@ -18,6 +18,8 @@ import { nameForPassableSymbol, passableSymbolForName, } from '@endo/pass-style'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; +import { encodeHex, decodeHex } from '@endo/hex'; /** @import {Passable, RemotableObject} from '@endo/pass-style' */ // FIXME define actual types @@ -50,6 +52,7 @@ const DASH = '-'.charCodeAt(0); * Of these, smallcaps currently uses the following: * * * `!` - escaped string + * * `*` - byteArray, hex-encoded * * `+` - non-negative bigint * * `-` - negative bigint * * `#` - manifest constant @@ -57,7 +60,7 @@ const DASH = '-'.charCodeAt(0); * * `$` - remotable * * `&` - promise * - * All other special characters (`"'()*,`) are reserved for future use. + * All other special characters (`"'(),`) are reserved for future use. * * The manifest constants that smallcaps currently uses for values: * * `#undefined` @@ -230,8 +233,7 @@ export const makeEncodeToSmallcaps = (encodeOptions = {}) => { return passable.map(encodeToSmallcapsRecur); } case 'byteArray': { - // TODO implement - throw Fail`marsal of byteArray not yet implemented: ${passable}`; + return `*${encodeHex(thawedBytes(passable))}`; } case 'tagged': { return { @@ -407,6 +409,11 @@ export const makeDecodeFromSmallcaps = (decodeOptions = {}) => { } return result; } + case '*': { + return frozenBytes( + decodeHex(encoding.slice(1), 'smallcaps byteArray'), + ); + } default: { throw Fail`Special char ${q( c, @@ -472,3 +479,4 @@ export const makeDecodeFromSmallcaps = (decodeOptions = {}) => { }; return harden(decodeFromSmallcaps); }; +harden(makeDecodeFromSmallcaps); diff --git a/packages/marshal/src/marshal-justin.js b/packages/marshal/src/marshal-justin.js index a50417cd96..323c505d44 100644 --- a/packages/marshal/src/marshal-justin.js +++ b/packages/marshal/src/marshal-justin.js @@ -177,6 +177,11 @@ const decodeToJustin = (encoding, shouldIndent = false, slots = []) => { assert.typeof(sym, 'symbol'); return; } + case 'byteArray': { + const { data } = rawTree; + assert.typeof(data, 'string'); + return; + } case 'tagged': { const { tag, payload } = rawTree; assert.typeof(tag, 'string'); @@ -339,6 +344,11 @@ const decodeToJustin = (encoding, shouldIndent = false, slots = []) => { } return out.next(`passableSymbolForName(${quote(registeredName)})`); } + case 'byteArray': { + const { data } = rawTree; + assert.typeof(data, 'string'); + return out.next(`frozenBytes(decodeHex(${quote(data)}))`); + } case 'tagged': { const { tag, payload } = rawTree; out.next(`makeTagged(${quote(tag)}`); diff --git a/packages/marshal/src/rankOrder.js b/packages/marshal/src/rankOrder.js index 420398882b..2634227cb6 100644 --- a/packages/marshal/src/rankOrder.js +++ b/packages/marshal/src/rankOrder.js @@ -2,6 +2,7 @@ import harden from '@endo/harden'; import { getEnvironmentOption as getenv } from '@endo/env-options'; import { Fail, q } from '@endo/errors'; import { getTag, passStyleOf, nameForPassableSymbol } from '@endo/pass-style'; +import { compareBytes } from '@endo/bytes/compare.js'; import { passStylePrefixes, recordNames, @@ -295,26 +296,23 @@ export const makeComparatorKit = (compareRemotables = (_x, _y) => NaN) => { return 1; } - // Account for gaps in the @endo/immutable-arraybuffer shim. - const leftArray = - Object.getPrototypeOf(left) === ArrayBuffer.prototype - ? new Uint8Array(left) - : new Uint8Array(left.slice(0)); - const rightArray = - Object.getPrototypeOf(right) === ArrayBuffer.prototype - ? new Uint8Array(right) - : new Uint8Array(right.slice(0)); - for (let i = 0; i < leftLen; i += 1) { - const leftByte = leftArray[i]; - const rightByte = rightArray[i]; - if (leftByte < rightByte) { - return -1; - } - if (leftByte > rightByte) { - return 1; - } - } - return 0; + // The byteArray pass style is a frozen Uint8Array backed by an + // immutable ArrayBuffer. On the emulated + // `@endo/immutable-arraybuffer` path the wrapper is a plain object + // with no integer-indexed own properties, so a direct `array[i]` + // read returns `undefined` rather than the byte: emulated frozen + // Uint8Arrays do not have integer-index behavior. `compareBytes` + // (and any other byte reader) needs genuine integer-indexable + // Uint8Arrays, so first copy each wrapper into a fresh mutable + // Uint8Array via `slice`, which the shim amplifies (and which is a + // genuine no-amplification copy on the native path). `slice(0)` + // honors the wrapper's own [0, length) window regardless of any + // `byteOffset`. The lengths are already known equal here, so the + // lexicographic `compareBytes` agrees with the shortlex order the + // length pre-check established. See `@endo/bytes/compare.js`. + const leftArray = /** @type {Uint8Array} */ (left).slice(0); + const rightArray = /** @type {Uint8Array} */ (right).slice(0); + return compareBytes(leftArray, rightArray); } case 'tagged': { // Lexicographic by `[Symbol.toStringTag]` then `.payload`. diff --git a/packages/marshal/src/types.js b/packages/marshal/src/types.js index 8ae36c7e21..1718b7d996 100644 --- a/packages/marshal/src/types.js +++ b/packages/marshal/src/types.js @@ -47,7 +47,8 @@ export {}; * } | * EncodingClass<'tagged'> & { tag: string, * payload: Encoding - * } + * } | + * EncodingClass<'byteArray'> & { data: string } * } EncodingUnion * * Note that the '@@asyncIterator' encoding is deprecated. Use 'symbol' instead. diff --git a/packages/marshal/test/byteArray.test.js b/packages/marshal/test/byteArray.test.js new file mode 100644 index 0000000000..745fcc1760 --- /dev/null +++ b/packages/marshal/test/byteArray.test.js @@ -0,0 +1,212 @@ +// @ts-nocheck +import test from '@endo/ses-ava/test.js'; + +import harden from '@endo/harden'; +import { passStyleOf } from '@endo/pass-style'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; +import { makeMarshal } from '../src/marshal.js'; +import { + makeEncodePassable, + makeDecodePassable, +} from '../src/encodePassable.js'; +import { compareRank } from '../src/rankOrder.js'; + +// A `byteArray` is a plain frozen `Uint8Array` backed by an immutable +// `ArrayBuffer` (per the narrowing in the byteArray pass style). Build one +// from raw bytes with `frozenBytes`, and read its contents back out as a +// fresh mutable copy with `thawedBytes`. +const mkByteArray = bytes => frozenBytes(new Uint8Array(bytes)); +const readBytes = byteArray => [...thawedBytes(byteArray)]; + +const fixtures = harden([ + { name: 'empty', bytes: [] }, + { name: 'single-zero', bytes: [0x00] }, + { name: 'single-ff', bytes: [0xff] }, + { name: 'two-zeroes', bytes: [0x00, 0x00] }, + { name: 'deadbeef', bytes: [0xde, 0xad, 0xbe, 0xef] }, + { name: 'long', bytes: Array.from({ length: 256 }, (_, i) => i) }, +]); + +test('smallcaps round-trips byteArray', t => { + const { serialize, unserialize } = makeMarshal(undefined, undefined, { + serializeBodyFormat: 'smallcaps', + errorTagging: 'off', + }); + for (const { name, bytes } of fixtures) { + const ba = mkByteArray(bytes); + const { body } = serialize(ba); + const decoded = unserialize({ body, slots: [] }); + t.is(passStyleOf(decoded), 'byteArray', name); + t.deepEqual(readBytes(decoded), bytes, `smallcaps ${name}`); + } +}); + +test('smallcaps byteArray uses "*" prefix with hex body', t => { + const { serialize } = makeMarshal(undefined, undefined, { + serializeBodyFormat: 'smallcaps', + errorTagging: 'off', + }); + const { body } = serialize(mkByteArray([0xde, 0xad, 0xbe, 0xef])); + // smallcaps body has a leading `#` sentinel before the JSON text. + t.true(body.includes('"*deadbeef"'), `got ${body}`); +}); + +test('capdata round-trips byteArray', t => { + const { serialize, unserialize } = makeMarshal(undefined, undefined, { + serializeBodyFormat: 'capdata', + errorTagging: 'off', + }); + for (const { name, bytes } of fixtures) { + const ba = mkByteArray(bytes); + const { body } = serialize(ba); + const decoded = unserialize({ body, slots: [] }); + t.is(passStyleOf(decoded), 'byteArray', name); + t.deepEqual(readBytes(decoded), bytes, `capdata ${name}`); + } +}); + +test('capdata byteArray uses @qclass "byteArray" with hex data', t => { + const { serialize } = makeMarshal(undefined, undefined, { + serializeBodyFormat: 'capdata', + errorTagging: 'off', + }); + const { body } = serialize(mkByteArray([0xde, 0xad, 0xbe, 0xef])); + t.true( + body.includes('"@qclass":"byteArray"') && body.includes('"deadbeef"'), + `got ${body}`, + ); +}); + +test('byteArray nested in copyArray, copyRecord, tagged', t => { + const { serialize, unserialize } = makeMarshal(undefined, undefined, { + serializeBodyFormat: 'smallcaps', + errorTagging: 'off', + }); + const ba = mkByteArray([1, 2, 3]); + const structure = harden({ + arr: [ba, ba], + rec: { k: ba }, + }); + const { body } = serialize(structure); + const decoded = unserialize({ body, slots: [] }); + t.is(passStyleOf(decoded.arr[0]), 'byteArray'); + t.is(passStyleOf(decoded.rec.k), 'byteArray'); + t.deepEqual(readBytes(decoded.arr[1]), [1, 2, 3]); +}); + +test('encodePassable round-trips byteArray (legacyOrdered)', t => { + const encode = makeEncodePassable({ format: 'legacyOrdered' }); + const decode = makeDecodePassable({ format: 'legacyOrdered' }); + for (const { name, bytes } of fixtures) { + const ba = mkByteArray(bytes); + const enc = encode(ba); + t.is(enc.charAt(0), 'a', `legacy ${name} starts with 'a'`); + const back = decode(enc); + t.deepEqual(readBytes(back), bytes, `legacy ${name}`); + } +}); + +test('encodePassable round-trips byteArray (compactOrdered)', t => { + const encode = makeEncodePassable({ format: 'compactOrdered' }); + const decode = makeDecodePassable({ format: 'compactOrdered' }); + for (const { name, bytes } of fixtures) { + const ba = mkByteArray(bytes); + const enc = encode(ba); + const back = decode(enc); + t.deepEqual(readBytes(back), bytes, `compact ${name}`); + } +}); + +test('encodePassable byteArray preserves shortlex order', t => { + const encode = makeEncodePassable({ format: 'legacyOrdered' }); + // Listed in the expected shortlex order. + const orderedBytes = [ + [], + [0x00], + [0x01], + [0xff], + [0x00, 0x00], + [0x00, 0x01], + [0x01, 0x00], + [0xff, 0xfe], + [0xff, 0xff], + [0x00, 0x00, 0x00], + ]; + const encodings = orderedBytes.map(bs => encode(mkByteArray(bs))); + const sorted = [...encodings].sort(); + t.deepEqual(sorted, encodings, `sorted=${sorted.join(',')}`); +}); + +test('encodePassable byteArray agrees with compareRank', t => { + const encode = makeEncodePassable({ format: 'legacyOrdered' }); + const values = harden([ + mkByteArray([]), + mkByteArray([0x00]), + mkByteArray([0xff]), + mkByteArray([0x00, 0x00]), + mkByteArray([0x00, 0x01]), + mkByteArray([0xff, 0xff]), + mkByteArray([0x00, 0x00, 0x00]), + ]); + for (let i = 0; i < values.length; i += 1) { + for (let j = 0; j < values.length; j += 1) { + const rank = compareRank(values[i], values[j]); + const encA = encode(values[i]); + const encB = encode(values[j]); + // eslint-disable-next-line no-nested-ternary + const lex = encA < encB ? -1 : encA > encB ? 1 : 0; + t.is( + Math.sign(rank), + lex, + `pair i=${i} j=${j}: rank ${rank} vs lex ${lex}`, + ); + } + } +}); + +test('encodePassable byteArray cover sits between promise and boolean', t => { + const encode = makeEncodePassable({ + format: 'legacyOrdered', + encodePromise: (_p, _r) => '?0', + }); + const promiseEnc = '?0'; + const boolTrue = encode(true); + const byteEnc = encode(mkByteArray([0xff])); + t.true(promiseEnc < byteEnc, `${promiseEnc} < ${byteEnc}`); + t.true(byteEnc < boolTrue, `${byteEnc} < ${boolTrue}`); +}); + +test('decodePassable rejects malformed byteArray body', t => { + const decode = makeDecodePassable({ format: 'legacyOrdered' }); + // The body after the leading 'a' must match /^(p[~]*[0-9]+:[0-9]+):([0-9a-f]*)$/. + // A body with no length-prefix-then-colon-then-hex shape must fail closed. + t.throws(() => decode('agarbage'), { message: /byteArray/ }); +}); + +test('decodePassable rejects byteArray length-vs-body mismatch', t => { + const encode = makeEncodePassable({ format: 'legacyOrdered' }); + const decode = makeDecodePassable({ format: 'legacyOrdered' }); + // Header claims byteLength=3 but the hex body has 4 bytes (8 hex chars). + // The mismatch path is the explicit length check between the header and the + // hex body, distinct from the regex shape check above. + const lengthThree = encode(mkByteArray([0xaa, 0xbb, 0xcc])); + // lengthThree is `a:aabbcc`; replace the body with 4 bytes. + const headerLen = lengthThree.lastIndexOf(':'); + const mismatched = `${lengthThree.slice(0, headerLen + 1)}aabbccdd`; + t.throws(() => decode(mismatched), { + message: /byteArray length mismatch/, + }); +}); + +test('capdata unserialize rejects byteArray with non-string data', t => { + const { unserialize } = makeMarshal(undefined, undefined, { + serializeBodyFormat: 'capdata', + errorTagging: 'off', + }); + // The decoder asserts typeof data === 'string'; a number must fail closed + // rather than silently passing through to the hex decoder. + const body = JSON.stringify({ '@qclass': 'byteArray', data: 42 }); + t.throws(() => unserialize({ body, slots: [] }), { + message: /invalid byteArray data typeof/, + }); +}); diff --git a/packages/marshal/test/encodePassable.test.js b/packages/marshal/test/encodePassable.test.js index e49da473b3..5c14eb0d16 100644 --- a/packages/marshal/test/encodePassable.test.js +++ b/packages/marshal/test/encodePassable.test.js @@ -16,7 +16,7 @@ import { import { compareRank, makeFullOrderComparatorKit } from '../src/rankOrder.js'; import { unsortedSample } from '../tools/marshal-test-data.js'; -const { arbPassable } = makeArbitraries(fc, ['byteArray']); +const { arbPassable } = makeArbitraries(fc); const statelessEncodePassableLegacy = makeEncodePassable(); diff --git a/packages/marshal/test/marshal-justin.test.js b/packages/marshal/test/marshal-justin.test.js index bdedb34f9d..8bf7966c40 100644 --- a/packages/marshal/test/marshal-justin.test.js +++ b/packages/marshal/test/marshal-justin.test.js @@ -8,6 +8,8 @@ import { makeTagged, passableSymbolForName, } from '@endo/pass-style'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; +import { decodeHex } from '@endo/hex'; import { makeMarshal } from '../src/marshal.js'; import { decodeToJustin, qp } from '../src/marshal-justin.js'; import { jsonJustinPairs } from '../tools/marshal-test-data.js'; @@ -42,6 +44,8 @@ const fakeJustinCompartment = () => { return new Compartment({ slot, slotToVal, + frozenBytes, + decodeHex, makeTagged, passableSymbolForName, }); diff --git a/packages/marshal/test/rankOrder.test.js b/packages/marshal/test/rankOrder.test.js index 2e673e80fc..c046ba9aae 100644 --- a/packages/marshal/test/rankOrder.test.js +++ b/packages/marshal/test/rankOrder.test.js @@ -309,3 +309,27 @@ test('coveredEntries empty range', t => { } t.is(result.length, 0); }); + +// Constructs a byteArray pass-style value (a hardened frozen `Uint8Array` +// backed by an immutable `ArrayBuffer`) from a list of byte values. On the +// emulated `@endo/immutable-arraybuffer` path this wrapper has no +// integer-indexed own properties, so a direct `wrapper[i]` read returns +// `undefined`; `compareRank` must read its bytes through an amplified path. +const byteArrayOf = bytes => { + const buffer = new ArrayBuffer(bytes.length); + new Uint8Array(buffer).set(bytes); + return harden(new Uint8Array(buffer.sliceToImmutable())); +}; + +test('compareRank orders byteArrays by shortlex, reading bytes correctly', t => { + const a = byteArrayOf([0x10, 0x20, 0x30]); + const b = byteArrayOf([0x10, 0x20, 0x31]); + t.is(compareRank(a, b), -1); + t.is(compareRank(b, a), 1); + t.is(compareRank(a, byteArrayOf([0x10, 0x20, 0x30])), 0); + + const short = byteArrayOf([0xff]); + const long = byteArrayOf([0x00, 0x00]); + t.is(compareRank(short, long), -1); + t.is(compareRank(long, short), 1); +}); diff --git a/packages/marshal/tools/marshal-test-data.js b/packages/marshal/tools/marshal-test-data.js index f3b412baa4..2ac7e9f5b8 100644 --- a/packages/marshal/tools/marshal-test-data.js +++ b/packages/marshal/tools/marshal-test-data.js @@ -1,11 +1,17 @@ import harden from '@endo/harden'; import { makeTagged, passableSymbolForName } from '@endo/pass-style'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; +import { decodeHex } from '@endo/hex'; import { exampleAlice, exampleBob, exampleCarol, } from '@endo/pass-style/tools.js'; +// Build a `byteArray` (a plain frozen `Uint8Array` backed by an immutable +// `ArrayBuffer`) from a hex string, for use as test data. +const byteArrayFromHex = hex => frozenBytes(decodeHex(hex)); + /** @import { Passable } from '@endo/pass-style' */ /** @@ -130,6 +136,15 @@ export const roundTripPairs = harden([ [[undefined], [{ '@qclass': 'undefined' }]], [{ foo: undefined }, { foo: { '@qclass': 'undefined' } }], + // byteArray + [ + byteArrayFromHex('0f'), + { + '@qclass': 'byteArray', + data: '0f', + }, + ], + // tagged [ makeTagged('x', 8), @@ -259,6 +274,9 @@ export const jsonJustinPairs = harden([ ['{"@qclass":"symbol","name":"foo"}', 'passableSymbolForName("foo")'], ['{"@qclass":"symbol","name":"@@@@foo"}', 'passableSymbolForName("@@@@foo")'], + // byteArray + ['{"@qclass":"byteArray","data":"0aff"}', 'frozenBytes(decodeHex("0aff"))'], + // Arrays and objects ['[{"@qclass":"undefined"}]', '[undefined]'], ['{"foo":{"@qclass":"undefined"}}', '{foo:undefined}'], @@ -335,6 +353,7 @@ export const unsortedSample = harden([ undefined, -Infinity, [5], + byteArrayFromHex('0f'), exampleAlice, [], passableSymbolForName('foo'), @@ -348,6 +367,7 @@ export const unsortedSample = harden([ [exampleAlice, 'a'], [exampleBob, 'z'], -0, + byteArrayFromHex('aa'), {}, [5, undefined], -3, @@ -357,6 +377,7 @@ export const unsortedSample = harden([ ]), true, 'bar', + byteArrayFromHex('0a'), [5, null], new Promise(() => {}), // forever unresolved makeTagged('nonsense', [ @@ -430,6 +451,10 @@ export const sortedSample = harden([ [exampleCarol, 'm'], [exampleBob, 'z'], + byteArrayFromHex('0a'), + byteArrayFromHex('0f'), + byteArrayFromHex('aa'), + false, true, true, diff --git a/packages/marshal/tsconfig.composite.json b/packages/marshal/tsconfig.composite.json index f7cd178710..b94aa791fb 100644 --- a/packages/marshal/tsconfig.composite.json +++ b/packages/marshal/tsconfig.composite.json @@ -5,6 +5,9 @@ "composite": true }, "references": [ + { + "path": "../bytes/tsconfig.composite.json" + }, { "path": "../common/tsconfig.composite.json" }, @@ -20,6 +23,12 @@ { "path": "../harden/tsconfig.composite.json" }, + { + "path": "../hex/tsconfig.composite.json" + }, + { + "path": "../immutable-arraybuffer/tsconfig.composite.json" + }, { "path": "../nat/tsconfig.composite.json" }, diff --git a/packages/ocapn-noise/src/network.js b/packages/ocapn-noise/src/network.js index 2859418315..9bae9a093b 100644 --- a/packages/ocapn-noise/src/network.js +++ b/packages/ocapn-noise/src/network.js @@ -77,16 +77,29 @@ const hexToBytes = hex => { }; /** - * Return a Uint8Array covering the contents of `buf`. Immutable - * ArrayBuffers must be sliced before a typed-array view works. + * Return a genuine, integer-indexable `Uint8Array` covering the contents + * of `buf`. * - * @param {ArrayBufferLike} buf + * The `byteArray` pass style is a frozen `Uint8Array` backed by an + * immutable `ArrayBuffer`. In an engine without native immutable + * ArrayBuffers the shim emulates it with a plain object that answers + * `instanceof Uint8Array` and iterates correctly but has no + * integer-indexed exotic slots, so `emulated[i]` reads `undefined`. + * `ArrayBuffer.isView` — not `instanceof` — is the reliable test for a + * genuine view (matching `@endo/immutable-arraybuffer`' `thawedBytes`): genuine + * views (including those over immutable buffers, which permit indexed + * reads) are returned as-is, while an emulated + * `@endo/immutable-arraybuffer` wrapper (which reports + * `ArrayBuffer.isView === false`) is copied into a fresh, genuine + * `Uint8Array`. + * + * @param {Uint8Array} buf * @returns {Uint8Array} */ const asUint8 = buf => - buf instanceof Uint8Array - ? buf - : new Uint8Array(/** @type {ArrayBuffer} */ (buf.slice())); + ArrayBuffer.isView(buf) + ? /** @type {Uint8Array} */ (buf) + : new Uint8Array(/** @type {Uint8Array} */ (buf).slice()); /** * Pull one whole message from a message-framed `Reader` @@ -722,7 +735,7 @@ export const makeOcapnNoiseNetwork = ({ }, }); - const peerEd25519Buffer = peerEd25519.slice().buffer; + const peerEd25519Buffer = peerEd25519.slice(); const sessionId = makeSessionId( localKey.keyPair.publicKey.id, cryptography.makeOcapnPublicKey(peerEd25519Buffer).id, @@ -1036,9 +1049,7 @@ export const makeOcapnNoiseNetwork = ({ const privateKey = new Uint8Array(32); getRandomValues(privateKey); const keyPair = cryptography.makeOcapnKeyPairFromPrivateKey(privateKey); - const publicKey = new Uint8Array( - /** @type {ArrayBuffer} */ (keyPair.publicKey.bytes.slice()), - ); + const publicKey = new Uint8Array(asUint8(keyPair.publicKey.bytes)); return { privateKey, publicKey }; }; @@ -1057,9 +1068,7 @@ export const makeOcapnNoiseNetwork = ({ // under one keyId but unable to complete a handshake as that // identity, which is a debugging cliff to fall off). const keyPair = cryptography.makeOcapnKeyPairFromPrivateKey(privateKey); - const derivedPublicKey = new Uint8Array( - /** @type {ArrayBuffer} */ (keyPair.publicKey.bytes.slice()), - ); + const derivedPublicKey = new Uint8Array(asUint8(keyPair.publicKey.bytes)); if (publicKey) { if (compareUint8Arrays(publicKey, derivedPublicKey) !== 0) { throw makeError( diff --git a/packages/ocapn-noise/src/types.d.ts b/packages/ocapn-noise/src/types.d.ts index e1fe0d355d..c0af97e9d8 100644 --- a/packages/ocapn-noise/src/types.d.ts +++ b/packages/ocapn-noise/src/types.d.ts @@ -1,6 +1,7 @@ import type { Reader, Writer } from '@endo/stream'; import type { OcapnCodec } from '@endo/ocapn/codec-interface'; import type { OcapnLocation, OcapnSignature } from '@endo/ocapn/components'; +import type { SessionId } from '@endo/ocapn/client/types'; /** * 32-byte Ed25519 signing keys. The same keypair backs both the Noise @@ -78,7 +79,7 @@ export interface OcapnNoiseTransport { * `@endo/ocapn`'s `NetworkSession` is itself stream-based. */ export interface OcapnNoiseSession { - sessionId: ArrayBufferLike; + sessionId: SessionId; selfIdentity: { location: OcapnLocation; locationSignature: OcapnSignature; @@ -91,7 +92,7 @@ export interface OcapnNoiseSession { }; remoteLocation: OcapnLocation; remoteLocationSignature: OcapnSignature; - remotePublicKeyBytes: ArrayBufferLike; + remotePublicKeyBytes: Uint8Array; isInitiator: boolean; reader: Reader; writer: Writer; diff --git a/packages/ocapn/package.json b/packages/ocapn/package.json index 79c58dda0f..fea182cfef 100644 --- a/packages/ocapn/package.json +++ b/packages/ocapn/package.json @@ -49,11 +49,13 @@ "test:update-snapshots": "ses-ava --update-snapshots" }, "dependencies": { + "@endo/ascii": "workspace:^", "@endo/bytes": "workspace:^", "@endo/cbor": "workspace:^", "@endo/eventual-send": "workspace:^", "@endo/harden": "workspace:^", "@endo/hex": "workspace:^", + "@endo/immutable-arraybuffer": "workspace:^", "@endo/init": "workspace:^", "@endo/marshal": "workspace:^", "@endo/nat": "workspace:^", diff --git a/packages/ocapn/src/bytewise-compare.js b/packages/ocapn/src/bytewise-compare.js index 80d993cf10..eb364b4746 100644 --- a/packages/ocapn/src/bytewise-compare.js +++ b/packages/ocapn/src/bytewise-compare.js @@ -1,6 +1,6 @@ // @ts-check -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; +import { thawedBytes } from '@endo/immutable-arraybuffer'; /** * @param {Uint8Array} left @@ -87,13 +87,10 @@ export function compareUint8Arrays( /** * Compare two immutable ArrayBuffers - * @param {ArrayBufferLike} left - * @param {ArrayBufferLike} right + * @param {Uint8Array} left + * @param {Uint8Array} right * @returns {number} */ export const compareImmutableArrayBuffers = (left, right) => { - return compareUint8Arrays( - bytesFromImmutable(left), - bytesFromImmutable(right), - ); + return compareUint8Arrays(thawedBytes(left), thawedBytes(right)); }; diff --git a/packages/ocapn/src/cbor/decode.js b/packages/ocapn/src/cbor/decode.js index 49903165b8..e017175d86 100644 --- a/packages/ocapn/src/cbor/decode.js +++ b/packages/ocapn/src/cbor/decode.js @@ -19,7 +19,7 @@ * See docs/cbor-encoding.md for the specification. */ -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { makeCborReader as makeCborReaderState, @@ -203,12 +203,12 @@ export class CborReader { } /** - * @returns {ArrayBufferLike} + * @returns {Uint8Array} */ readBytestring() { this.#decrementRemaining(); // Immutability conversion stays OCapN policy at the class layer. - return bytesToImmutable(readByteString(this.#reader)); + return frozenBytes(readByteString(this.#reader)); } /** @@ -430,7 +430,7 @@ export class CborReader { if (major === MAJOR_BYTESTRING) { // Use raw function to avoid double-decrement - const value = bytesToImmutable(readByteString(this.#reader)); + const value = frozenBytes(readByteString(this.#reader)); return { type: 'bytestring', value }; } diff --git a/packages/ocapn/src/cbor/diagnostic/util.js b/packages/ocapn/src/cbor/diagnostic/util.js index 6fab4599ee..6b455f1592 100644 --- a/packages/ocapn/src/cbor/diagnostic/util.js +++ b/packages/ocapn/src/cbor/diagnostic/util.js @@ -6,6 +6,39 @@ * Hex conversion and value comparison helpers. */ +const { isView } = ArrayBuffer; + +/** + * Normalize a `Uint8Array` to one that supports integer-indexed access + * (`bytes[i]`). The input is a plain mutable `Uint8Array`, a genuine frozen + * view over an immutable `ArrayBuffer`, or an emulated + * `@endo/immutable-arraybuffer` wrapper (which reports + * `ArrayBuffer.isView === false`). + * + * `ArrayBuffer.isView` is the single committed genuine-vs-emulated + * distinguisher (the `byteArray` narrowing, issue #573): a genuine view is + * integer-indexable in place, while an emulated frozen byteArray wrapper + * (produced by the `@endo/immutable-arraybuffer` shim) is a plain object that + * reports `isView === false` and reads `undefined` from `wrapper[i]`, so it + * must first be thawed into a fresh mutable `Uint8Array`. Without this, two + * distinct equal-length emulated byteArrays would compare `equal` + * (`undefined === undefined`). This mirrors the identically named helper in + * `@endo/bytes/src/compare.js`. + * + * @param {Uint8Array} input + * @returns {Uint8Array} + */ +const toIndexableUint8 = input => { + if (isView(input)) { + // A genuine `Uint8Array` is indexed in place (zero allocation). + return input; + } + // Not a genuine view: the emulated `@endo/immutable-arraybuffer` wrapper. + // `.slice(0)` yields a fresh mutable array, over which the `Uint8Array` is + // integer-indexable. + return new Uint8Array(/** @type {Uint8Array} */ (input).slice(0)); +}; + /** * Convert hex string to Uint8Array * @param {string} hex - Hex string (with or without spaces) @@ -69,10 +102,20 @@ export function equals(actual, expected) { // Handle ArrayBuffer/Uint8Array if (actual instanceof ArrayBuffer || actual instanceof Uint8Array) { - const actualBytes = - actual instanceof Uint8Array ? actual : new Uint8Array(actual); - const expectedBytes = - expected instanceof Uint8Array ? expected : new Uint8Array(expected); + if (!(expected instanceof ArrayBuffer || expected instanceof Uint8Array)) { + return false; + } + // A bare `ArrayBuffer` may reach here (the `equals` input is `any`); + // normalize it to a `Uint8Array` at this boundary so `toIndexableUint8` + // resolves no buffer-vs-view disjunction. It then thaws an emulated wrapper + // into a mutable copy so integer-indexed reads see the real bytes rather + // than `undefined`. + const actualBytes = toIndexableUint8( + actual instanceof Uint8Array ? actual : new Uint8Array(actual), + ); + const expectedBytes = toIndexableUint8( + expected instanceof Uint8Array ? expected : new Uint8Array(expected), + ); if (actualBytes.length !== expectedBytes.length) return false; for (let i = 0; i < actualBytes.length; i += 1) { diff --git a/packages/ocapn/src/cbor/encode.js b/packages/ocapn/src/cbor/encode.js index f3a8ce230e..4fdce741c1 100644 --- a/packages/ocapn/src/cbor/encode.js +++ b/packages/ocapn/src/cbor/encode.js @@ -31,6 +31,8 @@ import { writeUndefined, } from '@endo/cbor'; +import { thawedBytes } from '@endo/immutable-arraybuffer'; + /** * @import { OcapnWriter } from '../codec-interface.js' * @import { CborWriter as CborWriterState } from '@endo/cbor' @@ -55,16 +57,18 @@ const TAG_SYMBOL = 280n; // OCapN symbol (selector) const TAG_TAGGED_VALUE = 55_799n; // Self-described CBOR / OCapN tagged /** - * Write a byte string, accepting either a Uint8Array or an (immutable) - * ArrayBuffer. `@endo/cbor`'s `writeByteString` requires a Uint8Array, so the - * ArrayBuffer coercion stays here at the OCapN boundary. + * Write a byte string. The value is a `Uint8Array`: a plain mutable one, a + * genuine frozen view over an immutable `ArrayBuffer`, or an emulated + * `@endo/immutable-arraybuffer` wrapper (which reports + * `ArrayBuffer.isView === false`). `@endo/cbor`'s `writeByteString` requires a + * plain mutable `Uint8Array`, so the `thawedBytes` normalization stays here at + * the OCapN boundary. * * @param {CborWriterState} writer - * @param {Uint8Array | ArrayBufferLike} value + * @param {Uint8Array} value */ function writeBytestring(writer, value) { - const bytes = - value instanceof Uint8Array ? value : new Uint8Array(value.slice()); + const bytes = thawedBytes(value); writeByteString(writer, bytes); } @@ -146,7 +150,7 @@ export class CborWriter { } /** - * @param {ArrayBufferLike} value + * @param {Uint8Array} value */ writeBytestring(value) { writeBytestring(this.#writer, value); diff --git a/packages/ocapn/src/client/ocapn.js b/packages/ocapn/src/client/ocapn.js index 5a626e5aac..6b3f863950 100644 --- a/packages/ocapn/src/client/ocapn.js +++ b/packages/ocapn/src/client/ocapn.js @@ -526,7 +526,7 @@ const makeBootstrapObject = ( return object; }, /** - * @param {ArrayBufferLike} giftId + * @param {Uint8Array} giftId * @param {any} gift */ 'deposit-gift': (giftId, gift) => { diff --git a/packages/ocapn/src/client/ref-kit.js b/packages/ocapn/src/client/ref-kit.js index 1055e2dd40..7d37c31f29 100644 --- a/packages/ocapn/src/client/ref-kit.js +++ b/packages/ocapn/src/client/ref-kit.js @@ -71,7 +71,7 @@ import { makeSlot, parseSlot } from '../captp/pairwise.js'; * @property {(remotePromise: Promise) => object} makeLocalResolverForRemotePromise * @property {(answerPosition: bigint, promise: Promise) => Promise} makeLocalAnswerPromiseAndFulfill * @property {(position: bigint) => Promise} getLocalAnswerValue - * @property {(location: OcapnLocation, secret: string) => SturdyRef} makeSturdyRef + * @property {(location: OcapnLocation, secret: string | Uint8Array) => SturdyRef} makeSturdyRef * @property {(signedGive: HandoffGiveSigEnvelope) => Promise} provideHandoff * @property {(signedGive: HandoffGiveDetails) => HandoffGiveSigEnvelope} sendHandoff * @property {(value: object) => ValInfo} getInfoForVal diff --git a/packages/ocapn/src/client/sturdyrefs.js b/packages/ocapn/src/client/sturdyrefs.js index 568dff4069..e3d7481dff 100644 --- a/packages/ocapn/src/client/sturdyrefs.js +++ b/packages/ocapn/src/client/sturdyrefs.js @@ -6,9 +6,15 @@ */ import harden from '@endo/harden'; +import { thawedBytes } from '@endo/immutable-arraybuffer'; import { E } from '@endo/eventual-send'; import { makeTagged } from '@endo/pass-style'; -import { encodeSwissnum, swissnumFromBytes } from './util.js'; +import { + decodeSwissnum, + encodeSwissnum, + swissnumFromBytes, + swissnumToBytes, +} from './util.js'; /** * @import { CopyTagged } from '@endo/pass-style' @@ -101,7 +107,7 @@ export const enlivenSturdyRef = async ( /** * @typedef {object} SturdyRefTracker * @property {(location: OcapnLocation, secret: string | Uint8Array) => SturdyRef} makeSturdyRef - * @property {(secretBytes: ArrayBufferLike) => Promise} lookup + * @property {(secretBytes: Uint8Array) => Promise} lookup * Async look up a locally-held capability by the on-wire secret * bytes. Calls through to the injected locator with either the * ASCII-decoded string (for printable secrets) or the raw bytes (for @@ -113,25 +119,22 @@ export const enlivenSturdyRef = async ( * @returns {SturdyRefTracker} */ export const makeSturdyRefTracker = locator => { - const textDecoder = new TextDecoder('ascii', { fatal: true }); return harden({ makeSturdyRef: (location, secret) => makeSturdyRef(location, secret), lookup: async secretBytes => { - const view = - secretBytes instanceof Uint8Array - ? secretBytes - : new Uint8Array(/** @type {ArrayBuffer} */ (secretBytes.slice())); + const swissNum = swissnumFromBytes(thawedBytes(secretBytes)); // Try ASCII decoding first so locators keyed by friendly string // names continue to match. If the bytes aren't valid ASCII (e.g. // a Spritely-style random 24-byte secret), fall back to passing // the raw bytes through; locators that index by bytes can match // those, locators that don't will simply return undefined. + let secret; try { - const secret = textDecoder.decode(view); - return locator.get(secret); + secret = decodeSwissnum(swissNum); } catch { - return locator.get(view); + return locator.get(swissnumToBytes(swissNum)); } + return locator.get(secret); }, }); }; diff --git a/packages/ocapn/src/client/types.js b/packages/ocapn/src/client/types.js index c76e5acb55..3dfaa69df4 100644 --- a/packages/ocapn/src/client/types.js +++ b/packages/ocapn/src/client/types.js @@ -9,13 +9,17 @@ */ /** + * The byteArray pass style is a frozen `Uint8Array` backed by an + * immutable `ArrayBuffer`. The branded byteArray-shaped types below + * are `Uint8Array` at runtime (the current byteArray shape). + * * @typedef {string & { _brand: 'LocationId' }} LocationId * A string used for referencing, such as keys in Maps. Not part of OCapN spec. - * @typedef {ArrayBufferLike & { _brand: 'SessionId' }} SessionId + * @typedef {Uint8Array & { _brand: 'SessionId' }} SessionId * From OCapN spec. Id for a session between two peers. - * @typedef {ArrayBufferLike & { _brand: 'SwissNum' }} SwissNum + * @typedef {Uint8Array & { _brand: 'SwissNum' }} SwissNum * From OCapN spec. Used for resolving SturdyRefs. - * @typedef {ArrayBufferLike & { _brand: 'PublicKeyId' }} PublicKeyId + * @typedef {Uint8Array & { _brand: 'PublicKeyId' }} PublicKeyId * From OCapN spec. Identifier for a public key (double SHA-256 hash of key descriptor). */ @@ -96,7 +100,7 @@ * @property {SelfIdentity} selfIdentity - Our identity for this session, * supplied by the network (which authenticated to the peer using this * keypair during handshake). - * @property {ArrayBufferLike} remotePublicKeyBytes - Peer's raw public + * @property {Uint8Array} remotePublicKeyBytes - Peer's raw public * key bytes (needed to construct OcapnPublicKey for session). * @property {OcapnLocation} remoteLocation - Peer's location. * @property {import('../codecs/components.js').OcapnSignature} remoteLocationSignature - @@ -252,10 +256,10 @@ * `NetlayerHandlers.resumeSession`. * * @typedef {object} SessionResumption - * @property {ArrayBufferLike} sessionId + * @property {SessionId} sessionId * @property {OcapnLocation} peerLocation * @property {OcapnSignature} peerLocationSignature - * @property {ArrayBufferLike} peerPublicKeyBytes + * @property {Uint8Array} peerPublicKeyBytes * @property {Uint8Array} [selfPrivateKeyBytes] resume with the same * session keys the previous process used, so cross-restart handoff * signatures keep verifying; omitted, fresh keys are minted diff --git a/packages/ocapn/src/client/util.js b/packages/ocapn/src/client/util.js index 1166cb1f98..c42539a282 100644 --- a/packages/ocapn/src/client/util.js +++ b/packages/ocapn/src/client/util.js @@ -5,16 +5,16 @@ * @import { LocationId, SwissNum } from './types.js' */ -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { encodeAscii } from '@endo/ascii/encode.js'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; import { encodeHex } from '@endo/hex'; /** - * @param {ArrayBufferLike} value + * @param {Uint8Array} value * @returns {string} */ export const toHex = value => { - return encodeHex(bytesFromImmutable(value)); + return encodeHex(thawedBytes(value)); }; /** @@ -49,14 +49,21 @@ export const locationToLocationId = location => { }; const swissnumDecoder = new TextDecoder('ascii', { fatal: true }); -const swissnumEncoder = new TextEncoder(); /** - * @param {ArrayBufferLike} value + * @param {SwissNum} value * @returns {string} */ export const decodeSwissnum = value => { - return swissnumDecoder.decode(bytesFromImmutable(value)); + const bytes = thawedBytes(value); + for (let i = 0; i < bytes.length; i += 1) { + if (bytes[i] > 0x7f) { + throw RangeError( + `Non-ASCII byte 0x${bytes[i].toString(16)} at offset ${i} of swissnum`, + ); + } + } + return swissnumDecoder.decode(bytes); }; /** @@ -64,17 +71,8 @@ export const decodeSwissnum = value => { * @returns {SwissNum} */ export const encodeSwissnum = value => { - // Validate the value is strictly valid ASCII - for (let i = 0; i < value.length; i += 1) { - const code = value.charCodeAt(i); - if (code > 127) { - throw new Error( - `Invalid ASCII character in swissnum at position ${i}: ${value[i]}`, - ); - } - } - // @ts-expect-error - Branded type: SwissNum is ArrayBufferLike at runtime - return bytesToImmutable(swissnumEncoder.encode(value)); + // @ts-expect-error - Branded type: SwissNum is Uint8Array at runtime + return frozenBytes(encodeAscii(value, 'swissnum')); }; /** @@ -91,8 +89,8 @@ export const encodeSwissnum = value => { * @returns {SwissNum} */ export const swissnumFromBytes = bytes => { - // @ts-expect-error - Branded type: SwissNum is ArrayBufferLike at runtime - return bytesToImmutable(bytes); + // @ts-expect-error - Branded type: SwissNum is Uint8Array at runtime + return frozenBytes(bytes); }; /** @@ -104,5 +102,5 @@ export const swissnumFromBytes = bytes => { * @returns {Uint8Array} */ export const swissnumToBytes = swissNum => { - return bytesFromImmutable(swissNum); + return thawedBytes(swissNum); }; diff --git a/packages/ocapn/src/codec-interface.d.ts b/packages/ocapn/src/codec-interface.d.ts index f8ac3dd097..e91b8a2e8d 100644 --- a/packages/ocapn/src/codec-interface.d.ts +++ b/packages/ocapn/src/codec-interface.d.ts @@ -22,7 +22,7 @@ export type TypeAndMaybeValue = | { type: 'boolean'; value: boolean } | { type: 'float64'; value: number } | { type: 'integer'; value: bigint } - | { type: 'bytestring'; value: ArrayBufferLike } + | { type: 'bytestring'; value: Uint8Array } | { type: 'string'; value: string } | { type: 'selector'; value: string } | { type: 'null'; value: null } @@ -38,7 +38,7 @@ export type TypeAndMaybeValue = export type RecordLabelInfo = | { type: 'selector'; value: string } | { type: 'string'; value: string } - | { type: 'bytestring'; value: ArrayBufferLike }; + | { type: 'bytestring'; value: Uint8Array }; /** * Common interface for OCapN readers (decoders). @@ -52,7 +52,7 @@ export interface OcapnReader { readInteger(): bigint; readFloat64(): number; readString(): string; - readBytestring(): ArrayBufferLike; + readBytestring(): Uint8Array; readSelectorAsString(): string; peekTypeHint(): TypeHint; @@ -88,7 +88,7 @@ export interface OcapnWriter { writeInteger(value: bigint): void; writeFloat64(value: number): void; writeString(value: string): void; - writeBytestring(value: ArrayBufferLike): void; + writeBytestring(value: Uint8Array): void; writeSelectorFromString(value: string): void; enterRecord(elementCount?: number): void; diff --git a/packages/ocapn/src/codecs/components.js b/packages/ocapn/src/codecs/components.js index 5b6f9bff07..b9c0a919fe 100644 --- a/packages/ocapn/src/codecs/components.js +++ b/packages/ocapn/src/codecs/components.js @@ -87,8 +87,8 @@ const OcapnSignatureEddsaCodec = exactList('OcapnSignatureEddsa', [ * @typedef {object} OcapnSignature * @property {'sig-val'} type * @property {'eddsa'} scheme - * @property {ArrayBufferLike} r - * @property {ArrayBufferLike} s + * @property {Uint8Array} r + * @property {Uint8Array} s */ // ['sig-val ['eddsa ['r r_value] ['s s_value]]] @@ -115,7 +115,7 @@ export const OcapnSignatureCodec = makeOcapnListComponentCodec( * @property {'ecc'} scheme * @property {'Ed25519'} curve * @property {'eddsa'} flags - * @property {ArrayBufferLike} q + * @property {Uint8Array} q */ const OcapnPublicKeyEccCodec = exactList('OcapnPublicKeyEcc', [ diff --git a/packages/ocapn/src/codecs/descriptors.js b/packages/ocapn/src/codecs/descriptors.js index d6f2b64629..21c6774a7b 100644 --- a/packages/ocapn/src/codecs/descriptors.js +++ b/packages/ocapn/src/codecs/descriptors.js @@ -11,7 +11,7 @@ */ import harden from '@endo/harden'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { makeCodec, makeRecordUnionCodec } from '../syrup/codec.js'; import { @@ -50,7 +50,7 @@ import { encodeSwissnum } from '../client/util.js'; * @property {OcapnLocation} exporterLocation * @property {SessionId} exporterSessionId * @property {PublicKeyId} gifterSideId - * @property {ArrayBufferLike} giftId + * @property {Uint8Array} giftId */ /** @@ -321,13 +321,7 @@ export const makeDescCodecs = referenceKit => { syrupReader => { const node = OcapnPeerCodec.read(syrupReader); const swissNum = syrupReader.readBytestring(); - const textDecoder = new TextDecoder('ascii', { fatal: true }); - const secretBytes = - swissNum instanceof Uint8Array - ? swissNum - : new Uint8Array(/** @type {ArrayBuffer} */ (swissNum.slice())); - const secret = textDecoder.decode(secretBytes); - const value = referenceKit.makeSturdyRef(node, secret); + const value = referenceKit.makeSturdyRef(node, swissNum); return value; }, /** @@ -345,12 +339,7 @@ export const makeDescCodecs = referenceKit => { // wire verbatim so non-ASCII swissnums (e.g. Spritely Goblins' // 24-byte randoms) round-trip without corruption. const wireSecret = - typeof secret === 'string' - ? /** @type {ArrayBufferLike} */ (encodeSwissnum(secret)) - : secret.buffer.slice( - secret.byteOffset, - secret.byteOffset + secret.byteLength, - ); + typeof secret === 'string' ? encodeSwissnum(secret) : secret; writer.writeBytestring(wireSecret); }, 2, // 2 fields: node, swissNum @@ -410,7 +399,7 @@ const makeSigEnvelope = (object, signature) => { * @param {OcapnLocation} exporterLocation * @param {SessionId} exporterSessionId * @param {PublicKeyId} gifterSideId - * @param {ArrayBufferLike} giftId + * @param {Uint8Array} giftId * @returns {HandoffGive} */ export const makeHandoffGiveDescriptor = ( @@ -433,12 +422,12 @@ export const makeHandoffGiveDescriptor = ( /** * @param {HandoffGive} handoffGive * @param {OcapnCodec} codec - * @returns {ArrayBufferLike} + * @returns {Uint8Array} */ export const serializeHandoffGive = (handoffGive, codec) => { const writer = codec.makeWriter(); DescHandoffGiveCodec.write(handoffGive, writer); - return bytesToImmutable(writer.getBytes()); + return frozenBytes(writer.getBytes()); }; /** @@ -486,10 +475,10 @@ export const makeHandoffReceiveSigEnvelope = (handoffReceive, signature) => { /** * @param {HandoffReceive} handoffReceive * @param {OcapnCodec} codec - * @returns {ArrayBufferLike} + * @returns {Uint8Array} */ export const serializeHandoffReceive = (handoffReceive, codec) => { const writer = codec.makeWriter(); DescHandoffReceiveCodec.write(handoffReceive, writer); - return bytesToImmutable(writer.getBytes()); + return frozenBytes(writer.getBytes()); }; diff --git a/packages/ocapn/src/codecs/passable.js b/packages/ocapn/src/codecs/passable.js index 67cdd125b0..25525a1238 100644 --- a/packages/ocapn/src/codecs/passable.js +++ b/packages/ocapn/src/codecs/passable.js @@ -176,8 +176,15 @@ export const makePassableCodecs = descCodecs => { syrupWriter.writeString(value); } else if (typeof value === 'bigint') { syrupWriter.writeInteger(value); - } else if (value instanceof ArrayBuffer) { - syrupWriter.writeBytestring(value); + } else if (value instanceof Uint8Array || value instanceof ArrayBuffer) { + // A byteArray passable is now a `Uint8Array` (issue #573); the stale + // check here previously matched only a bare `ArrayBuffer`. Accept + // both like the sibling NumberPrefix dispatchers and normalize a + // cross-version raw `ArrayBuffer` to a `Uint8Array` at this boundary + // so writeBytestring resolves no buffer-vs-view disjunction. + syrupWriter.writeBytestring( + value instanceof Uint8Array ? value : new Uint8Array(value), + ); } else { throw new Error( `Unexpected value ${value} for OcapnPassableNumberPrefixUnionCodec`, diff --git a/packages/ocapn/src/cryptography.js b/packages/ocapn/src/cryptography.js index 48b9165b71..bc4151fe88 100644 --- a/packages/ocapn/src/cryptography.js +++ b/packages/ocapn/src/cryptography.js @@ -1,8 +1,7 @@ // @ts-check /* global crypto */ import harden from '@endo/harden'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; +import { frozenBytes, thawedBytes } from '@endo/immutable-arraybuffer'; import { concatBytes } from '@endo/bytes/concat.js'; import { ed25519 } from '@noble/curves/ed25519.js'; import { sha256 } from '@noble/hashes/sha2.js'; @@ -35,15 +34,15 @@ const sessionIdHashPrefixBytes = textEncoder.encode('prot0'); /** * @typedef {object} OcapnPublicKey * @property {PublicKeyId} id - * @property {ArrayBufferLike} bytes + * @property {Uint8Array} bytes * @property {OcapnPublicKeyDescriptor} descriptor - * @property {(msg: ArrayBufferLike, sig: OcapnSignature) => void} assertSignatureValid - Throws if signature is invalid + * @property {(msg: Uint8Array, sig: OcapnSignature) => void} assertSignatureValid - Throws if signature is invalid */ /** * @typedef {object} OcapnKeyPair * @property {OcapnPublicKey} publicKey - * @property {(msg: ArrayBufferLike) => OcapnSignature} sign + * @property {(msg: Uint8Array) => OcapnSignature} sign */ /** @@ -51,13 +50,13 @@ const sessionIdHashPrefixBytes = textEncoder.encode('prot0'); * @returns {Uint8Array} */ const ocapNSignatureToBytes = sig => { - const rBytes = bytesFromImmutable(sig.r); - const sBytes = bytesFromImmutable(sig.s); + const rBytes = thawedBytes(sig.r); + const sBytes = thawedBytes(sig.s); return concatBytes([rBytes, sBytes]); }; /** - * @param {ArrayBufferLike} publicKeyBytes + * @param {Uint8Array} publicKeyBytes * @returns {OcapnPublicKeyDescriptor} */ const makePublicKeyDescriptor = publicKeyBytes => { @@ -71,13 +70,13 @@ const makePublicKeyDescriptor = publicKeyBytes => { }; /** - * @param {ArrayBufferLike} peerIdOne - * @param {ArrayBufferLike} peerIdTwo + * @param {Uint8Array} peerIdOne + * @param {Uint8Array} peerIdTwo * @returns {SessionId} */ export const makeSessionId = (peerIdOne, peerIdTwo) => { - const peerIdOneBytes = bytesFromImmutable(peerIdOne); - const peerIdTwoBytes = bytesFromImmutable(peerIdTwo); + const peerIdOneBytes = thawedBytes(peerIdOne); + const peerIdTwoBytes = thawedBytes(peerIdTwo); const result = compareUint8Arrays(peerIdOneBytes, peerIdTwoBytes); const peerIds = result < 0 @@ -87,7 +86,7 @@ export const makeSessionId = (peerIdOne, peerIdTwo) => { const hash1 = sha256(sessionIdBytes); const hash2 = sha256(hash1); // @ts-expect-error - Branded type: SessionId is ArrayBufferLike at runtime - return bytesToImmutable(hash2); + return frozenBytes(hash2); }; /** @@ -95,17 +94,17 @@ export const makeSessionId = (peerIdOne, peerIdTwo) => { * browsers, and can be shimmed on embedded engines (an engine profile * that never mints gift ids may shim a throwing stub). * - * @returns {ArrayBufferLike} + * @returns {Uint8Array} */ export const randomGiftId = () => { const bytes = new Uint8Array(16); crypto.getRandomValues(bytes); - return bytesToImmutable(bytes); + return frozenBytes(bytes); }; /** * @typedef {object} Cryptography - * @property {(publicKeyBytes: ArrayBufferLike) => OcapnPublicKey} makeOcapnPublicKey + * @property {(publicKeyBytes: Uint8Array) => OcapnPublicKey} makeOcapnPublicKey * @property {(publicKeyDescriptor: OcapnPublicKeyDescriptor) => OcapnPublicKey} publicKeyDescriptorToPublicKey * @property {(privateKeyBytes: Uint8Array) => OcapnKeyPair} makeOcapnKeyPairFromPrivateKey * @property {() => OcapnKeyPair} makeOcapnKeyPair @@ -113,7 +112,7 @@ export const randomGiftId = () => { * @property {(location: OcapnLocation, keyPair: OcapnKeyPair, binding: ArrayBufferLike) => OcapnSignature} signLocation * @property {(location: OcapnLocation, signature: OcapnSignature, publicKey: OcapnPublicKey, binding: ArrayBufferLike) => void} assertLocationSignatureValid * @property {(handoffGive: HandoffGive, keyPair: OcapnKeyPair) => OcapnSignature} signHandoffGive - * @property {(receiverPublicKeyForGifter: OcapnPublicKey, exporterLocation: OcapnLocation, gifterExporterSessionId: SessionId, gifterSideId: PublicKeyId, giftId: ArrayBufferLike, gifterKeyForExporter: OcapnKeyPair) => HandoffGiveSigEnvelope} makeSignedHandoffGive + * @property {(receiverPublicKeyForGifter: OcapnPublicKey, exporterLocation: OcapnLocation, gifterExporterSessionId: SessionId, gifterSideId: PublicKeyId, giftId: Uint8Array, gifterKeyForExporter: OcapnKeyPair) => HandoffGiveSigEnvelope} makeSignedHandoffGive * @property {(handoffGive: HandoffGive, signature: OcapnSignature, publicKey: OcapnPublicKey) => void} assertHandoffGiveSignatureValid * @property {(handoffReceive: HandoffReceive, keyPair: OcapnKeyPair) => OcapnSignature} signHandoffReceive * @property {(handoffReceive: HandoffReceive, signature: OcapnSignature, publicKey: OcapnPublicKey) => void} assertHandoffReceiveSignatureValid @@ -141,11 +140,11 @@ export const makeCryptography = codec => { const hash1 = sha256(publicKeyDescriptorBytes); const hash2 = sha256(hash1); // @ts-expect-error - Branded type: PublicKeyId is ArrayBufferLike at runtime - return bytesToImmutable(hash2); + return frozenBytes(hash2); }; /** - * @param {ArrayBufferLike} publicKeyBytes + * @param {Uint8Array} publicKeyBytes * @returns {OcapnPublicKey} */ const makeOcapnPublicKey = publicKeyBytes => { @@ -155,13 +154,13 @@ export const makeCryptography = codec => { bytes: publicKeyBytes, descriptor: publicKeyDescriptor, /** - * @param {ArrayBufferLike} msgBytes + * @param {Uint8Array} msgBytes * @param {OcapnSignature} ocapnSig */ assertSignatureValid: (msgBytes, ocapnSig) => { const sigBytes = ocapNSignatureToBytes(ocapnSig); - const msgUint8 = bytesFromImmutable(msgBytes); - const pkUint8 = bytesFromImmutable(publicKeyBytes); + const msgUint8 = thawedBytes(msgBytes); + const pkUint8 = thawedBytes(publicKeyBytes); const isValid = ed25519.verify(sigBytes, msgUint8, pkUint8); if (!isValid) { throw new Error('Invalid signature'); @@ -176,17 +175,17 @@ export const makeCryptography = codec => { */ const makeOcapnKeyPairFromPrivateKey = privateKeyBytes => { const publicKeyBytes = ed25519.getPublicKey(privateKeyBytes); - const publicKeyBuffer = bytesToImmutable(publicKeyBytes); + const publicKeyBuffer = frozenBytes(publicKeyBytes); return { publicKey: makeOcapnPublicKey(publicKeyBuffer), sign: msg => { - const msgBytes = bytesFromImmutable(msg); + const msgBytes = thawedBytes(msg); const sigBytes = ed25519.sign(msgBytes, privateKeyBytes); return { type: 'sig-val', scheme: 'eddsa', - r: bytesToImmutable(sigBytes.slice(0, 32)), - s: bytesToImmutable(sigBytes.slice(32)), + r: frozenBytes(sigBytes.slice(0, 32)), + s: frozenBytes(sigBytes.slice(32)), }; }, }; @@ -275,7 +274,7 @@ export const makeCryptography = codec => { // wire bytes the OCapN python reference suite produces and // verifies, so tcp-testing-only interop is bit-for-bit unchanged. if (bindingBytes.length === 0) { - return bytesToImmutable(myLocationBytes); + return frozenBytes(myLocationBytes); } // With a non-empty binding (e.g. the Noise handshake hash on the // np netlayer), prepend a domain-separator and length-prefix the @@ -299,7 +298,7 @@ export const makeCryptography = codec => { out.set(bindingBytes, offset); offset += bindingBytes.length; out.set(myLocationBytes, offset); - return bytesToImmutable(out); + return frozenBytes(out); }; /** diff --git a/packages/ocapn/src/hub/hub.js b/packages/ocapn/src/hub/hub.js index 97bf56681b..379bea24b0 100644 --- a/packages/ocapn/src/hub/hub.js +++ b/packages/ocapn/src/hub/hub.js @@ -1,6 +1,5 @@ // @ts-check import harden from '@endo/harden'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; import { Far } from '@endo/marshal'; import { @@ -98,18 +97,27 @@ import { makeSturdyRef } from '../client/sturdyrefs.js'; const BOOTSTRAP_POSITION = '0'; const STATE_VERSION = 2; -/** @param {ArrayBufferLike | Uint8Array} bytes */ +const { isView } = ArrayBuffer; + +/** + * Hex-encode a byteArray `Uint8Array`. A byteArray passable is always a + * whole-buffer-spanning `Uint8Array` (issue #573), never a bare + * `ArrayBufferLike` nor some other `ArrayBufferView`, so this is typed + * `Uint8Array`. The runtime `isView` branch is *not* buffer-vs-view type + * generality — it is tolerance for the single emulation infidelity of the + * `@endo/immutable-arraybuffer` shim: an emulated frozen wrapper (as + * `makeSessionId`/`frozenBytes` yield) is *typed* `Uint8Array` yet is a plain + * object that reports `ArrayBuffer.isView === false` and is not + * integer-indexable, so it must first be copied into a fresh mutable + * `Uint8Array`. A genuine view — including one over a native immutable buffer + * — is read in place. This mirrors `@endo/bytes`' `toIndexableUint8`. + * + * @param {Uint8Array} bytes + */ const hexFromBytes = bytes => { - let view; - if (bytes instanceof Uint8Array) { - view = bytes; - } else { - view = new Uint8Array(/** @type {ArrayBuffer} */ (bytes)); - if (view.length === 0 && bytes.byteLength > 0) { - // An endo immutable ArrayBuffer: view it via its transfer seam. - view = bytesFromImmutable(bytes); - } - } + const view = isView(bytes) + ? bytes + : new Uint8Array(/** @type {Uint8Array} */ (bytes).slice(0)); return Array.from(view, byte => byte.toString(16).padStart(2, '0')).join(''); }; @@ -126,7 +134,7 @@ const bytesFromHex = hex => { * The publications table key for a swissnum in either of its accepted * forms. * - * @param {string | Uint8Array | ArrayBufferLike} swissnum + * @param {string | Uint8Array} swissnum */ const swissnumHex = swissnum => typeof swissnum === 'string' @@ -1926,7 +1934,7 @@ export const makeOcapnHub = ({ * instead of being dropped (the policy for sessions from beyond * the process boundary) * @param {(error: unknown) => void} [powers.onAbort] - * @param {{ sessionId: Uint8Array | ArrayBufferLike, peerPublicKeyQ: Uint8Array | ArrayBufferLike, selfPrivateKeyBytes?: Uint8Array | ArrayBufferLike }} [powers.identity] + * @param {{ sessionId: Uint8Array, peerPublicKeyQ: Uint8Array, selfPrivateKeyBytes?: Uint8Array }} [powers.identity] * the session's wire identity from the embedder's handshake * (including the hub side's session private key, which signs * gift handoff receives); omitted on reattach, the persisted diff --git a/packages/ocapn/src/netlayers/websocket.js b/packages/ocapn/src/netlayers/websocket.js index 69e3aac5ab..f0d753ff1c 100644 --- a/packages/ocapn/src/netlayers/websocket.js +++ b/packages/ocapn/src/netlayers/websocket.js @@ -3,8 +3,7 @@ import { randomBytes } from 'node:crypto'; import { WebSocket, WebSocketServer } from 'ws'; import harden from '@endo/harden'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; import { makeCryptography } from '../cryptography.js'; import { locationToLocationId } from '../client/util.js'; @@ -51,7 +50,7 @@ const BASE32_DECODE_TABLE = new Map( /** * @typedef {object} InitPeerAuth * @property {'init:peer-auth'} type - * @property {ArrayBufferLike} payload + * @property {Uint8Array} payload */ /** @@ -173,7 +172,7 @@ const encodeInitPeerAuth = payload => { InitPeerAuthCodec.write( { type: 'init:peer-auth', - payload: bytesToImmutable(payload), + payload: frozenBytes(payload), }, syrupWriter, ); @@ -288,9 +287,7 @@ export const makeWebSocketNetLayer = async ({ // init:peer-auth records on the wire. const cryptography = makeCryptography(syrupCodec); const designatorKeyPair = cryptography.makeOcapnKeyPair(); - const designatorPublicKey = bytesFromImmutable( - designatorKeyPair.publicKey.bytes, - ); + const designatorPublicKey = thawedBytes(designatorKeyPair.publicKey.bytes); const designator = base32Encode(designatorPublicKey); const server = new WebSocketServer({ @@ -363,7 +360,7 @@ export const makeWebSocketNetLayer = async ({ ); } const remotePublicKey = cryptography.makeOcapnPublicKey( - bytesToImmutable(remotePublicKeyBytes), + frozenBytes(remotePublicKeyBytes), ); logger.info('Connecting to websocket', { wsUrl }); @@ -395,7 +392,7 @@ export const makeWebSocketNetLayer = async ({ try { const envelope = decodeInitPeerAuthSigEnvelope(messageBytes); remotePublicKey.assertSignatureValid( - bytesToImmutable(challengeMessage), + frozenBytes(challengeMessage), envelope.signature, ); socketState.authenticated = true; @@ -464,9 +461,7 @@ export const makeWebSocketNetLayer = async ({ const initPeerAuth = decodeInitPeerAuth(messageBytes); // Sign the received bytes verbatim. The wrapping `init:peer-auth` // record prevents this from being used as a generic signing oracle. - const signature = designatorKeyPair.sign( - bytesToImmutable(messageBytes), - ); + const signature = designatorKeyPair.sign(frozenBytes(messageBytes)); const responseBytes = encodeInitPeerAuthSigEnvelope( initPeerAuth, signature, diff --git a/packages/ocapn/src/syrup/codec.js b/packages/ocapn/src/syrup/codec.js index cb5e4add87..92657770a3 100644 --- a/packages/ocapn/src/syrup/codec.js +++ b/packages/ocapn/src/syrup/codec.js @@ -7,8 +7,7 @@ import harden from '@endo/harden'; import { bytesFromText } from '@endo/bytes/from-string.js'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; import { ocapnPassStyleOf } from '../codecs/ocapn-pass-style.js'; @@ -19,11 +18,11 @@ import { ocapnPassStyleOf } from '../codecs/ocapn-pass-style.js'; const labelTextDecoder = new TextDecoder('utf-8', { fatal: true }); /** - * @param {ArrayBufferLike} buffer + * @param {Uint8Array} buffer * @returns {string} */ const decodeBytestringLabel = buffer => - labelTextDecoder.decode(bytesFromImmutable(buffer)); + labelTextDecoder.decode(thawedBytes(buffer)); /** * A codec that can read and write values using any OCapN reader/writer. * Works with both Syrup and CBOR codecs. @@ -182,13 +181,18 @@ export const makeExpectedLengthBytestringCodec = (codecName, length) => { return bytestring; }, write: (value, syrupWriter) => { - if (!(value instanceof ArrayBuffer)) { - throw Error(`Expected ArrayBuffer, got ${typeof value}`); + if (!(value instanceof Uint8Array || value instanceof ArrayBuffer)) { + throw Error(`Expected Uint8Array or ArrayBuffer, got ${typeof value}`); } if (value.byteLength !== length) { throw Error(`Expected length ${length}, got ${value.byteLength}`); } - syrupWriter.writeBytestring(value); + // Cross-version tolerance: a raw ArrayBuffer from an older peer is + // normalized to a Uint8Array here so writeBytestring resolves no + // buffer-vs-view disjunction. + syrupWriter.writeBytestring( + value instanceof Uint8Array ? value : new Uint8Array(value), + ); }, }); }; @@ -212,8 +216,13 @@ export const NumberPrefixCodec = makeCodec('NumberPrefix', { write: (value, syrupWriter) => { if (typeof value === 'string') { syrupWriter.writeString(value); - } else if (value instanceof ArrayBuffer) { - syrupWriter.writeBytestring(value); + } else if (value instanceof Uint8Array || value instanceof ArrayBuffer) { + // The byteArray pass style is now a Uint8Array (issue #573); a raw + // ArrayBuffer from an older peer is normalized to a Uint8Array here so + // writeBytestring resolves no buffer-vs-view disjunction. + syrupWriter.writeBytestring( + value instanceof Uint8Array ? value : new Uint8Array(value), + ); } else if (typeof value === 'bigint') { syrupWriter.writeInteger(value); } else { @@ -435,7 +444,7 @@ export const makeRecordCodec = ( } else if (effectiveLabelType === 'string') { syrupWriter.writeString(label); } else if (effectiveLabelType === 'bytestring') { - syrupWriter.writeBytestring(bytesToImmutable(bytesFromText(label))); + syrupWriter.writeBytestring(frozenBytes(bytesFromText(label))); } writeBody(value, syrupWriter); syrupWriter.exitRecord(); diff --git a/packages/ocapn/src/syrup/compare.js b/packages/ocapn/src/syrup/compare.js index 80d993cf10..863087c008 100644 --- a/packages/ocapn/src/syrup/compare.js +++ b/packages/ocapn/src/syrup/compare.js @@ -1,6 +1,6 @@ // @ts-check -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; +import { thawedBytes } from '@endo/immutable-arraybuffer'; /** * @param {Uint8Array} left @@ -86,14 +86,15 @@ export function compareUint8Arrays( } /** - * Compare two immutable ArrayBuffers - * @param {ArrayBufferLike} left - * @param {ArrayBufferLike} right + * Compare two byteArray-passable values. Each is a `Uint8Array`: a plain + * mutable one, a genuine frozen view over an immutable `ArrayBuffer`, or an + * emulated `@endo/immutable-arraybuffer` wrapper (which reports + * `ArrayBuffer.isView === false`). + * + * @param {Uint8Array} left + * @param {Uint8Array} right * @returns {number} */ export const compareImmutableArrayBuffers = (left, right) => { - return compareUint8Arrays( - bytesFromImmutable(left), - bytesFromImmutable(right), - ); + return compareUint8Arrays(thawedBytes(left), thawedBytes(right)); }; diff --git a/packages/ocapn/src/syrup/decode.js b/packages/ocapn/src/syrup/decode.js index 2241012c41..78cbb6a7bb 100644 --- a/packages/ocapn/src/syrup/decode.js +++ b/packages/ocapn/src/syrup/decode.js @@ -1,6 +1,6 @@ // @ts-check -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { BufferReader } from './buffer-reader.js'; @@ -64,7 +64,7 @@ function readBoolean(bufferReader, name) { /** @typedef {{type: 'float64', value: number}} ReadTypeFloat64Result */ // Number-prefixed types, value is read /** @typedef {{type: 'integer', value: bigint}} ReadTypeIntegerResult */ -/** @typedef {{type: 'bytestring', value: ArrayBufferLike}} ReadTypeBytestringResult */ +/** @typedef {{type: 'bytestring', value: Uint8Array}} ReadTypeBytestringResult */ /** @typedef {{type: 'string', value: string}} ReadTypeStringResult */ /** @typedef {{type: 'selector', value: string}} ReadTypeSelectorResult */ /** @typedef {ReadTypeBooleanResult | ReadTypeFloat64Result | ReadTypeIntegerResult | ReadTypeBytestringResult | ReadTypeStringResult | ReadTypeSelectorResult} ReadTypeAtomResult */ @@ -134,7 +134,7 @@ function readTypeAndMaybeValue(bufferReader, name) { const number = Number.parseInt(numberString, 10); const valueBytes = bufferReader.read(number); // Convert Uint8Array to immutable ArrayBuffer - const arrayBuffer = bytesToImmutable(valueBytes); + const arrayBuffer = frozenBytes(valueBytes); return { type: 'bytestring', value: arrayBuffer }; } if (typeByte === STRING_START) { @@ -201,7 +201,7 @@ function readSelectorAsString(bufferReader, name) { /** * @param {BufferReader} bufferReader * @param {string} name - * @returns {ArrayBufferLike} + * @returns {Uint8Array} */ function readBytestring(bufferReader, name) { return readAndAssertType(bufferReader, 'bytestring', name); @@ -210,7 +210,7 @@ function readBytestring(bufferReader, name) { /** * @param {BufferReader} bufferReader * @param {string} name - * @returns {{value: string, type: 'selector'} | {value: ArrayBufferLike, type: 'bytestring'} | {value: string, type: 'string'}} + * @returns {{value: string, type: 'selector'} | {value: Uint8Array, type: 'bytestring'} | {value: string, type: 'string'}} * see https://github.com/ocapn/syrup/issues/22 */ function readRecordLabel(bufferReader, name) { @@ -385,7 +385,7 @@ export class SyrupReader { } /** - * @returns {{value: string, type: 'selector'} | {value: ArrayBufferLike, type: 'bytestring'} | {value: string, type: 'string'}} + * @returns {{value: string, type: 'selector'} | {value: Uint8Array, type: 'bytestring'} | {value: string, type: 'string'}} */ readRecordLabel() { return readRecordLabel(this.bufferReader, this.name); @@ -474,7 +474,7 @@ export class SyrupReader { } /** - * @returns {ArrayBufferLike} + * @returns {Uint8Array} */ readBytestring() { return readBytestring(this.bufferReader, this.name); diff --git a/packages/ocapn/src/syrup/encode.js b/packages/ocapn/src/syrup/encode.js index 3b6efc29f4..0381cfe68e 100644 --- a/packages/ocapn/src/syrup/encode.js +++ b/packages/ocapn/src/syrup/encode.js @@ -74,12 +74,24 @@ function writeSelectorFromString(bufferWriter, value) { /** * @param {import('./buffer-writer.js').BufferWriter} bufferWriter - * @param {ArrayBufferLike} value + * @param {Uint8Array} value */ function writeBytestring(bufferWriter, value) { - // Convert ArrayBuffer to Uint8Array for internal operations - // Immutable ArrayBuffers need to be sliced first - const mutableBuffer = value.slice(); + // Convert to a fresh mutable Uint8Array for internal operations. + // The byteArray pass style is a frozen Uint8Array backed by an + // immutable ArrayBuffer; calling `value.slice()` on the view returns + // a fresh mutable Uint8Array of the right window, which is exactly + // what `writeStringlike` needs. An emulated `@endo/immutable-arraybuffer` + // wrapper (which reports `ArrayBuffer.isView === false`) is likewise + // sliced into a fresh mutable buffer and wrapped. + if (ArrayBuffer.isView(value)) { + const bytes = new Uint8Array( + value.buffer.slice(value.byteOffset, value.byteOffset + value.byteLength), + ); + writeStringlike(bufferWriter, bytes, ':'); + return; + } + const mutableBuffer = /** @type {Uint8Array} */ (value).slice(); const bytes = new Uint8Array(mutableBuffer); writeStringlike(bufferWriter, bytes, ':'); } @@ -162,7 +174,7 @@ export class SyrupWriter { } /** - * @param {ArrayBufferLike} value + * @param {Uint8Array} value */ writeBytestring(value) { writeBytestring(this.#bufferWriter, value); diff --git a/packages/ocapn/src/syrup/js-representation.js b/packages/ocapn/src/syrup/js-representation.js index 405b904956..e8715acbc9 100644 --- a/packages/ocapn/src/syrup/js-representation.js +++ b/packages/ocapn/src/syrup/js-representation.js @@ -90,8 +90,13 @@ export const NumberPrefixCodecWithSelectorAsSymbol = { } else if (typeof value === 'symbol') { const selectorString = getSyrupSelectorName(value); syrupWriter.writeSelectorFromString(selectorString); - } else if (value instanceof ArrayBuffer) { - syrupWriter.writeBytestring(value); + } else if (value instanceof Uint8Array || value instanceof ArrayBuffer) { + // The byteArray pass style is now a Uint8Array (issue #573); a raw + // ArrayBuffer from an older peer is normalized to a Uint8Array here so + // writeBytestring resolves no buffer-vs-view disjunction. + syrupWriter.writeBytestring( + value instanceof Uint8Array ? value : new Uint8Array(value), + ); } else if (typeof value === 'bigint') { syrupWriter.writeInteger(value); } else { @@ -131,7 +136,9 @@ export const AnyCodec = makeTypeHintUnionCodec( } else if (value instanceof Set) { // eslint-disable-next-line no-use-before-define return SetCodec; - } else if (value instanceof ArrayBuffer) { + } else if (value instanceof Uint8Array || value instanceof ArrayBuffer) { + // byteArray dispatch: a Uint8Array is the current shape; a raw + // ArrayBuffer is the legacy shape, retained for cross-version use. return BytestringCodec; } else if (typeof value === 'object' && value !== null) { if (value[Symbol.toStringTag] === 'Record') { diff --git a/packages/ocapn/test/bytewise-compare.test.js b/packages/ocapn/test/bytewise-compare.test.js index 102133534f..6b268b3d22 100644 --- a/packages/ocapn/test/bytewise-compare.test.js +++ b/packages/ocapn/test/bytewise-compare.test.js @@ -1,7 +1,7 @@ // @ts-check import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { compareImmutableArrayBuffers, @@ -27,29 +27,29 @@ test('right longer', t => { }); test('compareImmutableArrayBuffers - equal buffers', t => { - const buffer1 = bytesToImmutable(bytesFromText('hello')); - const buffer2 = bytesToImmutable(bytesFromText('hello')); + const buffer1 = frozenBytes(bytesFromText('hello')); + const buffer2 = frozenBytes(bytesFromText('hello')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), 0); }); test('compareImmutableArrayBuffers - left less than right', t => { - const buffer1 = bytesToImmutable(bytesFromText('abc')); - const buffer2 = bytesToImmutable(bytesFromText('xyz')); + const buffer1 = frozenBytes(bytesFromText('abc')); + const buffer2 = frozenBytes(bytesFromText('xyz')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), -1); }); test('compareImmutableArrayBuffers - left greater than right', t => { - const buffer1 = bytesToImmutable(bytesFromText('xyz')); - const buffer2 = bytesToImmutable(bytesFromText('abc')); + const buffer1 = frozenBytes(bytesFromText('xyz')); + const buffer2 = frozenBytes(bytesFromText('abc')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), 1); }); test('compareImmutableArrayBuffers - left is prefix of right', t => { - const buffer1 = bytesToImmutable(bytesFromText('hello')); - const buffer2 = bytesToImmutable(bytesFromText('helloworld')); + const buffer1 = frozenBytes(bytesFromText('hello')); + const buffer2 = frozenBytes(bytesFromText('helloworld')); const result = compareImmutableArrayBuffers(buffer1, buffer2); t.true(result < 0, 'left should be less than right'); @@ -57,31 +57,31 @@ test('compareImmutableArrayBuffers - left is prefix of right', t => { }); test('compareImmutableArrayBuffers - right is prefix of left', t => { - const buffer1 = bytesToImmutable(bytesFromText('helloworld')); - const buffer2 = bytesToImmutable(bytesFromText('hello')); + const buffer1 = frozenBytes(bytesFromText('helloworld')); + const buffer2 = frozenBytes(bytesFromText('hello')); const result = compareImmutableArrayBuffers(buffer1, buffer2); t.true(result > 0, 'left should be greater than right'); }); test('compareImmutableArrayBuffers - empty buffers', t => { - const buffer1 = bytesToImmutable(bytesFromText('')); - const buffer2 = bytesToImmutable(bytesFromText('')); + const buffer1 = frozenBytes(bytesFromText('')); + const buffer2 = frozenBytes(bytesFromText('')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), 0); }); test('compareImmutableArrayBuffers - empty vs non-empty', t => { - const buffer1 = bytesToImmutable(bytesFromText('')); - const buffer2 = bytesToImmutable(bytesFromText('a')); + const buffer1 = frozenBytes(bytesFromText('')); + const buffer2 = frozenBytes(bytesFromText('a')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), -1); t.is(compareImmutableArrayBuffers(buffer2, buffer1), 1); }); test('compareImmutableArrayBuffers - binary data', t => { - const buffer1 = bytesToImmutable(new Uint8Array([0x00, 0x01, 0x02])); - const buffer2 = bytesToImmutable(new Uint8Array([0x00, 0x01, 0x03])); + const buffer1 = frozenBytes(new Uint8Array([0x00, 0x01, 0x02])); + const buffer2 = frozenBytes(new Uint8Array([0x00, 0x01, 0x03])); t.is(compareImmutableArrayBuffers(buffer1, buffer2), -1); t.is(compareImmutableArrayBuffers(buffer2, buffer1), 1); @@ -89,8 +89,8 @@ test('compareImmutableArrayBuffers - binary data', t => { test('compareImmutableArrayBuffers - bytewise comparison', t => { // Test that comparison is bytewise, not lexicographic - const buffer1 = bytesToImmutable(new Uint8Array([0xff])); - const buffer2 = bytesToImmutable(new Uint8Array([0x00, 0x00])); + const buffer1 = frozenBytes(new Uint8Array([0xff])); + const buffer2 = frozenBytes(new Uint8Array([0x00, 0x00])); // 0xff > 0x00, so buffer1 > buffer2 despite being shorter t.is(compareImmutableArrayBuffers(buffer1, buffer2), 1); diff --git a/packages/ocapn/test/cbor/decode.test.js b/packages/ocapn/test/cbor/decode.test.js index 3a51ae3aa1..c9eeda4566 100644 --- a/packages/ocapn/test/cbor/decode.test.js +++ b/packages/ocapn/test/cbor/decode.test.js @@ -350,7 +350,7 @@ test('readTypeAndMaybeValue for bytestring', t => { const reader = decode('44deadbeef'); const result = reader.readTypeAndMaybeValue(); t.is(result.type, 'bytestring'); - const value = /** @type {ArrayBufferLike} */ (result.value); + const value = /** @type {Uint8Array} */ (result.value); t.is(value.byteLength, 4); }); diff --git a/packages/ocapn/test/cbor/diagnostic-equals.test.js b/packages/ocapn/test/cbor/diagnostic-equals.test.js new file mode 100644 index 0000000000..22e2f44b42 --- /dev/null +++ b/packages/ocapn/test/cbor/diagnostic-equals.test.js @@ -0,0 +1,61 @@ +// @ts-check + +// Regression coverage for `equals`/`diagnosticEquals` byte comparison against +// emulated frozen byteArray passables (issue #573 narrowing). An emulated +// `@endo/immutable-arraybuffer` wrapper is `instanceof Uint8Array` but reports +// `ArrayBuffer.isView === false` and reads `undefined` from `wrapper[i]`; a +// naive `actual instanceof Uint8Array ? actual : new Uint8Array(actual)` then +// integer-indexed comparison collapses distinct wrappers to equal +// (`undefined === undefined`). These tests only pass if the helper thaws the +// wrapper (via `ArrayBuffer.isView`) before comparing. + +import test from '@endo/ses-ava/test.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; +import { equals, diagnosticEquals } from '../../src/cbor/diagnostic/util.js'; + +test('equals: distinct emulated byteArrays with different bytes are unequal', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const b = frozenBytes(new Uint8Array([1, 2, 4])); + t.false(equals(a, b)); +}); + +test('equals: distinct emulated byteArrays with equal bytes are equal', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const aAgain = frozenBytes(new Uint8Array([1, 2, 3])); + t.true(equals(a, aAgain)); +}); + +test('equals: emulated byteArray vs genuine Uint8Array', t => { + const emulated = frozenBytes(new Uint8Array([1, 2, 3])); + const genuine = new Uint8Array([1, 2, 3]); + t.true(equals(emulated, genuine)); + t.true(equals(genuine, emulated)); + t.false(equals(emulated, new Uint8Array([1, 2, 4]))); +}); + +test('equals: emulated byteArrays of different lengths are unequal', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const abcd = frozenBytes(new Uint8Array([1, 2, 3, 4])); + t.false(equals(a, abcd)); +}); + +test('equals: bytes vs non-bytes is unequal, not a throw', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + t.false(equals(a, 3)); + t.false(equals(a, 'abc')); +}); + +test('equals: genuine ArrayBuffer inputs compare by bytes', t => { + const a = new Uint8Array([1, 2, 3]).buffer; + const b = new Uint8Array([1, 2, 3]).buffer; + const c = new Uint8Array([1, 2, 4]).buffer; + t.true(equals(a, b)); + t.false(equals(a, c)); +}); + +test('diagnosticEquals alias resolves to the same comparison', t => { + const a = frozenBytes(new Uint8Array([1, 2, 3])); + const b = frozenBytes(new Uint8Array([1, 2, 4])); + t.false(diagnosticEquals(a, b)); + t.true(diagnosticEquals(a, frozenBytes(new Uint8Array([1, 2, 3])))); +}); diff --git a/packages/ocapn/test/cbor/encode.test.js b/packages/ocapn/test/cbor/encode.test.js index c4b58a540c..0c0e6dba76 100644 --- a/packages/ocapn/test/cbor/encode.test.js +++ b/packages/ocapn/test/cbor/encode.test.js @@ -1,7 +1,7 @@ // @ts-check import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { makeCborWriter } from '../../src/cbor/encode.js'; import { cborToDiagnostic, @@ -205,7 +205,7 @@ test('encode string rejects unpaired surrogates', t => { test('encode empty byte string', t => { const { hex, diagnostic } = encode(w => - w.writeBytestring(new ArrayBuffer(0)), + w.writeBytestring(new Uint8Array(0)), ); t.is(hex, '40'); // Major 2, length 0 t.is(diagnostic, "h''"); @@ -214,7 +214,7 @@ test('encode empty byte string', t => { test('encode byte string', t => { const bytes = new Uint8Array([0xde, 0xad, 0xbe, 0xef]); const { hex, diagnostic } = encode(w => - w.writeBytestring(bytesToImmutable(bytes)), + w.writeBytestring(frozenBytes(bytes)), ); t.is(hex, '44deadbeef'); // Major 2, length 4, bytes t.is(diagnostic, "h'deadbeef'"); diff --git a/packages/ocapn/test/cbor/interop.test.js b/packages/ocapn/test/cbor/interop.test.js index 28e6a3aef5..ce180e19e7 100644 --- a/packages/ocapn/test/cbor/interop.test.js +++ b/packages/ocapn/test/cbor/interop.test.js @@ -11,7 +11,7 @@ import { Buffer } from 'buffer'; import test from '@endo/ses-ava/test.js'; import cbor from 'cbor'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { makeCborWriter } from '../../src/cbor/encode.js'; import { bytesToHexString, @@ -238,7 +238,7 @@ test('interop: emoji string (supplementary characters)', async t => { test('interop: empty byte string', async t => { const { value, diagnostic } = await encodeAndValidate(w => - w.writeBytestring(new ArrayBuffer(0)), + w.writeBytestring(new Uint8Array(0)), ); t.true(value instanceof Uint8Array || value instanceof Buffer); t.is(value.length, 0); @@ -248,7 +248,7 @@ test('interop: empty byte string', async t => { test('interop: byte string', async t => { const bytes = new Uint8Array([0xde, 0xad, 0xbe, 0xef]); const { value, diagnostic } = await encodeAndValidate(w => - w.writeBytestring(bytesToImmutable(bytes)), + w.writeBytestring(frozenBytes(bytes)), ); t.is(value.length, 4); t.deepEqual(Array.from(value), [0xde, 0xad, 0xbe, 0xef]); @@ -464,7 +464,7 @@ test('interop: complex nested structure', async t => { writer.writeArrayHeader(3); writer.writeBoolean(true); writer.writeFloat64(1.5); - writer.writeBytestring(bytesToImmutable(new Uint8Array([1, 2, 3]))); + writer.writeBytestring(frozenBytes(new Uint8Array([1, 2, 3]))); const bytes = writer.getBytes(); const value = await cbor.decodeFirst(bytes); diff --git a/packages/ocapn/test/codecs/_codecs_util.js b/packages/ocapn/test/codecs/_codecs_util.js index f36ad18fc9..8121e9b09c 100644 --- a/packages/ocapn/test/codecs/_codecs_util.js +++ b/packages/ocapn/test/codecs/_codecs_util.js @@ -12,7 +12,7 @@ */ import harden from '@endo/harden'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { encodeHex } from '@endo/hex'; import { Far } from '@endo/marshal'; import { HandledPromise } from '@endo/eventual-send'; @@ -120,10 +120,10 @@ export const gifterLocation = harden({ hints: { host: '127.0.0.1', port: '54824' }, }); -export const exampleSigParamBytes = bytesToImmutable( +export const exampleSigParamBytes = frozenBytes( Uint8Array.from({ length: 32 }, (_, i) => i), ); -export const examplePubKeyQBytes = bytesToImmutable( +export const examplePubKeyQBytes = frozenBytes( Uint8Array.from({ length: 32 }, (_, i) => i * 2), ); @@ -147,19 +147,19 @@ export const receiverKeyForExporter = makeOcapnKeyPairFromPrivateKey( ); export const exampleExporterSessionId = /** @type {SessionId} */ ( - bytesToImmutable(Uint8Array.from({ length: 32 }, (_, i) => i * 7)) + frozenBytes(Uint8Array.from({ length: 32 }, (_, i) => i * 7)) ); export const exampleGifterSideId = /** @type {PublicKeyId} */ ( - bytesToImmutable(Uint8Array.from({ length: 32 }, (_, i) => i * 8)) + frozenBytes(Uint8Array.from({ length: 32 }, (_, i) => i * 8)) ); -export const exampleGiftId = bytesToImmutable( +export const exampleGiftId = frozenBytes( Uint8Array.from({ length: 32 }, (_, i) => i * 9), ); export const exampleReceiverSessionId = /** @type {SessionId} */ ( - bytesToImmutable(Uint8Array.from({ length: 32 }, (_, i) => i * 10)) + frozenBytes(Uint8Array.from({ length: 32 }, (_, i) => i * 10)) ); export const exampleReceiverSideId = /** @type {PublicKeyId} */ ( - bytesToImmutable(Uint8Array.from({ length: 32 }, (_, i) => i * 11)) + frozenBytes(Uint8Array.from({ length: 32 }, (_, i) => i * 11)) ); /** diff --git a/packages/ocapn/test/codecs/_syrup_util.js b/packages/ocapn/test/codecs/_syrup_util.js index 5e6fdcddaf..19370407e4 100644 --- a/packages/ocapn/test/codecs/_syrup_util.js +++ b/packages/ocapn/test/codecs/_syrup_util.js @@ -1,7 +1,7 @@ // @ts-check import { bytesFromText } from '@endo/bytes/from-string.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { concatImmutables } from '@endo/bytes/concat-immutables.js'; const textEncoder = new TextEncoder(); @@ -15,33 +15,29 @@ const textEncoder = new TextEncoder(); /** * @param {string} s - * @returns {ArrayBuffer} + * @returns {Uint8Array} */ const selectorSyrup = s => { const b = textEncoder.encode(s); - return bytesToImmutable( - bytesFromText(`${b.length}'${String.fromCharCode(...b)}`), - ); + return frozenBytes(bytesFromText(`${b.length}'${String.fromCharCode(...b)}`)); }; /** * @param {number} i - * @returns {ArrayBuffer} + * @returns {Uint8Array} */ export const intSyrup = i => - bytesToImmutable( - bytesFromText(`${Math.floor(Math.abs(i))}${i < 0 ? '-' : '+'}`), - ); + frozenBytes(bytesFromText(`${Math.floor(Math.abs(i))}${i < 0 ? '-' : '+'}`)); /** * @param {string} label - * @param {Array} items - * @returns {ArrayBuffer} + * @param {Array} items + * @returns {Uint8Array} */ export const recordSyrup = (label, ...items) => concatImmutables([ - bytesToImmutable(bytesFromText('<')), + frozenBytes(bytesFromText('<')), selectorSyrup(label), ...items, - bytesToImmutable(bytesFromText('>')), + frozenBytes(bytesFromText('>')), ]); diff --git a/packages/ocapn/test/codecs/descriptors.test.js b/packages/ocapn/test/codecs/descriptors.test.js index ed1791da70..f3390f6812 100644 --- a/packages/ocapn/test/codecs/descriptors.test.js +++ b/packages/ocapn/test/codecs/descriptors.test.js @@ -6,8 +6,7 @@ import test from '@endo/ses-ava/test.js'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { throws } from '../_util.js'; import { @@ -56,8 +55,8 @@ const table = [ type: 'desc:sig-envelope', object: { type: 'desc:handoff-receive', - receivingSession: bytesToImmutable(bytesFromText('123')), - receivingSide: bytesToImmutable(bytesFromText('456')), + receivingSession: frozenBytes(bytesFromText('123')), + receivingSide: frozenBytes(bytesFromText('456')), handoffCount: 1n, signedGive: { type: 'desc:sig-envelope', @@ -76,11 +75,11 @@ const table = [ designator: '1234', hints: { host: '127.0.0.1', port: '54822' }, }, - exporterSessionId: bytesToImmutable( + exporterSessionId: frozenBytes( bytesFromText('exporter-session-id'), ), - gifterSideId: bytesToImmutable(bytesFromText('gifter-side-id')), - giftId: bytesToImmutable(bytesFromText('gift-id')), + gifterSideId: frozenBytes(bytesFromText('gifter-side-id')), + giftId: frozenBytes(bytesFromText('gift-id')), }, signature: { type: 'sig-val', @@ -112,7 +111,7 @@ test('descriptor fails with negative integer [syrup]', t => { const testKit = makeCodecTestKit(); const codec = testKit.DescImportObjectCodec; const syrup = recordSyrup('desc:import-object', intSyrup(-1)); - const syrupBytes = bytesFromImmutable(syrup); + const syrupBytes = thawedBytes(syrup); const syrupReader = SyrupCodec.makeReader(syrupBytes, { name: 'import-object with negative integer', }); diff --git a/packages/ocapn/test/codecs/operations.test.js b/packages/ocapn/test/codecs/operations.test.js index 7be68ea070..a89a193ac1 100644 --- a/packages/ocapn/test/codecs/operations.test.js +++ b/packages/ocapn/test/codecs/operations.test.js @@ -7,7 +7,7 @@ import harden from '@endo/harden'; */ import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { decodeHex } from '@endo/hex'; @@ -22,9 +22,9 @@ import { /** * @param {string} hex - * @returns {ArrayBuffer} + * @returns {Uint8Array} */ -const hexToImmutableBuffer = hex => bytesToImmutable(decodeHex(hex)); +const hexToImmutableBuffer = hex => frozenBytes(decodeHex(hex)); /** @type {CodecTestEntry[]} */ export const table = [ @@ -155,20 +155,14 @@ export const table = [ makeValue: testKit => ({ type: 'op:deliver', to: testKit.referenceKit.provideRemoteObjectValue(0n), - args: [ - makeSelector('fetch'), - bytesToImmutable(bytesFromText('swiss-number')), - ], + args: [makeSelector('fetch'), frozenBytes(bytesFromText('swiss-number'))], answerPosition: 3n, resolveMeDesc: testKit.makeLocalObject(5n), }), makeExpectedValue: testKit => ({ type: 'op:deliver', to: testKit.makeLocalObject(0n), - args: [ - makeSelector('fetch'), - bytesToImmutable(bytesFromText('swiss-number')), - ], + args: [makeSelector('fetch'), frozenBytes(bytesFromText('swiss-number'))], answerPosition: 3n, resolveMeDesc: testKit.referenceKit.provideRemoteObjectValue(5n), }), @@ -325,7 +319,7 @@ export const table = [ makeSelector('foo'), 1n, false, - bytesToImmutable(Uint8Array.from([0x62, 0x61, 0x72])), + frozenBytes(Uint8Array.from([0x62, 0x61, 0x72])), ['baz'], ]), answerPosition: false, @@ -338,7 +332,7 @@ export const table = [ makeSelector('foo'), 1n, false, - bytesToImmutable(Uint8Array.from([0x62, 0x61, 0x72])), + frozenBytes(Uint8Array.from([0x62, 0x61, 0x72])), ['baz'], ], answerPosition: false, diff --git a/packages/ocapn/test/codecs/passable.test.js b/packages/ocapn/test/codecs/passable.test.js index a1f48eb04b..813bd267db 100644 --- a/packages/ocapn/test/codecs/passable.test.js +++ b/packages/ocapn/test/codecs/passable.test.js @@ -6,8 +6,7 @@ import test from '@endo/ses-ava/test.js'; import harden from '@endo/harden'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { thawedBytes, frozenBytes } from '@endo/immutable-arraybuffer'; import { makeTagged } from '@endo/pass-style'; import { makeSelector } from '../../src/selector.js'; @@ -40,11 +39,11 @@ const table = [ { name: 'string hello', value: 'hello' }, { name: 'byte array hello', - value: bytesToImmutable(new Uint8Array([0x68, 0x65, 0x6c, 0x6c, 0x6f])), + value: frozenBytes(new Uint8Array([0x68, 0x65, 0x6c, 0x6c, 0x6f])), }, { name: 'byte array', - value: bytesToImmutable(new Uint8Array([0x68, 0x65, 0x6c, 0x6c, 0x6f])), + value: frozenBytes(new Uint8Array([0x68, 0x65, 0x6c, 0x6c, 0x6f])), }, { name: 'selector', @@ -149,7 +148,7 @@ const table = [ throw Error('SturdyRef has no details'); } t.deepEqual(details.location, exporterLocation); - t.is(details.secret, '123'); + t.deepEqual([...details.secret], [0x31, 0x32, 0x33]); }, }, { @@ -163,7 +162,7 @@ const table = [ throw Error('SturdyRef has no details'); } t.deepEqual(details.location, exporterLocation); - t.is(details.secret, '123'); + t.deepEqual([...details.secret], [0x31, 0x32, 0x33]); }, }, // Tagged objects containing references @@ -208,7 +207,7 @@ const table = [ throw Error('SturdyRef has no details'); } t.deepEqual(details.location, exporterLocation); - t.is(details.secret, '456'); + t.deepEqual([...details.secret], [0x34, 0x35, 0x36]); }, }, ]; @@ -226,7 +225,7 @@ runTableTestsAllCodecs( test('error on unknown record type in passable [syrup]', t => { const codec = PassableCodec; const syrup = recordSyrup('unknown-record-type'); - const syrupBytes = bytesFromImmutable(syrup); + const syrupBytes = thawedBytes(syrup); const syrupReader = SyrupCodec.makeReader(syrupBytes, { name: 'unknown record type', }); diff --git a/packages/ocapn/test/cryptography.test.js b/packages/ocapn/test/cryptography.test.js index 44b2445439..eaf7595211 100644 --- a/packages/ocapn/test/cryptography.test.js +++ b/packages/ocapn/test/cryptography.test.js @@ -2,7 +2,7 @@ import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { makeCryptography, makeSessionId } from '../src/cryptography.js'; import { syrupCodec } from '../src/syrup/index.js'; @@ -56,7 +56,7 @@ test('makeWithdrawGiftDescriptor', t => { }, gifterExporterSessionId, gifterKey.publicKey.id, - bytesToImmutable(bytesFromText('gift-id')), + frozenBytes(bytesFromText('gift-id')), ); const handoffGiveSignature = signHandoffGive( handoffGiveDescriptor, diff --git a/packages/ocapn/test/snapshots/api-surface.test.js.md b/packages/ocapn/test/snapshots/api-surface.test.js.md index 5d07e871e2..32310a6ea3 100644 --- a/packages/ocapn/test/snapshots/api-surface.test.js.md +++ b/packages/ocapn/test/snapshots/api-surface.test.js.md @@ -1,4 +1,4 @@ -# Snapshot report for `test/api-surface.test.js` +# Snapshot report for `packages/ocapn/test/api-surface.test.js` The actual snapshot is saved in `api-surface.test.js.snap`. @@ -11,7 +11,7 @@ Generated by [AVA](https://avajs.dev). `PUBLIC API SURFACE␊ ==================␊ Entry points: Client, NetlayerHandlers, TcpTestOnlyNetLayer␊ - Reachable types: 13␊ + Reachable types: 14␊ ␊ SKIPPED MEMBERS (underscore-prefixed, not walked):␊ - Client._debug␊ @@ -64,8 +64,8 @@ Generated by [AVA](https://avajs.dev). ␊ OcapnSignature:␊ Properties:␊ - - r␊ - - s␊ + - r → [ArrayBufferView]␊ + - s → [ArrayBufferView]␊ - scheme␊ - type␊ ␊ @@ -81,13 +81,15 @@ Generated by [AVA](https://avajs.dev). - abort␊ - getBootstrap␊ ␊ + SessionId:␊ + ␊ SessionResumption:␊ Properties:␊ - peerLocation → [OcapnLocation]␊ - peerLocationSignature → [OcapnSignature]␊ - - peerPublicKeyBytes␊ + - peerPublicKeyBytes → [ArrayBufferView]␊ - selfPrivateKeyBytes␊ - - sessionId␊ + - sessionId → [SessionId]␊ ␊ SocketOperations:␊ Methods:␊ diff --git a/packages/ocapn/test/snapshots/api-surface.test.js.snap b/packages/ocapn/test/snapshots/api-surface.test.js.snap index 9a3035bb08..b1c79960d1 100644 Binary files a/packages/ocapn/test/snapshots/api-surface.test.js.snap and b/packages/ocapn/test/snapshots/api-surface.test.js.snap differ diff --git a/packages/ocapn/test/sturdyref.test.js b/packages/ocapn/test/sturdyref.test.js index 84e6094d86..583861c5f5 100644 --- a/packages/ocapn/test/sturdyref.test.js +++ b/packages/ocapn/test/sturdyref.test.js @@ -4,9 +4,43 @@ import { E } from '@endo/eventual-send'; import { Far } from '@endo/marshal'; import { passStyleOf } from '@endo/pass-style'; import { test, testWithErrorUnwrapping, makeTestClient } from './_util.js'; -import { isSturdyRef, getSturdyRefDetails } from '../src/client/sturdyrefs.js'; +import { + decodeSwissnum, + encodeSwissnum, + swissnumFromBytes, + swissnumToBytes, +} from '../src/client/util.js'; +import { + isSturdyRef, + getSturdyRefDetails, + makeSturdyRefTracker, +} from '../src/client/sturdyrefs.js'; import { ocapnPassStyleOf } from '../src/codecs/ocapn-pass-style.js'; +test('swissnum text conversion rejects U+0080 without changing raw bytes', t => { + t.throws(() => encodeSwissnum('\u0080'), { instanceOf: RangeError }); + + const swissNum = swissnumFromBytes(Uint8Array.of(0x80)); + t.throws(() => decodeSwissnum(swissNum), { instanceOf: RangeError }); + t.deepEqual([...swissnumToBytes(swissNum)], [0x80]); +}); + +test('SturdyRef lookup preserves non-ASCII swissnum bytes', async t => { + /** @type {unknown[]} */ + const lookedUpSecrets = []; + const tracker = makeSturdyRefTracker({ + get: secret => { + lookedUpSecrets.push(secret); + return 'found'; + }, + }); + + t.is(await tracker.lookup(swissnumFromBytes(Uint8Array.of(0x80))), 'found'); + const [lookedUpSecret] = lookedUpSecrets; + t.true(lookedUpSecret instanceof Uint8Array); + t.deepEqual([.../** @type {Uint8Array} */ (lookedUpSecret)], [0x80]); +}); + testWithErrorUnwrapping('SturdyRef is a tagged type', async t => { const { client: clientA, location: locationB } = await makeTestClient({ debugLabel: 'A', diff --git a/packages/ocapn/test/syrup/_table.js b/packages/ocapn/test/syrup/_table.js index df85f0debc..3d1c84688c 100644 --- a/packages/ocapn/test/syrup/_table.js +++ b/packages/ocapn/test/syrup/_table.js @@ -1,4 +1,4 @@ -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { SyrupSelectorFor } from '../../src/syrup/js-representation.js'; @@ -24,7 +24,7 @@ export const table = [ { syrup: 'f', value: false }, { syrup: '5"hello', value: 'hello' }, { syrup: "5'hello", value: SyrupSelectorFor('hello') }, - { syrup: '5:hello', value: bytesToImmutable(bytesFromText('hello')) }, + { syrup: '5:hello', value: frozenBytes(bytesFromText('hello')) }, { syrup: '[1+2+3+]', value: [1n, 2n, 3n] }, { syrup: '[3"abc3"def]', value: ['abc', 'def'] }, { syrup: '{1"a10+1"b20+}', value: { a: 10n, b: 20n } }, diff --git a/packages/ocapn/test/syrup/codec.test.js b/packages/ocapn/test/syrup/codec.test.js index 2e55bede90..cfcd067143 100644 --- a/packages/ocapn/test/syrup/codec.test.js +++ b/packages/ocapn/test/syrup/codec.test.js @@ -3,8 +3,7 @@ import test from '@endo/ses-ava/test.js'; import path from 'path'; import fs from 'fs'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; +import { frozenBytes, thawedBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { bytesToText } from '@endo/bytes/to-string.js'; import { makeSyrupReader } from '../../src/syrup/decode.js'; @@ -97,7 +96,7 @@ test('zoo.bin', t => { syrupReader.enterSet(); while (!syrupReader.peekSetEnd()) { result.eats.push( - bytesToText(bytesFromImmutable(syrupReader.readBytestring()), { + bytesToText(thawedBytes(syrupReader.readBytestring()), { fatal: true, }), ); @@ -110,10 +109,9 @@ test('zoo.bin', t => { t.is(syrupReader.readSelectorAsString(), 'weight'); result.weight = syrupReader.readFloat64(); t.is(syrupReader.readSelectorAsString(), 'species'); - result.species = bytesToText( - bytesFromImmutable(syrupReader.readBytestring()), - { fatal: true }, - ); + result.species = bytesToText(thawedBytes(syrupReader.readBytestring()), { + fatal: true, + }); syrupReader.exitDictionary(); return result; }, @@ -124,7 +122,7 @@ test('zoo.bin', t => { syrupWriter.writeSelectorFromString('eats'); syrupWriter.enterSet(value.eats.length); for (const eat of value.eats) { - syrupWriter.writeBytestring(bytesToImmutable(bytesFromText(eat))); + syrupWriter.writeBytestring(frozenBytes(bytesFromText(eat))); } syrupWriter.exitSet(); syrupWriter.writeSelectorFromString('name'); @@ -134,9 +132,7 @@ test('zoo.bin', t => { syrupWriter.writeSelectorFromString('weight'); syrupWriter.writeFloat64(value.weight); syrupWriter.writeSelectorFromString('species'); - syrupWriter.writeBytestring( - bytesToImmutable(bytesFromText(value.species)), - ); + syrupWriter.writeBytestring(frozenBytes(bytesFromText(value.species))); syrupWriter.exitDictionary(); }, }; diff --git a/packages/ocapn/test/syrup/compare.test.js b/packages/ocapn/test/syrup/compare.test.js index d6ad8a484a..2a0087facf 100644 --- a/packages/ocapn/test/syrup/compare.test.js +++ b/packages/ocapn/test/syrup/compare.test.js @@ -1,7 +1,7 @@ // @ts-check import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { bytesFromText } from '@endo/bytes/from-string.js'; import { compareImmutableArrayBuffers, @@ -27,29 +27,29 @@ test('right longer', t => { }); test('compareImmutableArrayBuffers - equal buffers', t => { - const buffer1 = bytesToImmutable(bytesFromText('hello')); - const buffer2 = bytesToImmutable(bytesFromText('hello')); + const buffer1 = frozenBytes(bytesFromText('hello')); + const buffer2 = frozenBytes(bytesFromText('hello')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), 0); }); test('compareImmutableArrayBuffers - left less than right', t => { - const buffer1 = bytesToImmutable(bytesFromText('abc')); - const buffer2 = bytesToImmutable(bytesFromText('xyz')); + const buffer1 = frozenBytes(bytesFromText('abc')); + const buffer2 = frozenBytes(bytesFromText('xyz')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), -1); }); test('compareImmutableArrayBuffers - left greater than right', t => { - const buffer1 = bytesToImmutable(bytesFromText('xyz')); - const buffer2 = bytesToImmutable(bytesFromText('abc')); + const buffer1 = frozenBytes(bytesFromText('xyz')); + const buffer2 = frozenBytes(bytesFromText('abc')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), 1); }); test('compareImmutableArrayBuffers - left is prefix of right', t => { - const buffer1 = bytesToImmutable(bytesFromText('hello')); - const buffer2 = bytesToImmutable(bytesFromText('helloworld')); + const buffer1 = frozenBytes(bytesFromText('hello')); + const buffer2 = frozenBytes(bytesFromText('helloworld')); const result = compareImmutableArrayBuffers(buffer1, buffer2); t.true(result < 0, 'left should be less than right'); @@ -57,31 +57,31 @@ test('compareImmutableArrayBuffers - left is prefix of right', t => { }); test('compareImmutableArrayBuffers - right is prefix of left', t => { - const buffer1 = bytesToImmutable(bytesFromText('helloworld')); - const buffer2 = bytesToImmutable(bytesFromText('hello')); + const buffer1 = frozenBytes(bytesFromText('helloworld')); + const buffer2 = frozenBytes(bytesFromText('hello')); const result = compareImmutableArrayBuffers(buffer1, buffer2); t.true(result > 0, 'left should be greater than right'); }); test('compareImmutableArrayBuffers - empty buffers', t => { - const buffer1 = bytesToImmutable(bytesFromText('')); - const buffer2 = bytesToImmutable(bytesFromText('')); + const buffer1 = frozenBytes(bytesFromText('')); + const buffer2 = frozenBytes(bytesFromText('')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), 0); }); test('compareImmutableArrayBuffers - empty vs non-empty', t => { - const buffer1 = bytesToImmutable(bytesFromText('')); - const buffer2 = bytesToImmutable(bytesFromText('a')); + const buffer1 = frozenBytes(bytesFromText('')); + const buffer2 = frozenBytes(bytesFromText('a')); t.is(compareImmutableArrayBuffers(buffer1, buffer2), -1); t.is(compareImmutableArrayBuffers(buffer2, buffer1), 1); }); test('compareImmutableArrayBuffers - binary data', t => { - const buffer1 = bytesToImmutable(new Uint8Array([0x00, 0x01, 0x02])); - const buffer2 = bytesToImmutable(new Uint8Array([0x00, 0x01, 0x03])); + const buffer1 = frozenBytes(new Uint8Array([0x00, 0x01, 0x02])); + const buffer2 = frozenBytes(new Uint8Array([0x00, 0x01, 0x03])); t.is(compareImmutableArrayBuffers(buffer1, buffer2), -1); t.is(compareImmutableArrayBuffers(buffer2, buffer1), 1); @@ -89,8 +89,8 @@ test('compareImmutableArrayBuffers - binary data', t => { test('compareImmutableArrayBuffers - bytewise comparison', t => { // Test that comparison is bytewise, not lexicographic - const buffer1 = bytesToImmutable(new Uint8Array([0xff])); - const buffer2 = bytesToImmutable(new Uint8Array([0x00, 0x00])); + const buffer1 = frozenBytes(new Uint8Array([0xff])); + const buffer2 = frozenBytes(new Uint8Array([0x00, 0x00])); // 0xff > 0x00, so buffer1 > buffer2 despite being shorter t.is(compareImmutableArrayBuffers(buffer1, buffer2), 1); diff --git a/packages/ocapn/test/syrup/reader.test.js b/packages/ocapn/test/syrup/reader.test.js index 2977442209..d7b99ddece 100644 --- a/packages/ocapn/test/syrup/reader.test.js +++ b/packages/ocapn/test/syrup/reader.test.js @@ -2,7 +2,7 @@ import test from '@endo/ses-ava/test.js'; import * as fs from 'fs'; import path from 'path'; -import { bytesFromImmutable } from '@endo/bytes/from-immutable.js'; +import { thawedBytes } from '@endo/immutable-arraybuffer'; import { bytesToText } from '@endo/bytes/to-string.js'; import { makeSyrupReader } from '../../src/syrup/decode.js'; @@ -15,10 +15,10 @@ const zooBin = Uint8Array.from(zooBinRaw); /** * - * @param {ArrayBufferLike} bytes + * @param {Uint8Array} bytes * @returns {string} */ -const toUtf8 = bytes => bytesToText(bytesFromImmutable(bytes), { fatal: true }); +const toUtf8 = bytes => bytesToText(thawedBytes(bytes), { fatal: true }); test('exciting a dictionary without entering it', t => { const syrup = '}'; diff --git a/packages/ocapn/tsconfig.composite.json b/packages/ocapn/tsconfig.composite.json index 9d1cd38ed9..8c031097d5 100644 --- a/packages/ocapn/tsconfig.composite.json +++ b/packages/ocapn/tsconfig.composite.json @@ -5,6 +5,9 @@ "composite": true }, "references": [ + { + "path": "../ascii/tsconfig.composite.json" + }, { "path": "../bytes/tsconfig.composite.json" }, @@ -20,6 +23,9 @@ { "path": "../hex/tsconfig.composite.json" }, + { + "path": "../immutable-arraybuffer/tsconfig.composite.json" + }, { "path": "../init/tsconfig.composite.json" }, diff --git a/packages/pass-style/package.json b/packages/pass-style/package.json index 9b1199717f..23f8ec0e47 100644 --- a/packages/pass-style/package.json +++ b/packages/pass-style/package.json @@ -43,6 +43,7 @@ "@endo/promise-kit": "workspace:^" }, "devDependencies": { + "@endo/immutable-arraybuffer": "workspace:^", "@endo/init": "workspace:^", "@endo/ses-ava": "workspace:^", "@fast-check/ava": "catalog:dev", diff --git a/packages/pass-style/src/byteArray.js b/packages/pass-style/src/byteArray.js index 9930e1971f..575312b106 100644 --- a/packages/pass-style/src/byteArray.js +++ b/packages/pass-style/src/byteArray.js @@ -7,6 +7,7 @@ import { X, Fail } from '@endo/errors'; const { getPrototypeOf, getOwnPropertyDescriptor } = Object; const { ownKeys, apply } = Reflect; +const { isView } = ArrayBuffer; // Detects the presence of immutable ArrayBuffer support on the underlying // platform and provides either suitable values from that implementation or @@ -16,11 +17,20 @@ const { ownKeys, apply } = Reflect; // threshold we consider the proposal stabilised: the question is whether // the current JS implementation does or does not implement it, not // whether the implementation is partial or divergent. The brand check -// that distinguishes emulated immutable buffers from genuine ArrayBuffers -// is the `immutable` accessor on `ArrayBuffer.prototype`, installed by +// that distinguishes an immutable backing buffer from an ordinary mutable +// one is the `immutable` accessor on `ArrayBuffer.prototype`, installed by // the shim (or natively present on platforms that have shipped the -// proposal). The prototype identity check remains as a structural guard -// against a tampered prototype chain. +// proposal). Note this axis is immutable-vs-mutable, not emulated-vs-genuine: +// the accessor reports `true` for BOTH a genuine native immutable buffer +// (e.g. on XS, or a post-Stage-3 Node) AND the `@endo/immutable-arraybuffer` +// emulated wrapper, and `false`/absent for a mutable buffer (which is always +// genuine). It is the `byteArray` brand and the must-I-copy trigger, not an +// emulated-vs-genuine discriminator; where that finer distinction is needed +// it is drawn separately by `ArrayBuffer.isView` — the single committed +// emulated-vs-genuine fidelity loss: a genuine view (native integer-indexed +// exotic, or an ordinary mutable view) is `isView`, whereas the emulated +// wrapper is a plain object and is not. The prototype identity check remains +// as a structural guard against a tampered prototype chain. const { prototype: arrayBufferPrototype } = ArrayBuffer; const immutableDescriptor = getOwnPropertyDescriptor( arrayBufferPrototype, @@ -31,21 +41,275 @@ const immutableGetter = immutableDescriptor?.get ) || (() => false); -// The permitted own-property keys on an emulated immutable buffer, paired -// with the `typeof` of the data-property value the key must carry. The +// Capture the `%TypedArrayPrototype%.at` method so we can read a byte +// through the genuine integer-indexed protocol without going through the +// wrapper's own (possibly shadowing) data property. On the emulated +// freezable-TypedArray path installed by `@endo/immutable-arraybuffer`, +// this captured reference points at the shim-installed amplifier, which +// resolves the wrapper to its hidden genuine TypedArray and reads from +// the underlying immutable buffer. On the native path (post-Stage-3), it +// points at the genuine `%TypedArrayPrototype%.at`, which reads via the +// integer-indexed exotic. Either way, the read bypasses any own data +// property on the wrapper. +const { prototype: uint8ArrayPrototype } = Uint8Array; +const typedArrayPrototype = getPrototypeOf(uint8ArrayPrototype); +const { at: typedArrayAt } = typedArrayPrototype; + +// The permitted own-property keys on the immutable `ArrayBuffer` backing +// a `byteArray` Uint8Array wrapper, paired with the `typeof` of the +// data-property value the key must carry. The // `@endo/immutable-arraybuffer` package installs `[Symbol.toStringTag] = // 'ImmutableArrayBuffer'` as an own property on each emulated immutable // (not on the shared prototype) so `concordance` and similar // `Object.prototype.toString.call`-sniffing consumers route the value // through their unrenderable-value path rather than into `Buffer.from` // (which throws on emulated immutables because they are not exotic -// objects). The byteArray brand check tolerates exactly the keys named -// here; for each, it verifies that the key carries a non-enumerable data -// property whose value's `typeof` matches the entry. Anything else still -// fails. +// objects). The backing-buffer sub-check tolerates exactly the keys +// named here; for each, it verifies that the key carries a +// non-enumerable data property whose value's `typeof` matches the +// entry. Anything else still fails. /** @type {Map} */ const allowedOwnDataProperties = new Map([[Symbol.toStringTag, 'string']]); +// Tests whether an arbitrary key is the canonical string form of a +// non-negative integer that fits into a JavaScript Number's safe-integer +// range, the keys spec-mandated as own properties on a TypedArray's +// integer-indexed exotic surface. Spec callout: a "valid integer index" +// for TypedArrays uses `CanonicalNumericIndexString` and is constrained +// to non-negative integers strictly less than the array's `length`. +// `String(Number(key)) === key` rules out leading zeros, decimal points, +// scientific notation, and the trailing `'-0'` form; the explicit +// `Number.isInteger` and non-negativity checks rule out NaN, Infinity, +// and negative integers. The caller still verifies the index is below +// `length` after this returns true. +/** + * @param {string | symbol} key + * @returns {boolean} + */ +const isCanonicalIndexKey = key => { + if (typeof key !== 'string') { + return false; + } + const n = Number(key); + return Number.isInteger(n) && n >= 0 && String(n) === key && n <= 2 ** 53 - 1; +}; + +/** + * Sub-check on the backing immutable `ArrayBuffer` of a `Uint8Array` + * wrapper. Validates that the buffer is a plain frozen immutable + * `ArrayBuffer` with the expected prototype, the `immutable` accessor + * returning true, and at most the canonical `[Symbol.toStringTag]` + * own slot installed by `@endo/immutable-arraybuffer`. Raw immutable + * `ArrayBuffer` values themselves are no longer accepted as the + * `byteArray` pass style; this helper is only reached as a sub-check + * on a `Uint8Array` wrapper's `.buffer`. + * + * @param {ArrayBuffer} candidate + */ +const assertRestValidImmutableArrayBuffer = candidate => { + getPrototypeOf(candidate) === arrayBufferPrototype || + assert.fail(X`Malformed ByteArray ${candidate}`, TypeError); + apply(immutableGetter, candidate, []) || + Fail`Must be an immutable ArrayBuffer: ${candidate}`; + for (const key of ownKeys(candidate)) { + const expectedValueType = allowedOwnDataProperties.get(key); + expectedValueType !== undefined || + assert.fail( + X`ByteArrays must not have own properties: ${candidate}`, + TypeError, + ); + const descriptor = getOwnPropertyDescriptor(candidate, key); + // The descriptor cannot be undefined: `key` was just enumerated by + // `ownKeys(candidate)`. The conjunction below asserts the shape + // the brand contract requires: a non-enumerable data property whose + // value has the expected `typeof`. The dynamic `typeof` comparison + // against the allowlist's recorded type-name is intentional here; + // the standard `valid-typeof` lint rule expects a literal RHS. + const valueType = + descriptor && 'value' in descriptor ? typeof descriptor.value : undefined; + (descriptor !== undefined && + descriptor.enumerable === false && + valueType === expectedValueType) || + assert.fail( + X`ByteArray own-property ${key} must be a non-enumerable data property of typeof ${expectedValueType}: ${candidate}`, + TypeError, + ); + } +}; + +/** + * Validates that the candidate is a "plain frozen `Uint8Array` backed by a + * plain frozen immutable `ArrayBuffer`". The unifying definition accepts + * exactly two well-formed shapes, discriminated by `ArrayBuffer.isView` (the + * single committed emulated-vs-genuine fidelity loss) and then constrained on + * their own-index count: + * + * - **Emulated-wrapper shape** (produced by `@endo/immutable-arraybuffer`): + * a plain ordinary object whose `[[Prototype]]` is `Uint8Array.prototype`, + * for which `ArrayBuffer.isView` is `false` and which has **no own + * integer-indexed properties**, regardless of `length`. The shim exposes + * data through the prototype-chain amplifier; there are never any own + * indexed slots on the wrapper itself. + * + * - **Native shape** (produced by a TC39-spec-following engine once the + * Immutable ArrayBuffer proposal ships): a genuine integer-indexed exotic + * for which `ArrayBuffer.isView` is `true`, with exactly `length`-many own + * enumerable indexed data properties, each matching the underlying buffer + * byte at that offset. + * + * `ArrayBuffer.isView` selects which shape a candidate must match: a non-view + * must carry zero own indexed properties, a view exactly `length`-many. Any + * other count for that shape — an emulated wrapper tampered to carry indexed + * own writes, or a native exotic stripped of its indexed slots — is + * post-construction tampering and is rejected. Non-index own properties are + * rejected in both shapes. Indexed own properties present on a native-shape + * view but whose value disagrees with the underlying buffer byte are also + * rejected. + * + * The view must additionally span its whole backing buffer one-to-one + * (`byteOffset === 0 && length === buffer.byteLength`): a sub-view is + * rejected (the restrictive choice, issue #573), so the marshalled value + * never conveys a hidden tail of the buffer beyond what the view reveals. + * + * Assumes the candidate has already passed the `isFrozen` gate that + * `passStyleOf` applies before reaching any helper. + * + * @param {Uint8Array} candidate + */ +const assertRestValidPlainFrozenUint8Array = candidate => { + getPrototypeOf(candidate) === uint8ArrayPrototype || + assert.fail(X`Malformed ByteArray ${candidate}`, TypeError); + // `candidate.buffer` is typed as `ArrayBufferLike` (a union that + // includes `SharedArrayBuffer`); narrow to `ArrayBuffer` for the + // sub-check. The `confirmCanBeByteArray` guard already established + // that the buffer is an `ArrayBuffer` whose `immutable` accessor + // returned true, so the narrowing is safe at runtime. + const buffer = /** @type {ArrayBuffer} */ (candidate.buffer); + // The buffer must itself be a plain frozen immutable `ArrayBuffer`. + apply(immutableGetter, buffer, []) || + Fail`Uint8Array byteArray must be backed by an immutable ArrayBuffer: ${candidate}`; + assertRestValidImmutableArrayBuffer(buffer); + // Whole-buffer span (restrictive, issue #573): the view must cover its + // entire backing buffer one-to-one. A sub-view (`byteOffset > 0`, or a + // `length` shorter than `buffer.byteLength`) is rejected, because its + // backing buffer can carry more data than the view intends to reveal: a + // data-reachability hazard, since the marshalled value would convey the + // hidden tail of the buffer to any holder who reconstructs the view. + // `frozenBytes` always slices its window into a fresh immutable buffer, + // so every value it produces is a whole-buffer-spanning view and is + // unaffected; a hand-constructed sub-view must be re-sliced into its own + // immutable buffer (`frozenBytes(subview)`) to become a byteArray. This + // is the restrictive choice recorded in the design (Design Decisions §3), + // tracked for possible relaxation to the permissive sub-view form at + // https://github.com/endojs/endo-but-for-bots/issues/573 . + candidate.byteOffset === 0 || + assert.fail( + X`Plain frozen Uint8Array byteArray must span its whole backing buffer (expected byteOffset 0, got ${candidate.byteOffset}): ${candidate}`, + TypeError, + ); + candidate.length === buffer.byteLength || + assert.fail( + X`Plain frozen Uint8Array byteArray must span its whole backing buffer (length ${candidate.length} must equal buffer byteLength ${buffer.byteLength}): ${candidate}`, + TypeError, + ); + // `length` is the count of bytes the wrapper exposes. It is read via + // the prototype's `length` accessor, which on the emulated path + // delegates to the hidden genuine TypedArray and on the native path + // reads the integer-indexed exotic's `[[ArrayLength]]` slot. + const length = candidate.length; + // Collect and validate own keys in one pass, counting indexed keys as we go. + // `ArrayBuffer.isView` (below) selects which shape this candidate must + // match: on the emulated path (`!isView`) the count must be 0 (no own + // indexed properties at all, regardless of length); on the native path + // (`isView`) the count must be exactly `length`. Any other count for the + // selected shape indicates tampering and is rejected after the loop. + let ownIndexCount = 0; + for (const key of ownKeys(candidate)) { + if (!isCanonicalIndexKey(key)) { + assert.fail( + X`Plain frozen Uint8Array byteArray must not have own non-index properties: ${candidate}`, + TypeError, + ); + } + const index = Number(/** @type {string} */ (key)); + index < length || + assert.fail( + X`Plain frozen Uint8Array byteArray own index ${key} must be below length ${length}: ${candidate}`, + TypeError, + ); + const descriptor = getOwnPropertyDescriptor(candidate, key); + // The descriptor cannot be undefined: `key` was just enumerated by + // `ownKeys(candidate)`. Integer-indexed own properties on a frozen + // `Uint8Array` are enumerable data properties per spec (after freeze: + // non-writable, non-configurable). On the native exotic path the + // shape is forced by the integer-indexed-exotic internal methods. + const value = + descriptor && 'value' in descriptor ? descriptor.value : undefined; + (descriptor !== undefined && + 'value' in descriptor && + descriptor.enumerable === true && + typeof value === 'number') || + assert.fail( + X`Plain frozen Uint8Array byteArray own index ${key} must be an enumerable number-valued data property: ${candidate}`, + TypeError, + ); + // The own data property's value must match the byte the wrapper + // reads through the integer-indexed protocol. On the native exotic + // path the equality is structural (the own property *is* the + // integer-indexed read). The captured `typedArrayAt` bypasses any + // own data property on the wrapper and reads through the prototype + // chain, ensuring the comparison is against the underlying buffer byte. + const byteFromBuffer = apply(typedArrayAt, candidate, [index]); + value === byteFromBuffer || + assert.fail( + X`Plain frozen Uint8Array byteArray own index ${key} value ${value} must equal underlying byte ${byteFromBuffer}: ${candidate}`, + TypeError, + ); + ownIndexCount += 1; + } + // Accept only the two well-formed shapes, using `ArrayBuffer.isView` as the + // committed emulated-vs-genuine discriminator to select which one applies: + // - Emulated path (`!isView`): 0 own indexed properties (plain object, + // any length). + // - Native path (`isView`): exactly `length`-many own indexed properties. + // Gating on `isView` is strictly more precise than accepting either count + // unconditionally: it rejects an emulated wrapper tampered to carry `length` + // own indexed writes (a single shadowing write, or a full set, before + // freeze) and a native exotic stripped to zero — cases the bare + // `0 || length` test would have admitted — even when each written value + // matches the underlying buffer byte. + (isView(candidate) ? ownIndexCount === length : ownIndexCount === 0) || + assert.fail( + X`Plain frozen Uint8Array byteArray own indexed-property count ${ownIndexCount} does not match its shape (a genuine view, ArrayBuffer.isView true, needs exactly length ${length}; an emulated wrapper, isView false, needs 0): ${candidate}`, + TypeError, + ); +}; + +/** + * Discriminates the single accepted shape for the `byteArray` pass style: + * a plain frozen `Uint8Array` whose backing buffer is a plain frozen + * immutable `ArrayBuffer`. Raw immutable `ArrayBuffer` values, previously + * accepted, are no longer recognised as `byteArray`; producers must wrap + * them in `new Uint8Array(iab)` and harden the result. + * + * The check is fast and conservative: it confirms the shape's + * top-level brand without recursing into the buffer. `assertRestValid` + * performs the deeper validation. + * + * @param {unknown} candidate + * @returns {boolean} + */ +const confirmCanBeByteArray = candidate => { + if (candidate instanceof Uint8Array) { + const { buffer } = candidate; + return ( + buffer instanceof ArrayBuffer && + /** @type {boolean} */ (apply(immutableGetter, buffer, [])) + ); + } + return false; +}; + /** * @type {PassStyleHelper} */ @@ -53,39 +317,11 @@ export const ByteArrayHelper = harden({ styleName: 'byteArray', confirmCanBeValid: (candidate, reject) => - (candidate instanceof ArrayBuffer && candidate.immutable) || - (reject && reject`Immutable ArrayBuffer expected: ${candidate}`), + confirmCanBeByteArray(candidate) || + (reject && + reject`Uint8Array on immutable ArrayBuffer expected: ${candidate}`), assertRestValid: (candidate, _passStyleOfRecur) => { - getPrototypeOf(candidate) === arrayBufferPrototype || - assert.fail(X`Malformed ByteArray ${candidate}`, TypeError); - apply(immutableGetter, candidate, []) || - Fail`Must be an immutable ArrayBuffer: ${candidate}`; - for (const key of ownKeys(candidate)) { - const expectedValueType = allowedOwnDataProperties.get(key); - expectedValueType !== undefined || - assert.fail( - X`ByteArrays must not have own properties: ${candidate}`, - TypeError, - ); - const descriptor = getOwnPropertyDescriptor(candidate, key); - // The descriptor cannot be undefined: `key` was just enumerated by - // `ownKeys(candidate)`. The conjunction below asserts the shape - // the brand contract requires: a non-enumerable data property whose - // value has the expected `typeof`. The dynamic `typeof` comparison - // against the allowlist's recorded type-name is intentional here; - // the standard `valid-typeof` lint rule expects a literal RHS. - const valueType = - descriptor && 'value' in descriptor - ? typeof descriptor.value - : undefined; - (descriptor !== undefined && - descriptor.enumerable === false && - valueType === expectedValueType) || - assert.fail( - X`ByteArray own-property ${key} must be a non-enumerable data property of typeof ${expectedValueType}: ${candidate}`, - TypeError, - ); - } + assertRestValidPlainFrozenUint8Array(candidate); }, }); diff --git a/packages/pass-style/src/passStyle-helpers.js b/packages/pass-style/src/passStyle-helpers.js index 4355838c6e..1325cbe8ba 100644 --- a/packages/pass-style/src/passStyle-helpers.js +++ b/packages/pass-style/src/passStyle-helpers.js @@ -59,6 +59,16 @@ hideAndHardenFunction(isObject); /** * Duplicates packages/ses/src/make-hardener.js to avoid a dependency. * + * Deliberately a genuine TypedArray brand check via the `%TypedArray%` + * `[Symbol.toStringTag]` getter, NOT `ArrayBuffer.isView`. Both are + * unspoofable internal-slot checks, but `isView` is also true for a + * `DataView`. `passStyleOf` uses this only to sharpen a diagnostic — the + * "Cannot pass mutable typed arrays" message — for genuine TypedArrays; the + * DataView-inclusive `isView` would mislabel a `DataView` (never a + * `byteArray`) as a mutable typed array. (`byteArray.js` commits to `isView` + * for a different question — emulated-vs-native shape on an already-known + * `Uint8Array` — where DataViews are already excluded.) + * * @param {unknown} object */ export const isTypedArray = object => { diff --git a/packages/pass-style/src/passStyleOf.js b/packages/pass-style/src/passStyleOf.js index 1167df02b3..d92608b89a 100644 --- a/packages/pass-style/src/passStyleOf.js +++ b/packages/pass-style/src/passStyleOf.js @@ -192,6 +192,29 @@ const makePassStyleOf = passStyleHelpers => { return helper.styleName; } } + // A TypedArray that was not claimed by any helper (most commonly a + // Uint8Array backed by a mutable ArrayBuffer) must not fall through to + // the remotable path with a confusing "non-methods" error. This + // normally only fires under unsafe harden taming, where `isFrozen` may + // return true for unfrozen objects so the early `isFrozen` gate above + // is bypassed; it also fires on a native Immutable-ArrayBuffer engine + // for a genuine frozen TypedArray over an immutable buffer. + // + // The message must name the actual unmet requirement. A `Uint8Array` + // only reaches here backed by a *mutable* ArrayBuffer — an + // immutable-backed one is always claimed by the byteArray helper — so + // "mutable" is accurate for it. But a non-`Uint8Array` typed array + // fails for its element type, whether its backing buffer is mutable or + // immutable; blaming mutability there is misleading (a frozen + // non-`Uint8Array` typed array over an immutable buffer is not mutable + // at all — erights review, endojs/endo-but-for-bots#475). Name the + // element-type requirement in that case instead. + isTypedArray(inner) && + assert.fail( + inner instanceof Uint8Array + ? X`Cannot pass mutable typed arrays like ${inner}.` + : X`Cannot pass typed arrays other than Uint8Array like ${inner}.`, + ); assertValid(remotableHelper, inner, passStyleOfRecur); return 'remotable'; } diff --git a/packages/pass-style/src/types.d.ts b/packages/pass-style/src/types.d.ts index 329862ebd4..4ec0fe9f7c 100644 --- a/packages/pass-style/src/types.d.ts +++ b/packages/pass-style/src/types.d.ts @@ -124,6 +124,10 @@ export type PassStyleOf = { (p: Error): 'error'; (p: CopyTagged): 'tagged'; (p: readonly any[]): 'copyArray'; + // A `Uint8Array` is also `Iterable`; place its byteArray + // overload before the Iterable-as-remotable fallback so the more + // specific shape wins TypeScript overload resolution. + (p: Uint8Array): 'byteArray'; (p: Iterable): 'remotable'; (p: Iterator): 'remotable'; >(p: T): ExtractStyle; @@ -194,9 +198,10 @@ export type PassableCap = export type CopyArray = readonly T[]; /** - * A hardened immutable ArrayBuffer. + * A hardened frozen `Uint8Array` whose backing buffer is a hardened + * immutable `ArrayBuffer`. */ -export type ByteArray = ArrayBuffer; +export type ByteArray = Uint8Array; /** * A Passable dictionary in which each key is a string and each value is Passable. diff --git a/packages/pass-style/test/byte-array.test.js b/packages/pass-style/test/byte-array.test.js index 920e782121..4c35960bfd 100644 --- a/packages/pass-style/test/byte-array.test.js +++ b/packages/pass-style/test/byte-array.test.js @@ -3,19 +3,21 @@ import test from '@endo/ses-ava/test.js'; import harden from '@endo/harden'; import { passStyleOf } from '../src/passStyleOf.js'; -test('passStyleOf recognizes immutable ArrayBuffer as byteArray', t => { +test('passStyleOf recognizes frozen Uint8Array on immutable ArrayBuffer', t => { const buf = new ArrayBuffer(4); const view = new Uint8Array(buf); view[0] = 0xde; view[1] = 0xad; view[2] = 0xbe; view[3] = 0xef; - const immutable = harden(buf.sliceToImmutable()); - t.is(/** @type {string} */ (passStyleOf(immutable)), 'byteArray'); + const immutable = buf.sliceToImmutable(); + const bytes = harden(new Uint8Array(immutable)); + t.is(/** @type {string} */ (passStyleOf(bytes)), 'byteArray'); }); test('passStyleOf byteArray with empty buffer', t => { const buf = new ArrayBuffer(0); - const immutable = harden(buf.sliceToImmutable()); - t.is(/** @type {string} */ (passStyleOf(immutable)), 'byteArray'); + const immutable = buf.sliceToImmutable(); + const bytes = harden(new Uint8Array(immutable)); + t.is(/** @type {string} */ (passStyleOf(bytes)), 'byteArray'); }); diff --git a/packages/pass-style/test/byteArray.test.js b/packages/pass-style/test/byteArray.test.js index 87aead11bd..b864242c91 100644 --- a/packages/pass-style/test/byteArray.test.js +++ b/packages/pass-style/test/byteArray.test.js @@ -1,33 +1,60 @@ // @ts-nocheck -// Coverage for the byteArray brand check's `allowedOwnDataProperties` -// contract. The check tolerates the `[Symbol.toStringTag]` own property -// that `@endo/immutable-arraybuffer` installs on each emulated immutable -// buffer, but only when that own property is a non-enumerable data -// property whose value is a string. Anything else still fails. +// Coverage for the byteArray brand check after narrowing to the +// plain-frozen-`Uint8Array` shape only. // -// The non-canonical-shape cases below augment a real emulated immutable -// with extra own properties before hardening so the `passStyleOf` entry -// point reaches the byteArray brand check. The canonical-shape case -// asserts that the emulated immutable installed by the lib passes the -// check unchanged. +// The brand check now accepts exactly one shape: a plain frozen +// `Uint8Array` whose backing buffer is a plain frozen immutable +// `ArrayBuffer`. Raw immutable `ArrayBuffer` values, previously +// accepted, are no longer recognised as `byteArray`. +// +// The "plain" Uint8Array definition accepts exactly two well-formed +// wrapper shapes: +// - Emulated path: no own indexed properties at all, regardless of +// length. The `@endo/immutable-arraybuffer` shim produces this shape; +// any own indexed property is post-construction tampering and is +// rejected even when the value agrees with the underlying buffer byte. +// - Native path: exactly `length`-many own indexed properties, each an +// enumerable data property whose value matches the underlying buffer +// byte. This is the shape a spec-conformant engine will produce once +// the Immutable ArrayBuffer proposal ships natively. +// Non-index own properties are rejected on both paths. +// +// The first section asserts that raw immutable `ArrayBuffer` values are +// now rejected. The second section covers the surviving +// `Uint8Array`-on-IAB acceptance shape and its tampering rejections. import test from '@endo/ses-ava/test.js'; import harden from '@endo/harden'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { passStyleOf } from '../src/passStyleOf.js'; -const { defineProperty, getOwnPropertyDescriptor } = Object; +const { defineProperty, freeze, getOwnPropertyDescriptor } = Object; +const { ownKeys } = Reflect; -test('byteArray accepts an emulated immutable ArrayBuffer with the standard toStringTag slot', t => { +// --------------------------------------------------------------------------- +// Raw immutable ArrayBuffer is no longer a byteArray. +// --------------------------------------------------------------------------- + +test('byteArray rejects a raw immutable ArrayBuffer (previously accepted)', t => { + // Before the narrowing, an immutable `ArrayBuffer` produced by + // `sliceToImmutable` was recognised as a `byteArray`. The accepted shape + // is now `Uint8Array`-only; raw IAB values must be wrapped in + // `new Uint8Array(iab)` and hardened by producers. After the narrowing, + // a raw IAB hits no brand-matching helper and falls through to the + // remotable check, which rejects it because an `ArrayBuffer` is not a + // remotable. const iab = harden(new ArrayBuffer(0).sliceToImmutable()); - t.is(passStyleOf(iab), 'byteArray'); + t.throws(() => passStyleOf(iab)); }); -test('byteArray accepts an emulated immutable with the well-formed standard slot shape', t => { - // Sanity-check the contract: the lib installs `[Symbol.toStringTag]` as a +test('byteArray rejects a raw immutable ArrayBuffer with the standard toStringTag slot', t => { + // Sanity check: the lib still installs `[Symbol.toStringTag]` as a // non-enumerable, non-writable, non-configurable data property whose - // value is the string `'ImmutableArrayBuffer'`. The byteArray check - // permits exactly this shape; if the lib ever ships a different shape - // this test catches the drift. + // value is the string `'ImmutableArrayBuffer'`. That slot used to be + // tolerated by the `byteArray` acceptance arm for raw IAB; the entire + // arm is gone now, so the slot is moot for raw IAB acceptance. The + // backing-buffer sub-check still tolerates exactly this slot when the + // IAB is reached via a `Uint8Array` wrapper. const iab = new ArrayBuffer(0).sliceToImmutable(); const descriptor = getOwnPropertyDescriptor(iab, Symbol.toStringTag); t.deepEqual(descriptor, { @@ -36,62 +63,302 @@ test('byteArray accepts an emulated immutable with the well-formed standard slot enumerable: false, configurable: false, }); - t.is(passStyleOf(harden(iab)), 'byteArray'); + t.throws(() => passStyleOf(harden(iab))); }); -test('byteArray rejects an emulated immutable carrying an extra own data property', t => { +// --------------------------------------------------------------------------- +// Plain frozen Uint8Array backed by plain frozen immutable ArrayBuffer. +// --------------------------------------------------------------------------- + +test('byteArray accepts a plain frozen Uint8Array backed by an immutable ArrayBuffer', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([1, 2, 3, 4]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + harden(view); + t.is(passStyleOf(view), 'byteArray'); +}); + +test('byteArray accepts a plain frozen Uint8Array on a zero-length immutable ArrayBuffer (emulated, no own indexed properties)', t => { const iab = new ArrayBuffer(0).sliceToImmutable(); - defineProperty(iab, 'unexpected', { - value: 'extra', + const view = new Uint8Array(iab); + harden(view); + t.is(passStyleOf(view), 'byteArray'); +}); + +// --------------------------------------------------------------------------- +// Whole-buffer span (restrictive, issue #573). +// +// A byteArray view must cover its entire backing buffer one-to-one. A +// sub-view (`byteOffset > 0`, or a `length` shorter than the buffer's +// `byteLength`) is rejected, so a marshalled byteArray never conveys a +// hidden tail of its backing buffer beyond what the view reveals. The +// restrictive form is tracked for possible relaxation at +// https://github.com/endojs/endo-but-for-bots/issues/573 . +// --------------------------------------------------------------------------- + +test('byteArray accepts a whole-buffer-spanning view (byteOffset 0, length === buffer.byteLength)', t => { + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([1, 2, 3, 4]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + // Witness the whole-buffer span explicitly. + t.is(view.byteOffset, 0); + t.is(view.length, iab.byteLength); + harden(view); + t.is(passStyleOf(view), 'byteArray'); +}); + +test('byteArray rejects a sub-view with a non-zero byteOffset', t => { + // A window into the middle of a larger immutable buffer. Its bytes are a + // strict subset of the buffer, so the buffer carries more data than the + // view reveals: the data-reachability hazard the restrictive span check + // closes. Producers must re-slice such a window into its own immutable + // buffer (via `frozenBytes`) to obtain a byteArray. + const ab = new ArrayBuffer(8); + new Uint8Array(ab).set([1, 2, 3, 4, 5, 6, 7, 8]); + const iab = ab.sliceToImmutable(); + const sub = new Uint8Array(iab, 2, 4); + t.is(sub.byteOffset, 2); + harden(sub); + t.throws(() => passStyleOf(sub), { + message: /must span its whole backing buffer.*byteOffset 0/, + }); +}); + +test('byteArray rejects a sub-view shorter than its backing buffer (byteOffset 0, short length)', t => { + // A prefix window: byteOffset is 0 but the view stops short of the + // buffer's end, leaving a hidden tail. Rejected for the same + // data-reachability reason. + const ab = new ArrayBuffer(8); + new Uint8Array(ab).set([1, 2, 3, 4, 5, 6, 7, 8]); + const iab = ab.sliceToImmutable(); + const sub = new Uint8Array(iab, 0, 4); + t.is(sub.byteOffset, 0); + t.is(sub.length, 4); + t.not(sub.length, iab.byteLength); + harden(sub); + t.throws(() => passStyleOf(sub), { + message: /must span its whole backing buffer.*must equal buffer byteLength/, + }); +}); + +test('frozenBytes of a sub-view re-slices into a whole-buffer-spanning byteArray', t => { + // `frozenBytes` slices the view's window into a fresh immutable buffer, + // so the value it produces always spans its whole buffer one-to-one and + // passes the restrictive check even when the input is a sub-view. + const ab = new ArrayBuffer(8); + new Uint8Array(ab).set([1, 2, 3, 4, 5, 6, 7, 8]); + const window = new Uint8Array(ab, 2, 4); + const passable = frozenBytes(window); + t.is(passable.byteOffset, 0); + t.is(passable.length, passable.buffer.byteLength); + t.is(passStyleOf(passable), 'byteArray'); + t.deepEqual([...passable], [3, 4, 5, 6]); +}); + +test('byteArray accepts a plain frozen non-empty emulated Uint8Array with no own indexed properties', t => { + // The emulated freezable-TypedArray wrapper is a plain ordinary object + // with no own integer-indexed properties regardless of length. Data is + // accessible through the prototype-chain amplifier that resolves to the + // hidden genuine TypedArray. This test confirms the acceptance criterion + // is "no own indexed properties" rather than "zero length". + const ab = new ArrayBuffer(8); + new Uint8Array(ab).set([10, 20, 30, 40, 50, 60, 70, 80]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + // Verify the emulated wrapper has no own indexed properties. + t.deepEqual( + ownKeys(view).filter(k => typeof k === 'string' && /^\d+$/.test(k)), + [], + ); + harden(view); + t.is(passStyleOf(view), 'byteArray'); +}); + +test('byteArray rejects a Uint8Array backed by a mutable ArrayBuffer', t => { + // A `Uint8Array` on a mutable backing buffer cannot be frozen on either + // the emulated or the native path: integer-indexed exotic slots are + // non-configurable accessor-like properties and `Object.freeze` is + // defined to reject them. `passStyleOf` therefore rejects the wrapper + // with the "Cannot pass mutable typed arrays" message at the early + // `isFrozen` gate, before any helper is consulted. + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([1, 2, 3, 4]); + const view = new Uint8Array(ab); + t.throws(() => passStyleOf(view), { + message: /Cannot pass mutable typed arrays/, + }); +}); + +test('byteArray rejects a Uint8Array on an immutable ArrayBuffer with a shadowing index that disagrees with the buffer', t => { + // The emulated wrapper is a plain ordinary object whose `[[Prototype]]` + // is `Uint8Array.prototype`. Before freezing, `view[0] = 99` succeeds + // by creating an own data property on the wrapper that shadows the + // prototype-based integer-indexed read. The underlying immutable + // buffer's byte 0 is unchanged. After freezing, the wrapper carries an + // own enumerable data property `'0'` whose value (99) disagrees with + // the byte the underlying buffer would yield (10); the brand check + // catches the disagreement. + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([10, 20, 30, 40]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + view[0] = 99; + harden(view); + t.throws(() => passStyleOf(view), { + message: /must equal underlying byte/, + }); +}); + +test('byteArray rejects an emulated Uint8Array whose own indexed property matches the buffer byte', t => { + // On the emulated path the wrapper is a plain ordinary object with no own + // indexed properties; any own indexed property is post-construction + // tampering. The brand check rejects the wrapper even when the own + // property's value happens to agree with the underlying buffer byte, + // because an emulated wrapper (`ArrayBuffer.isView === false`) must have + // zero own indexed properties (not one, not length-many: zero). + const ab = new ArrayBuffer(4); + new Uint8Array(ab).set([10, 20, 30, 40]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + // Write the same value the buffer already holds at index 0. + view[0] = 10; + harden(view); + t.throws(() => passStyleOf(view), { + message: + /own indexed-property count.*does not match its shape.*emulated wrapper, isView false, needs 0/, + }); +}); + +test('byteArray rejects a Uint8Array on an immutable ArrayBuffer with an out-of-range own index', t => { + // An own data property whose key is a canonical integer but whose + // index is at or beyond the wrapper's `length`. This shape can arise + // on the emulated path through a deliberate `defineProperty` after + // construction; the brand check rejects it as not in `[0, length)`. + const ab = new ArrayBuffer(2); + new Uint8Array(ab).set([1, 2]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + defineProperty(view, '2', { + value: 3, writable: false, - enumerable: false, + enumerable: true, configurable: false, }); - harden(iab); - t.throws(() => passStyleOf(iab), { - message: /ByteArrays must not have own properties/, + freeze(view); + t.throws(() => passStyleOf(view), { + message: /must be below length/, }); }); -test('byteArray rejects an emulated immutable carrying an extra own accessor property', t => { +test('byteArray rejects a Uint8Array on an immutable ArrayBuffer with a non-index own property', t => { + // An own data property whose key is not a canonical integer. The + // brand check rejects all such keys regardless of descriptor shape. const iab = new ArrayBuffer(0).sliceToImmutable(); - defineProperty(iab, 'sneakyAccessor', { - get: () => 'gotcha', + const view = new Uint8Array(iab); + defineProperty(view, 'extra', { + value: 'hello', + writable: false, enumerable: false, configurable: false, }); - harden(iab); - t.throws(() => passStyleOf(iab), { - message: /ByteArrays must not have own properties/, + harden(view); + t.throws(() => passStyleOf(view), { + message: /must not have own non-index properties/, }); }); -test('byteArray rejects an emulated immutable carrying an extra enumerable own property', t => { - const iab = new ArrayBuffer(0).sliceToImmutable(); - defineProperty(iab, 'enumerableExtra', { - value: 1, +test('byteArray rejects a Uint8Array on an immutable ArrayBuffer with a non-canonical numeric key', t => { + // A key like `'01'` or `'1.5'` is not a canonical integer index per + // `CanonicalNumericIndexString`. The brand check rejects it as a + // non-index own property even though `Number(key)` is finite. + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + defineProperty(view, '01', { + value: 0, writable: false, enumerable: true, configurable: false, }); - harden(iab); - t.throws(() => passStyleOf(iab), { - message: /ByteArrays must not have own properties/, + freeze(view); + t.throws(() => passStyleOf(view), { + message: /must not have own non-index properties/, }); }); -test('byteArray rejects an emulated immutable carrying a same-key Symbol with a different name', t => { - // A second emulated-immutable-shaped Symbol-keyed own property (not the - // canonical `Symbol.toStringTag`) must trip the own-key allowlist. - const iab = new ArrayBuffer(0).sliceToImmutable(); - defineProperty(iab, Symbol.iterator, { - value: () => {}, +test('byteArray rejects a Uint8Array on an immutable ArrayBuffer with an accessor own property at an integer index', t => { + // An accessor own property at a canonical index key is not a data + // property of the spec-required shape; reject it. The wrapper's + // shape after this `defineProperty` plus `freeze` is observable as a + // canonical index key with an accessor descriptor; the brand check + // rejects on the missing `value` slot. + const ab = new ArrayBuffer(2); + new Uint8Array(ab).set([1, 2]); + const iab = ab.sliceToImmutable(); + const view = new Uint8Array(iab); + defineProperty(view, '0', { + get: () => 1, + enumerable: true, + configurable: false, + }); + freeze(view); + t.throws(() => passStyleOf(view), { + message: /must be an enumerable number-valued data property/, + }); +}); + +test('byteArray rejects a Uint8Array whose backing immutable ArrayBuffer carries an extraneous own property', t => { + // The backing-buffer sub-check still walks the IAB's own keys with the + // canonical allowlist (`[Symbol.toStringTag]` only). A buffer carrying + // an unallowed own data property fails the sub-check before the + // wrapper's own keys are walked. + const iab = new ArrayBuffer(2).sliceToImmutable(); + defineProperty(iab, 'tampered', { + value: 1, writable: false, enumerable: false, configurable: false, }); - harden(iab); - t.throws(() => passStyleOf(iab), { + const view = new Uint8Array(iab); + harden(view); + t.throws(() => passStyleOf(view), { message: /ByteArrays must not have own properties/, }); }); + +test('passStyleOf fall-through blames element type, not mutability, for a non-Uint8Array typed array', t => { + // A genuine TypedArray claimed by no helper falls through to the late + // guard in `passStyleOf`. That guard is only reachable when the early + // `isFrozen` gate is bypassed: under unsafe harden taming (`isFrozen` + // returns true for unfrozen objects) on the shim leg, or on a native + // Immutable-ArrayBuffer engine for a genuine frozen TypedArray over an + // immutable buffer. Under safe taming these values are instead caught by + // the early gate with the (accurate) mutable message; see the + // `@endo/marshal` `stringify` errors test. + // + // `Object.isFrozen({}) === true` reliably detects the unsafe taming + // (isFrozen is unreliable there), which is the only way to reach the + // fall-through on this (shim) leg. + if (!Object.isFrozen({})) { + t.pass('safe taming: late guard unreachable on this leg'); + return; + } + // A non-`Uint8Array` typed array fails for its element type. The message + // must not blame mutability (erights review, + // endojs/endo-but-for-bots#475): a frozen non-`Uint8Array` typed array + // over an immutable buffer reaches this same guard, where mutability is + // genuinely not the problem. + const int16 = harden(new Int16Array(1)); + const int16Err = t.throws(() => passStyleOf(int16), { + message: /Cannot pass typed arrays other than Uint8Array/, + }); + t.notRegex(int16Err.message, /mutable/); + // A `Uint8Array` reaches this guard only with a mutable backing buffer + // (an immutable-backed one is always claimed by the byteArray helper), + // so "mutable" remains accurate for it. + const u8 = harden(new Uint8Array(1)); + t.throws(() => passStyleOf(u8), { + message: /Cannot pass mutable typed arrays/, + }); +}); diff --git a/packages/pass-style/tools/arb-passable.js b/packages/pass-style/tools/arb-passable.js index a89b37c71c..89e193bb73 100644 --- a/packages/pass-style/tools/arb-passable.js +++ b/packages/pass-style/tools/arb-passable.js @@ -42,9 +42,13 @@ export const makeArbitraries = (fc, exclusions = []) => { // because we may go through a phase where only `sliceToImmutable` is // provided when the shim is run on Hermes. // See https://github.com/endojs/endo/pull/2785 - ...[fc.uint8Array().map(arr => arr.buffer.sliceToImmutable())].filter( - () => !exclusions.includes('byteArray'), - ), + // + // The byteArray pass style is now a plain frozen `Uint8Array` backed + // by a plain frozen immutable `ArrayBuffer`. Wrap the IAB in + // `new Uint8Array(...)` before harden so it satisfies the brand check. + ...[ + fc.uint8Array().map(arr => new Uint8Array(arr.buffer.sliceToImmutable())), + ].filter(() => !exclusions.includes('byteArray')), fc.constantFrom(-0, NaN, Infinity, -Infinity), // `noNullPrototype` keeps fast-check 4 from generating `{__proto__:null}` // objects, which are not valid copyRecords (they must inherit from diff --git a/packages/patterns/src/patterns/patternMatchers.js b/packages/patterns/src/patterns/patternMatchers.js index 5abc8468e4..696c57b06c 100644 --- a/packages/patterns/src/patterns/patternMatchers.js +++ b/packages/patterns/src/patterns/patternMatchers.js @@ -1311,7 +1311,7 @@ const makePatternKit = () => { // prettier-ignore return ( confirmKind(specimen, 'byteArray', reject) && - (/** @type {ArrayBuffer} */ (specimen).byteLength <= byteLengthLimit || + (/** @type {Uint8Array} */ (specimen).byteLength <= byteLengthLimit || reject && reject`byteArray ${specimen} must not be bigger than ${byteLengthLimit}`) ); }, diff --git a/packages/patterns/src/type-from-pattern.ts b/packages/patterns/src/type-from-pattern.ts index 1ae8669f55..0e37389bea 100644 --- a/packages/patterns/src/type-from-pattern.ts +++ b/packages/patterns/src/type-from-pattern.ts @@ -120,7 +120,7 @@ type PassableFromPattern

= */ type TFLeafMap = { any: Passable; - byteArray: ArrayBuffer; + byteArray: Uint8Array; string: Payload; number: Payload; bigint: Payload; @@ -163,7 +163,7 @@ type TFKindMap = { bigint: bigint; string: string; symbol: symbol; - byteArray: ArrayBuffer; // TODO: update to Uint8Array when @endo/pass-style changes the byteArray type + byteArray: Uint8Array; copyRecord: CopyRecord; copyArray: CopyArray; copySet: CopySet; diff --git a/packages/patterns/test/pattern-limits.test.js b/packages/patterns/test/pattern-limits.test.js index 49692dfcaf..c8e3463192 100644 --- a/packages/patterns/test/pattern-limits.test.js +++ b/packages/patterns/test/pattern-limits.test.js @@ -216,14 +216,16 @@ const runTests = (successCase, failCase) => { } // byteLengthLimit { - const specimen = new ArrayBuffer(1000).transferToImmutable(); + const specimen = new Uint8Array( + new ArrayBuffer(1000).transferToImmutable(), + ); successCase(specimen, M.byteArray()); successCase(specimen, M.byteArray(harden({ byteLengthLimit: 1001 }))); successCase(specimen, M.byteArray(harden({ byteLengthLimit: 1000 }))); failCase( specimen, M.byteArray(harden({ byteLengthLimit: 999 })), - /byteArray "\[.*ArrayBuffer\]" must not be bigger than 999/, + /byteArray ".*" must not be bigger than 999/, ); } // numSetElementsLimit diff --git a/packages/patterns/test/types.test-d.ts b/packages/patterns/test/types.test-d.ts index 92d5cd3c69..113d1c2f96 100644 --- a/packages/patterns/test/types.test-d.ts +++ b/packages/patterns/test/types.test-d.ts @@ -268,11 +268,11 @@ const passable: Passable = null as any; expectTypeOf(null as unknown as T).toExtend(); } -// M.byteArray() → ArrayBuffer (via kind) +// M.byteArray() → Uint8Array (via kind) { const p = M.byteArray(); type T = TypeFromPattern; - expectTypeOf(null as unknown as T).toEqualTypeOf(); + expectTypeOf(null as unknown as T).toEqualTypeOf(); } // M.record() → CopyRecord diff --git a/packages/relay-server/src/protocol.js b/packages/relay-server/src/protocol.js index 090147af4c..971ab5914f 100644 --- a/packages/relay-server/src/protocol.js +++ b/packages/relay-server/src/protocol.js @@ -56,15 +56,20 @@ export const encodeFrame = (type, payload = new Uint8Array(0)) => { }; /** - * @param {Uint8Array | ArrayBuffer} data + * Decode a wire frame. Callers normalize whatever the transport hands them + * (a raw `ArrayBuffer` from a WebSocket binary message, a Node `Buffer`) into + * a `Uint8Array` at the transport edge — see the `ws.on('message', …)` handlers + * in `relay.js` and the daemon's `ws-relay.js` — so this decoder is typed + * `Uint8Array` only, with no buffer-vs-view disjunction. + * + * @param {Uint8Array} data * @returns {{ type: number, payload: Uint8Array }} */ export const decodeFrame = data => { - const bytes = data instanceof Uint8Array ? data : new Uint8Array(data); - if (bytes.length < 1) { + if (data.length < 1) { throw new Error('Empty frame'); } - return { type: bytes[0], payload: bytes.subarray(1) }; + return { type: data[0], payload: data.subarray(1) }; }; // --- Channel ID helpers (4 bytes, big-endian) --- diff --git a/packages/ses/src/make-hardener.js b/packages/ses/src/make-hardener.js index 300c34d2d7..58a377d10c 100644 --- a/packages/ses/src/make-hardener.js +++ b/packages/ses/src/make-hardener.js @@ -68,7 +68,19 @@ assert(getTypedArrayToStringTag); // Exported for tests. /** - * Duplicates packages/marshal/src/helpers/passStyle-helpers.js to avoid a dependency. + * Duplicates packages/pass-style/src/passStyle-helpers.js to avoid a dependency. + * + * Deliberately a genuine TypedArray brand check via the `%TypedArray%` + * `[Symbol.toStringTag]` getter, NOT `ArrayBuffer.isView`. Both are + * unspoofable internal-slot checks, but `isView` is also true for a + * `DataView`, whereas only a TypedArray is an integer-indexed exotic whose + * permanently-writable indexed elements make `Object.freeze` throw. That + * freeze-throw is the sole reason `harden` special-cases here (see + * `freezeTypedArray`); a `DataView` freezes normally and must take the + * ordinary `freeze` path, so the DataView-inclusive `isView` would be the + * wrong, less-precise test. (`byteArray.js` commits to `isView` for a + * different question — emulated-vs-native shape on an already-known + * `Uint8Array` — where DataViews are already excluded.) * * @param {unknown} object */ diff --git a/packages/ses/test/immutable-arraybuffer.test.js b/packages/ses/test/immutable-arraybuffer.test.js index dce8fdc0aa..3d98457c16 100644 --- a/packages/ses/test/immutable-arraybuffer.test.js +++ b/packages/ses/test/immutable-arraybuffer.test.js @@ -12,3 +12,43 @@ test('ses Immutable ArrayBuffer shim installed and hardened', t => { t.true(isFrozen(iabProto)); t.true(isFrozen(iabProto.slice)); }); + +test('ses: emulated freezable Uint8Array is hardened and Object.isFrozen(view) === true after lockdown', t => { + // After lockdown(), harden() has been applied to all primordials. + // The emulated freezable wrapper is a plain object inheriting from + // Uint8Array.prototype (a frozen prototype), so Object.isFrozen(view) + // is true after `Object.freeze(view)` (which harden invokes transitively + // via the prototype-walk phase). + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + // Object.freeze on a plain object succeeds (no integer-indexed exotic slots). + Object.freeze(view); + t.true(isFrozen(view)); +}); + +test('ses: permits walk does not complain about %TypedArrayPrototype% slots the shim installs', t => { + // If lockdown() completed without throwing a permits-walk TypeError, the + // shim-installed slots (buffer accessor replacement, mutator-throws wrappers, + // read-delegate wrappers) all fit within the existing %TypedArrayPrototype% + // permits entry. This test asserts the post-lockdown surface is present. + const tp = getPrototypeOf(Uint8Array.prototype); + t.true('buffer' in tp); + t.true('byteLength' in tp); + t.true('copyWithin' in tp); + t.true('fill' in tp); + t.true('reverse' in tp); + t.true('set' in tp); + t.true('sort' in tp); +}); + +test('ses: emulated freezable view mutator methods still throw after lockdown', t => { + // The harden phase freezes all primordials; verify the lib's discrimination + // logic (brand WeakMap lookup) still works correctly after hardening. + const iab = new ArrayBuffer(4).sliceToImmutable(); + const view = new Uint8Array(iab); + t.throws(() => view.fill(0), { instanceOf: TypeError }); + t.throws(() => view.set([0]), { instanceOf: TypeError }); + t.throws(() => view.reverse(), { instanceOf: TypeError }); + t.throws(() => view.sort(), { instanceOf: TypeError }); + t.throws(() => view.copyWithin(0, 1), { instanceOf: TypeError }); +}); diff --git a/packages/ses/test/is-typed-array.test.js b/packages/ses/test/is-typed-array.test.js index 09d15514db..c6b239dbcf 100644 --- a/packages/ses/test/is-typed-array.test.js +++ b/packages/ses/test/is-typed-array.test.js @@ -54,4 +54,9 @@ test('isTypedArray negative cases', t => { t.assert(!isTypedArray(() => {})); t.assert(!isTypedArray([])); t.assert(!isTypedArray(new ArrayBuffer(1))); + // A DataView is the discriminating case: `ArrayBuffer.isView` is true for it, + // but it is not an integer-indexed exotic TypedArray, so the getter-based + // brand check must (and does) report false. This is why these sites use the + // getter rather than `ArrayBuffer.isView`. + t.assert(!isTypedArray(new DataView(new ArrayBuffer(4)))); }); diff --git a/packages/thixotrope/package.json b/packages/thixotrope/package.json index fffc6ff148..0d4e73314e 100644 --- a/packages/thixotrope/package.json +++ b/packages/thixotrope/package.json @@ -41,11 +41,11 @@ }, "dependencies": { "@endo/base64": "workspace:^", - "@endo/bytes": "workspace:^", "@endo/errors": "workspace:^", "@endo/eventual-send": "workspace:^", "@endo/far": "workspace:^", "@endo/harden": "workspace:^", + "@endo/immutable-arraybuffer": "workspace:^", "@endo/ocapn": "workspace:^", "@endo/stream": "workspace:^", "@noble/hashes": "^2.3.0" diff --git a/packages/thixotrope/src/daemon.js b/packages/thixotrope/src/daemon.js index f3e634793f..ada5ec28fd 100644 --- a/packages/thixotrope/src/daemon.js +++ b/packages/thixotrope/src/daemon.js @@ -2,7 +2,7 @@ /* global crypto */ import harden from '@endo/harden'; import { decodeBase64, encodeBase64 } from '@endo/base64'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { Fail, q } from '@endo/errors'; import { E, Far } from '@endo/far'; import { makeOcapn } from '@endo/ocapn'; @@ -91,7 +91,7 @@ const randomHex128 = () => { }; const textEncoder = new TextEncoder(); -const SHELL_SWISSNUM = bytesToImmutable(textEncoder.encode('shell')); +const SHELL_SWISSNUM = frozenBytes(textEncoder.encode('shell')); // The endpoint's pseudo-worker id: its session records (resource // descriptions, pending answers) live in this worker store. const ENDPOINT_ID = 'e'.repeat(32); @@ -632,8 +632,8 @@ export const makeThixotropeDaemon = async ({ const session = await endpointSessionP; const bytes = typeof secret === 'string' - ? bytesToImmutable(textEncoder.encode(secret)) - : bytesToImmutable(secret); + ? frozenBytes(textEncoder.encode(secret)) + : frozenBytes(secret); return E(/** @type {any} */ (session.getBootstrap())).fetch(bytes); }; diff --git a/packages/thixotrope/test/durable-worker-session-xs.test.js b/packages/thixotrope/test/durable-worker-session-xs.test.js index 55ba88b1f3..3562c53b07 100644 --- a/packages/thixotrope/test/durable-worker-session-xs.test.js +++ b/packages/thixotrope/test/durable-worker-session-xs.test.js @@ -16,7 +16,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { E } from '@endo/eventual-send'; import { makeOcapn } from '@endo/ocapn'; import { makeOcapnHub } from '@endo/ocapn/hub'; @@ -47,7 +47,7 @@ if (!available) { const textEncoder = new TextEncoder(); /** @param {string} text */ -const bytesOf = text => bytesToImmutable(textEncoder.encode(text)); +const bytesOf = text => frozenBytes(textEncoder.encode(text)); const SHELL_SWISSNUM = bytesOf('shell'); const COUNTER_SOURCE = ` diff --git a/packages/thixotrope/test/durable-worker-session.test.js b/packages/thixotrope/test/durable-worker-session.test.js index 74065ce32f..d4f7532bef 100644 --- a/packages/thixotrope/test/durable-worker-session.test.js +++ b/packages/thixotrope/test/durable-worker-session.test.js @@ -12,7 +12,7 @@ */ import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { E } from '@endo/eventual-send'; import { makeOcapn } from '@endo/ocapn'; import { makeOcapnHub } from '@endo/ocapn/hub'; @@ -28,7 +28,7 @@ import { makeMemoryStore } from '../src/store-fs.js'; const textEncoder = new TextEncoder(); /** @param {string} text */ -const bytesOf = text => bytesToImmutable(textEncoder.encode(text)); +const bytesOf = text => frozenBytes(textEncoder.encode(text)); const SHELL_SWISSNUM = bytesOf('shell'); const COUNTER_SOURCE = ` diff --git a/packages/thixotrope/test/hub.test.js b/packages/thixotrope/test/hub.test.js index 38571b3c1d..f31cc86c7a 100644 --- a/packages/thixotrope/test/hub.test.js +++ b/packages/thixotrope/test/hub.test.js @@ -12,7 +12,7 @@ import test from '@endo/ses-ava/test.js'; import harden from '@endo/harden'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { E } from '@endo/eventual-send'; import { makeOcapn } from '@endo/ocapn'; import { makeOcapnHub } from '@endo/ocapn/hub'; @@ -23,7 +23,7 @@ import { makeWorkerPeer } from '../src/worker-peer.js'; const textEncoder = new TextEncoder(); /** @param {string} text */ -const bytesOf = text => bytesToImmutable(textEncoder.encode(text)); +const bytesOf = text => frozenBytes(textEncoder.encode(text)); const SHELL_SWISSNUM = bytesOf('shell'); const COUNTER_SOURCE = ` diff --git a/packages/thixotrope/test/worker-peer-xs.test.js b/packages/thixotrope/test/worker-peer-xs.test.js index 1fc2ef4d17..4eacde529a 100644 --- a/packages/thixotrope/test/worker-peer-xs.test.js +++ b/packages/thixotrope/test/worker-peer-xs.test.js @@ -21,7 +21,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { E } from '@endo/eventual-send'; import { makeOcapn } from '@endo/ocapn'; import { syrupCodec } from '@endo/ocapn/syrup'; @@ -48,7 +48,7 @@ if (!available) { } // Wire swissnums are (immutable) bytes, as `enlivenSturdyRef` encodes. -const SHELL_SWISSNUM = bytesToImmutable(new TextEncoder().encode('shell')); +const SHELL_SWISSNUM = frozenBytes(new TextEncoder().encode('shell')); const COUNTER_SOURCE = ` (() => { diff --git a/packages/thixotrope/test/worker-peer.test.js b/packages/thixotrope/test/worker-peer.test.js index 288d6bfb02..f9d4592b8a 100644 --- a/packages/thixotrope/test/worker-peer.test.js +++ b/packages/thixotrope/test/worker-peer.test.js @@ -1,7 +1,7 @@ // @ts-check import test from '@endo/ses-ava/test.js'; -import { bytesToImmutable } from '@endo/bytes/to-immutable.js'; +import { frozenBytes } from '@endo/immutable-arraybuffer'; import { E } from '@endo/eventual-send'; import { makeOcapn } from '@endo/ocapn'; import { syrupCodec } from '@endo/ocapn/syrup'; @@ -10,7 +10,7 @@ import { makePipeNetwork } from '../src/pipe-network.js'; import { makeWorkerPeer } from '../src/worker-peer.js'; // Wire swissnums are (immutable) bytes, as `enlivenSturdyRef` encodes. -const SHELL_SWISSNUM = bytesToImmutable(new TextEncoder().encode('shell')); +const SHELL_SWISSNUM = frozenBytes(new TextEncoder().encode('shell')); const COUNTER_SOURCE = ` (() => { diff --git a/packages/thixotrope/tsconfig.composite.json b/packages/thixotrope/tsconfig.composite.json index 482b88dcd2..3540e6e809 100644 --- a/packages/thixotrope/tsconfig.composite.json +++ b/packages/thixotrope/tsconfig.composite.json @@ -8,9 +8,6 @@ { "path": "../base64/tsconfig.composite.json" }, - { - "path": "../bytes/tsconfig.composite.json" - }, { "path": "../errors/tsconfig.composite.json" }, @@ -23,6 +20,9 @@ { "path": "../harden/tsconfig.composite.json" }, + { + "path": "../immutable-arraybuffer/tsconfig.composite.json" + }, { "path": "../ocapn/tsconfig.composite.json" }, diff --git a/yarn.lock b/yarn.lock index 2fe26efbfa..0eb7b0a756 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1366,7 +1366,7 @@ __metadata: languageName: unknown linkType: soft -"@endo/ascii@workspace:packages/ascii": +"@endo/ascii@workspace:^, @endo/ascii@workspace:packages/ascii": version: 0.0.0-use.local resolution: "@endo/ascii@workspace:packages/ascii" dependencies: @@ -2477,6 +2477,7 @@ __metadata: version: 0.0.0-use.local resolution: "@endo/immutable-arraybuffer@workspace:packages/immutable-arraybuffer" dependencies: + "@endo/harden": "workspace:^" ava: "catalog:dev" c8: "catalog:dev" eslint: "catalog:dev" @@ -2643,11 +2644,14 @@ __metadata: version: 0.0.0-use.local resolution: "@endo/marshal@workspace:packages/marshal" dependencies: + "@endo/bytes": "workspace:^" "@endo/common": "workspace:^" "@endo/env-options": "workspace:^" "@endo/errors": "workspace:^" "@endo/eventual-send": "workspace:^" "@endo/harden": "workspace:^" + "@endo/hex": "workspace:^" + "@endo/immutable-arraybuffer": "workspace:^" "@endo/init": "workspace:^" "@endo/lockdown": "workspace:^" "@endo/nat": "workspace:^" @@ -2816,12 +2820,14 @@ __metadata: version: 0.0.0-use.local resolution: "@endo/ocapn@workspace:packages/ocapn" dependencies: + "@endo/ascii": "workspace:^" "@endo/bytes": "workspace:^" "@endo/cbor": "workspace:^" "@endo/chacha12": "workspace:^" "@endo/eventual-send": "workspace:^" "@endo/harden": "workspace:^" "@endo/hex": "workspace:^" + "@endo/immutable-arraybuffer": "workspace:^" "@endo/init": "workspace:^" "@endo/lockdown": "workspace:^" "@endo/marshal": "workspace:^" @@ -2865,6 +2871,7 @@ __metadata: "@endo/errors": "workspace:^" "@endo/eventual-send": "workspace:^" "@endo/harden": "workspace:^" + "@endo/immutable-arraybuffer": "workspace:^" "@endo/init": "workspace:^" "@endo/is-well-formed-string": "workspace:^" "@endo/promise-kit": "workspace:^" @@ -3370,12 +3377,12 @@ __metadata: resolution: "@endo/thixotrope@workspace:packages/thixotrope" dependencies: "@endo/base64": "workspace:^" - "@endo/bytes": "workspace:^" "@endo/compartment-mapper": "workspace:^" "@endo/errors": "workspace:^" "@endo/eventual-send": "workspace:^" "@endo/far": "workspace:^" "@endo/harden": "workspace:^" + "@endo/immutable-arraybuffer": "workspace:^" "@endo/init": "workspace:^" "@endo/lockdown": "workspace:^" "@endo/ocapn": "workspace:^"