Add @endo/capn-proto: Cap'n Proto RPC implementation by kumavis · Pull Request #3213 · endojs/endo

kumavis · 2026-04-27T18:06:24Z

Summary

Adds @endo/capn-proto, a pure-JavaScript implementation of the Cap'n Proto RPC protocol built on @endo/eventual-send. Remote capabilities surface as HandledPromise-backed Presence values usable through the wavy-dot E() operator, so eventual-send code talks to a Cap'n Proto peer without protocol-specific glue.

Scope

RPC Level 1 (basic two-party RPC, promise pipelining, embargoes).
RPC Level 3 (three-party handoff via Provide / Accept / vine fallback).
Out of scope: Level 2 (saved references / sturdy refs) and Level 4 (Join).

What's implemented

Full wire format: segments, struct / list / far / capability pointers, text + data, stream framing.
Four-table state machine (Questions, Answers, Imports, Exports) with proper Finish / Release lifecycle.
Promise pipelining: calls on unresolved promises are dispatched as PromisedAnswer with transform paths.
Disembargo (both arms) for the Tribble four-way race during L3 handoff.
HandledPromise handler so E(remote).method(args) does the right thing transparently.
FinalizationRegistry-driven GC for imports.
Interface registry mapping loadSchema(...).registerInterface(...) to method ordinals — no name-to-ordinal heuristic.
Two-party VatNetwork plus a loopback helper for tests.

Interop validation

Wire format is verified against the reference C++ implementation by piping every message variant through the apt-installable capnp CLI (1.0.1 on Ubuntu) and checking the round-trip in CI:

L1: Bootstrap, Call, Return, Finish, Release, Resolve, Disembargo.
L3: Provide.recipient, Accept.provision, ThirdPartyCapDescriptor.id, and the embargo / vine variants of Disembargo.

A note on L3 wire-format compatibility

Cap'n Proto's L3 surface shifted between the 1.0.x line (currently shipped by Debian / Ubuntu / Homebrew, what CI uses) and current mainline (1.5-dev as of May 2026):

libcapnp 1.0.x exposes only the L0 connect / accept VatNetwork hooks — L3 is marked TODO(someday) in the C++ header, so there is no way to instantiate an L3-capable VatNetwork against 1.0.x even with custom subclassing.
Current mainline adds the L3 VatNetwork API (canIntroduceTo, introduceTo, connectToIntroduced, awaitThirdParty, completeThirdParty, forwardThirdPartyToContact) AND breaks the on-the-wire shapes: Accept.embargo widens from Bool to Data, Disembargo.context.accept widens from Void to Data, and the provide arm is dropped (B forwards the accept-arm Disembargo with a promisedAnswer target instead).

This package targets the 1.0.x shape so byte-level interop tests work in CI without building libcapnp from source. A live-L3 test fixture against a custom C++ VatNetwork would require both a wire-format port to mainline and a build-from-source CI step — those are documented in the README and deferred.

CI hygiene

Workflow security findings from zizmor are addressed on this branch: excessive GITHUB_TOKEN permissions narrowed, persist-credentials: false set on checkout where credentials aren't needed, and stale action refs updated.

Test plan

Unit suite (157 tests) passes on Node 22 against capnp 1.0.1.
Byte-level interop suite round-trips every L1 and L3 message variant through the reference CLI.
Live L3 against a custom C++ VatNetwork — deferred (see README).

https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

changeset-bot · 2026-04-27T18:06:37Z

⚠️ No Changeset found

Latest commit: 5957ef5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copilot

Pull request overview

Introduces a new package, @endo/capn-proto, providing a pure-JavaScript Cap’n Proto RPC implementation layered on @endo/eventual-send, including a hand-written wire-format encoder/decoder, a connection/table state machine, and an extensive test suite with optional byte-level interop verification against the capnp CLI.

Changes:

Added Cap’n Proto wire-format primitives (segments, pointers, lists/structs, text/data) and stream framing.
Implemented RPC connection logic (four tables, import/export registries, promise pipelining, dispatch, partial Level 3 three-party handoff, trap integration).
Added unit tests plus an interop test and a dedicated GitHub Actions workflow to validate wire compatibility.

Reviewed changes

Copilot reviewed 49 out of 50 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
packages/capn-proto/package.json	New package manifest, exports, scripts, deps.
packages/capn-proto/README.md	Package overview + quick start documentation.
packages/capn-proto/rpc-twoparty.capnp	Two-party network schema file (vendored).
packages/capn-proto/src/index.js	Public entrypoint exports (API surface).
packages/capn-proto/src/rpc-system.js	Top-level `makeCapnp` factory and public facade.
packages/capn-proto/src/connection.js	Core per-peer connection: tables, encoding/decoding, call/resolve/abort flow.
packages/capn-proto/src/dispatch.js	Inbound message handlers for RPC message variants.
packages/capn-proto/src/handler.js	`HandledPromise` handler for translating E() calls into Call messages.
packages/capn-proto/src/interfaces.js	Interface registry for (interfaceId, methodId) resolution.
packages/capn-proto/src/payload-codec.js	JSON + capTable payload encoding/decoding convention.
packages/capn-proto/src/pipeline.js	PromisedAnswer transform-path helpers.
packages/capn-proto/src/loopback.js	Loopback harness for tests / in-process pairing.
packages/capn-proto/src/two-party.js	Simple TwoParty VatNetwork implementation for tests/basic use.
packages/capn-proto/src/three-party.js	Level 3 three-party handoff scaffolding and message handlers.
packages/capn-proto/src/embargo.js	Disembargo bookkeeping for pipelining ordering guarantees.
packages/capn-proto/src/exports.js	Export-table identity/refcount management.
packages/capn-proto/src/imports.js	Import-table presence creation and identity preservation.
packages/capn-proto/src/finalize.js	Weak-value map with `FinalizationRegistry` support.
packages/capn-proto/src/tables/id-allocator.js	32-bit ID allocator with free list.
packages/capn-proto/src/tables/four-tables.js	Questions/Answers/Imports/Exports table container.
packages/capn-proto/src/trap.js	SharedArrayBuffer/Atomics “trap” adapters built on `@endo/captp`.
packages/capn-proto/src/wire/segment.js	Segment arena + message builder/reader.
packages/capn-proto/src/wire/framing.js	Stream framing encode/decode for segments.
packages/capn-proto/src/wire/pointer.js	Pointer encoding/decoding + far-pointer resolution.
packages/capn-proto/src/wire/struct.js	Struct allocation + primitive field read/write helpers.
packages/capn-proto/src/wire/list.js	List allocation + composite list helpers.
packages/capn-proto/src/wire/text.js	Text/Data helpers on top of list-of-byte encoding.
packages/capn-proto/src/proto/schema.js	Hard-coded rpc.capnp layout constants used by codecs.
packages/capn-proto/src/proto/messages.js	RPC Message encoders/decoders on top of wire primitives.
packages/capn-proto/test/basic.test.js	Basic bootstrap + call + error propagation tests.
packages/capn-proto/test/abort.test.js	Abort behavior tests.
packages/capn-proto/test/crosstalk.test.js	Concurrent bidirectional call tests.
packages/capn-proto/test/disembargo.test.js	Disembargo encode/decode + tracker behavior tests.
packages/capn-proto/test/gc.test.js	Finish/Release lifecycle and stats assertions.
packages/capn-proto/test/identity.test.js	Presence identity preservation tests.
packages/capn-proto/test/interfaces.test.js	InterfaceRegistry behavior tests.
packages/capn-proto/test/interop.test.js	Optional interop with reference `capnp` CLI (spawn-based).
packages/capn-proto/test/loopback.test.js	Loopback helper tests.
packages/capn-proto/test/payload-codec.test.js	Payload codec round-trip tests.
packages/capn-proto/test/pipelining.test.js	Promise pipelining semantics tests.
packages/capn-proto/test/promise-resolve.test.js	Resolve-message behavior for returned promises (resolve/reject).
packages/capn-proto/test/proto/messages.test.js	Message codec round-trip tests for all variants used.
packages/capn-proto/test/three-party.test.js	Provide/Accept/Disembargo encode/decode and third-party cap descriptor tests.
packages/capn-proto/test/trap.test.js	Trap constants + export presence tests.
packages/capn-proto/test/tribble-4way.test.js	Ordering test intended for Tribble race scenario.
packages/capn-proto/test/wire/framing.test.js	Framing round-trip and header sizing tests.
packages/capn-proto/test/wire/pointer.test.js	Pointer encoding round-trip tests for pointer kinds.
.github/workflows/capn-proto-interop.yml	CI workflow to run interop test with `capnproto` installed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Repo-wide prettier formatting on every capn-proto source and test file (restores `lint:prettier` CI gate). Also addresses the Copilot review on PR #3213: - src/proto/messages.js: consolidate imports at top of file (the earlier diff interleaved them with helper declarations, which would trip `import/first`); drop unused `readPtrAt`, `primitiveElementByteOffset`, `LIST_BYTE` imports and their `void` suppressions. - src/dispatch.js, src/three-party.js: drop unused `Fail` imports and trailing `void Fail;` markers. - src/dispatch.js: handleReturn now also `delete`s the QuestionEntry and releases the questionId after sending Finish, so the allocator can recycle ids instead of growing monotonically. - src/wire/struct.js: writeUint16 / writeUint32 / writeUint64 / writeBool now bounds-check against the struct's data section and Fail loudly instead of writing past the boundary. - src/wire/pointer.js: writePointer now validates each sub-field against its representable range (signed 30-bit offset, uint16 data/ptr words, uint29 elemCount, uint32 segmentId / cap index) and Fails on out-of-range input instead of silently masking. - src/wire/segment.js: new MessageBuilder.allocateInSegment(segId, words) primitive that allocates within a specific segment, growing it if needed. Used by allocStruct / allocList / allocCompositeList to place far-pointer landing pads next to their payloads. The previous code called `allocate(2)` and asserted segId match, which would Fail spuriously if an earlier segment had spare room. - src/three-party.js: handleDisembargoAccept tracks the real Provide questionId via a target-id → questionId map populated on initiateProvide, replacing the hard-coded `questionId: 0`. - README.md: replace the inaccurate Quick start (which referenced a non-existent `network` argument and a `getBootstrap(remoteVatId)` signature) with one that matches the actual `makeCapnp({ send, bootstrap })` API. - test/tribble-4way.test.js: rewrite to actually exercise four independent makeCapnp instances chained A→B→C→D and assert E-order at Bob, removing the dead `wireUp`/`inboxes`/`schedule` scaffolding the previous version had after collapsing to a 2-vat shortcut. All 78 tests pass locally. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

Copilot

Pull request overview

This PR adds a new workspace package, @endo/capn-proto, implementing Cap’n Proto RPC in pure JavaScript on top of @endo/eventual-send, including wire-format codecs, connection state/tables, and a fairly comprehensive test suite plus a CI interop workflow.

Changes:

Introduces a full Cap’n Proto RPC message codec + wire-format (segments/pointers/lists/structs/framing) and an RPC connection implementation (tables, imports/exports, dispatch, pipelining, three-party handoff).
Adds package scaffolding (exports, tsconfigs, README) and unit tests covering many protocol and runtime behaviors (pipelining, identity, GC lifecycle, interop).
Adds a GitHub Actions workflow to validate byte-level interoperability against the capnp CLI.

Reviewed changes

Copilot reviewed 51 out of 52 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
yarn.lock	Adds workspace entry for `@endo/capn-proto` and its deps.
packages/capn-proto/package.json	New package manifest, exports, scripts, deps.
packages/capn-proto/README.md	Package documentation and quick-start usage.
packages/capn-proto/tsconfig.json	Typechecking config for package sources/tests.
packages/capn-proto/tsconfig.build.json	Build typecheck config excluding tests.
packages/capn-proto/rpc-twoparty.capnp	Two-party network schema file.
packages/capn-proto/src/index.js	Public entrypoint exports (API + codecs + wire helpers).
packages/capn-proto/src/rpc-system.js	Top-level `makeCapnp` factory wiring connection + interfaces.
packages/capn-proto/src/connection.js	Core connection state, send/dispatch, tables integration.
packages/capn-proto/src/dispatch.js	Inbound message handlers (Call/Return/Resolve/etc.).
packages/capn-proto/src/handler.js	`HandledPromise` handler that turns `E()` calls into Calls.
packages/capn-proto/src/interfaces.js	Interface registry mapping method names ↔ ordinals.
packages/capn-proto/src/payload-codec.js	JSON + capTable payload encoding/decoding.
packages/capn-proto/src/pipeline.js	Helpers for transform paths / pipelining utilities.
packages/capn-proto/src/exports.js	Export table / refcount bookkeeping.
packages/capn-proto/src/imports.js	Import table / Presence creation + identity preservation.
packages/capn-proto/src/tables/four-tables.js	Per-connection question/answer/import/export tables.
packages/capn-proto/src/tables/id-allocator.js	32-bit ID allocator used by tables.
packages/capn-proto/src/embargo.js	Disembargo tracking utilities.
packages/capn-proto/src/three-party.js	Level-3 three-party handoff flow implementation.
packages/capn-proto/src/two-party.js	Two-party VatNetwork helper implementation.
packages/capn-proto/src/loopback.js	Test/helper loopback transport pairing two instances.
packages/capn-proto/src/trap.js	SharedArrayBuffer/Atomics “trap” adapters for capn-proto bytes.
packages/capn-proto/src/finalize.js	Weak-value map w/ FinalizationRegistry-based finalization.
packages/capn-proto/src/proto/schema.js	rpc.capnp layout constants for encoders/decoders.
packages/capn-proto/src/proto/messages.js	Encoders/decoders for RPC Message variants.
packages/capn-proto/src/wire/segment.js	Segment arena + message builder/reader.
packages/capn-proto/src/wire/framing.js	Stream framing encode/decode for segments.
packages/capn-proto/src/wire/pointer.js	Pointer encode/decode + far pointer resolution.
packages/capn-proto/src/wire/struct.js	Struct read/write helpers and bounds checks.
packages/capn-proto/src/wire/list.js	List allocation and composite list support.
packages/capn-proto/src/wire/text.js	Text/Data helpers on top of list-of-byte encoding.
packages/capn-proto/test/basic.test.js	Bootstrap + basic call/exception tests.
packages/capn-proto/test/pipelining.test.js	Promise pipelining behavior tests.
packages/capn-proto/test/promise-resolve.test.js	Resolve message behavior for promised results.
packages/capn-proto/test/crosstalk.test.js	Concurrent bidirectional calls stress tests.
packages/capn-proto/test/identity.test.js	Presence identity preservation tests.
packages/capn-proto/test/gc.test.js	Finish/Release lifecycle + stats assertions.
packages/capn-proto/test/abort.test.js	Abort behavior for outstanding questions.
packages/capn-proto/test/disembargo.test.js	Disembargo encoding + embargo tracker tests.
packages/capn-proto/test/three-party.test.js	Provide/Accept/Disembargo encode/decode tests.
packages/capn-proto/test/tribble-4way.test.js	4-vat Tribble ordering invariant test.
packages/capn-proto/test/loopback.test.js	Loopback helper behavior tests.
packages/capn-proto/test/interfaces.test.js	InterfaceRegistry behavior tests.
packages/capn-proto/test/payload-codec.test.js	Payload codec round-trip tests for types/caps.
packages/capn-proto/test/trap.test.js	Trap export/constant sanity tests.
packages/capn-proto/test/proto/messages.test.js	Message codec round-trip tests.
packages/capn-proto/test/wire/pointer.test.js	Pointer read/write round-trip tests.
packages/capn-proto/test/wire/framing.test.js	Framing encode/decode tests.
packages/capn-proto/test/interop.test.js	Interop tests vs `capnp` CLI encode/decode.
.github/workflows/capn-proto-interop.yml	CI workflow running interop tests with capnproto installed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

This PR adds a new workspace package, @endo/capn-proto, providing a pure-JavaScript Cap’n Proto RPC implementation on top of @endo/eventual-send, including wire-format codecs, connection state machine, and a substantial test suite plus an interop CI workflow.

Changes:

Introduces @endo/capn-proto package with RPC connection logic (four tables), dispatch/handler integration, and Level 3 (three-party) scaffolding.
Implements Cap’n Proto wire primitives (segments, pointers, structs/lists, framing, packed encoding) and rpc.capnp message codecs.
Adds extensive unit/integration/interop tests and a GitHub Actions workflow to validate byte-level compatibility with the capnp CLI.

Reviewed changes

Copilot reviewed 55 out of 56 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
yarn.lock	Adds workspace lock entry for `@endo/capn-proto`.
.github/workflows/capn-proto-interop.yml	CI job installing `capnp` to run interop tests.
packages/capn-proto/package.json	New package manifest, exports, scripts, deps.
packages/capn-proto/README.md	Package documentation + quick start usage.
packages/capn-proto/tsconfig.json	Package TS config for lint/type checking.
packages/capn-proto/tsconfig.build.json	Build TS config.
packages/capn-proto/rpc.capnp	Cap’n Proto RPC schema source (added).
packages/capn-proto/rpc-twoparty.capnp	Two-party schema file (added).
packages/capn-proto/src/index.js	Public entrypoint exports (API surface).
packages/capn-proto/src/rpc-system.js	Top-level factory wrapping a single connection.
packages/capn-proto/src/connection.js	Core connection state, send/receive, tables, codec integration.
packages/capn-proto/src/dispatch.js	Inbound message handlers (Call/Return/etc.).
packages/capn-proto/src/handler.js	`HandledPromise` handler that turns E() into Call messages.
packages/capn-proto/src/interfaces.js	Interface registry mapping method names ↔ ordinals.
packages/capn-proto/src/payload-codec.js	JSON+capTable payload encoding/decoding.
packages/capn-proto/src/loopback.js	Test/helper transport wiring for two endpoints.
packages/capn-proto/src/two-party.js	Two-party VatNetwork implementation.
packages/capn-proto/src/trap.js	Trap wrappers reusing captp atomics transport.
packages/capn-proto/src/three-party.js	Level-3 three-party handoff logic (Provide/Accept/Disembargo).
packages/capn-proto/src/embargo.js	Disembargo bookkeeping/tracker.
packages/capn-proto/src/exports.js	Export table identity/refcount management.
packages/capn-proto/src/imports.js	Import table presence creation + weak tracking.
packages/capn-proto/src/finalize.js	Weak-value map w/ FinalizationRegistry support.
packages/capn-proto/src/tables/four-tables.js	Per-connection question/answer/import/export tables.
packages/capn-proto/src/tables/id-allocator.js	32-bit ID allocator with reuse.
packages/capn-proto/src/proto/schema.js	rpc.capnp layout constants for codecs.
packages/capn-proto/src/proto/messages.js	Encoders/decoders for rpc.capnp message variants.
packages/capn-proto/src/wire/segment.js	Segment arena + allocator.
packages/capn-proto/src/wire/pointer.js	Pointer encoding/decoding + far resolution.
packages/capn-proto/src/wire/struct.js	Struct read/write helpers.
packages/capn-proto/src/wire/list.js	List allocation + decoding helpers.
packages/capn-proto/src/wire/text.js	Text/Data helpers on top of List(Byte).
packages/capn-proto/src/wire/framing.js	Stream framing/unframing for segments.
packages/capn-proto/src/wire/packed.js	Packed encoding/decoding implementation.
packages/capn-proto/test/basic.test.js	Basic bootstrap + call + exception tests.
packages/capn-proto/test/pipelining.test.js	Promise pipelining behavior tests.
packages/capn-proto/test/promise-resolve.test.js	Resolve-message behavior for returned promises.
packages/capn-proto/test/crosstalk.test.js	Concurrent bidirectional call tests.
packages/capn-proto/test/identity.test.js	Presence/reference identity preservation tests.
packages/capn-proto/test/gc.test.js	Finish/answer lifecycle tests.
packages/capn-proto/test/gc-finalize.test.js	FinalizationRegistry-driven Release tests.
packages/capn-proto/test/engine-gc.js	Helper to obtain an engine GC function in tests.
packages/capn-proto/test/disembargo.test.js	Disembargo tracking + echo behavior tests.
packages/capn-proto/test/three-party.test.js	Provide/Accept/Disembargo codec + host-side L3 flow tests.
packages/capn-proto/test/tribble-4way.test.js	Tribble routing/E-order invariants tests.
packages/capn-proto/test/integration.test.js	End-to-end scenarios modeled after upstream rpc tests.
packages/capn-proto/test/interfaces.test.js	Interface registry behavior tests.
packages/capn-proto/test/payload-codec.test.js	Payload codec round-trip tests.
packages/capn-proto/test/loopback.test.js	Loopback helper behavior tests.
packages/capn-proto/test/trap.test.js	Trap export/constants tests.
packages/capn-proto/test/interop.test.js	`capnp` CLI byte-level interoperability tests.
packages/capn-proto/test/proto/messages.test.js	Message codec round-trip tests.
packages/capn-proto/test/wire/pointer.test.js	Pointer codec round-trip tests.
packages/capn-proto/test/wire/framing.test.js	Framing round-trip tests.
packages/capn-proto/test/wire/packed.test.js	Packed encoding unit tests.
packages/capn-proto/test/abort.test.js	Abort behavior tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kumavis · 2026-05-04T18:57:19Z

We want to identify differences with ocapn:

the bijection of types expressible on the wire. For example Dates. is there a tagged types equivalent?
brand equality

kumavis · 2026-05-06T03:05:41Z

Update — supersedes the earlier comment. The JSON-payload codec was removed in 783ecbf (so the bijection is now schema-typed-only). Re-stating the deltas with that column dropped:

Wire-type bijection

Type	capn-proto schema-typed codec	ocapn pass-style
`null`, `undefined`	not at value level (`Void` only)	first-class
`boolean`	`Bool`	first-class
`number`	`Float64` etc.	first-class
`bigint`	`Int64`/`UInt64` (fixed-width, no arbitrary precision)	first-class
`string`	`Text`	first-class
`Uint8Array`	`Data`	first-class
`symbol`	✗	first-class (registered/well-known)
`Date`	✗	✗ (no built-in tagged type for Date)
`Map`, `Set`	✗	✗
`RegExp`, `URL`	✗	✗
`Error` (as data)	✗ (errors are control-flow on the wire, not values)	first-class (`desc:error`)
Plain `Array`	`List(T)` (homogeneous; `List(AnyPointer)` not yet supported)	`copyArray`
Plain `Object`	nested struct from the schema	`copyRecord`
Capability	dedicated cap pointer	`desc:remote-object`
Promise	n/a (settles to one of the others)	`desc:remote-promise`
Sturdy ref	✗	`desc:sturdyref` (location + swissnum)
Signed handoff	✗	`desc:handoff-give` / `desc:handoff-receive` (PublicKey + Signature)

Tagged-type extension points. Cap'n Proto's wire format is structural — the schema language has no tagged type. ocapn's pass-style (via @endo/marshal's makeTagged + desc:tagged) is the protocol-emergence hook for sturdy refs, errors, handoff envelopes, and any user-defined extension. capn-proto users that need "Date inside a struct field" today have to stash it as Int64 (millis) or Text (ISO string) by convention, and there's no place in the wire format to add a desc:tagged equivalent without a schema change.

Dates specifically. No built-in support in either package today. Closest path forward in capn-proto is a documented schema convention (e.g. an instant @0 :Int64 field interpreted as Unix millis); a tagged-record extension would require an @endo/marshal-style higher-level codec on top.

Brand equality

Aspect	capn-proto	ocapn
Same Presence imported again on same connection	identical (`importIdToPresence` weak-value map, `src/imports.js`)	identical (FinalizingMap, `pairwise.js`)
Same JS object exported again on same connection	identical (`valToExportId` WeakMap, `src/exports.js`)	identical (`valueToSlot` WeakMap, `pairwise.js`)
Same Presence across different connections	distinct ids; `cap-home-registry.js` records origin so encoder doesn't mis-route, but no equality across vats	grant-tracker / sturdy refs let two sessions reconcile the same underlying object
Lifetime	refCount → 0 ⇒ entry dropped, peer expected to send `Release`; `FinalizationRegistry` fires on Presence GC	refCount → 0 OR FinalizingMap GC hook (auto on unreachable)
Persistent identity (across reconnect / restart)	none — connection-local only	sturdy refs (`src/client/sturdyrefs.js`): tagged value `ocapn-sturdyref` carrying location + swissnum; recoverable via `E(remoteBootstrap).fetch(swissNum)`
Crypto-backed brand proofs	none	`desc:handoff-give` / `desc:handoff-receive` carry `PublicKey` + `Signature`; receiver can verify the introducer custody
Promise identity	per-question-id (single questioner connection)	settler-multiplexed: multiple peers can settle on the same remote-promise slot

Bottom line. Within a single connection the two packages have the same brand-equality discipline (WeakMap-keyed reference equality preserved while at least one side holds the value). The big delta is that ocapn has crypto-backed persistent identity (sturdy refs + signed handoff envelopes); capn-proto has none. That's intentional: Cap'n Proto's spec puts persistence at L2 (sturdy refs, not yet implemented here) and signed introductions are not part of the spec at all. Anything beyond connection-lifetime identity is currently out of scope for this package.

A broader protocol-level comparison (architecture, wire format, lifecycle, GC, error model, handoff, etc.) is now in the package README under ## Comparison with @endo/ocapn.

Generated by Claude Code

Repo-wide prettier formatting on every capn-proto source and test file (restores `lint:prettier` CI gate). Also addresses the Copilot review on PR #3213: - src/proto/messages.js: consolidate imports at top of file (the earlier diff interleaved them with helper declarations, which would trip `import/first`); drop unused `readPtrAt`, `primitiveElementByteOffset`, `LIST_BYTE` imports and their `void` suppressions. - src/dispatch.js, src/three-party.js: drop unused `Fail` imports and trailing `void Fail;` markers. - src/dispatch.js: handleReturn now also `delete`s the QuestionEntry and releases the questionId after sending Finish, so the allocator can recycle ids instead of growing monotonically. - src/wire/struct.js: writeUint16 / writeUint32 / writeUint64 / writeBool now bounds-check against the struct's data section and Fail loudly instead of writing past the boundary. - src/wire/pointer.js: writePointer now validates each sub-field against its representable range (signed 30-bit offset, uint16 data/ptr words, uint29 elemCount, uint32 segmentId / cap index) and Fails on out-of-range input instead of silently masking. - src/wire/segment.js: new MessageBuilder.allocateInSegment(segId, words) primitive that allocates within a specific segment, growing it if needed. Used by allocStruct / allocList / allocCompositeList to place far-pointer landing pads next to their payloads. The previous code called `allocate(2)` and asserted segId match, which would Fail spuriously if an earlier segment had spare room. - src/three-party.js: handleDisembargoAccept tracks the real Provide questionId via a target-id → questionId map populated on initiateProvide, replacing the hard-coded `questionId: 0`. - README.md: replace the inaccurate Quick start (which referenced a non-existent `network` argument and a `getBootstrap(remoteVatId)` signature) with one that matches the actual `makeCapnp({ send, bootstrap })` API. - test/tribble-4way.test.js: rewrite to actually exercise four independent makeCapnp instances chained A→B→C→D and assert E-order at Bob, removing the dead `wireUp`/`inboxes`/`schedule` scaffolding the previous version had after collapsing to a 2-vat shortcut. All 78 tests pass locally. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

Copilot

Pull request overview

Copilot reviewed 78 out of 79 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (3)

packages/capn-proto/src/wire/packed.js:1

Preallocating u8.length * 10 can create very large transient allocations (and GC pressure) for big messages even though the packed output is typically near or below the unpacked size. Consider switching to a growth strategy (chunked append / doubling buffer) or a two-pass approach (like unpack’s measureUnpackedLength) to compute a tighter bound before allocating.
packages/capn-proto/src/wire/streaming.js:1
The streaming framed-message parser is new core behavior (used by TCP) but there isn’t a dedicated unit test shown here that deterministically covers chunk-splitting edge cases (partial header, header split across chunks, multiple messages in one chunk, truncated payload error paths, and the >512 segments rejection). Adding a focused test file for makeFramedStreamParser would make these behaviors stable and avoid relying on nondeterministic TCP chunking in integration tests.
packages/capn-proto/src/wire/streaming.js:1
The streaming framed-message parser is new core behavior (used by TCP) but there isn’t a dedicated unit test shown here that deterministically covers chunk-splitting edge cases (partial header, header split across chunks, multiple messages in one chunk, truncated payload error paths, and the >512 segments rejection). Adding a focused test file for makeFramedStreamParser would make these behaviors stable and avoid relying on nondeterministic TCP chunking in integration tests.

Implements the Cap'n Proto RPC protocol in pure JS using HandledPromise / Presence semantics from @endo/eventual-send and FinalizationRegistry-based GC, structured similarly to @endo/captp and @endo/ocapn: - Real Cap'n Proto binary wire format: segments, struct/list/far/cap pointers, primitive and composite lists, Text/Data, stream framing. - Hand-written codecs for the rpc.capnp Message union (all 14 ordinals including obsoleteSave/obsoleteDelete/join echoed as Unimplemented), with field ordinals matching the upstream schema. - Four-table state machine (Questions / Answers / Exports / Imports) per peer, with 32-bit unsigned IDs and free-list reuse. - HandledPromise handler that uses the framework's `returnedP` argument to fold the user-facing promise into our pipeline handler for downstream E() calls. - Per-interface ordinal map registry: methods are addressed by (interfaceId :UInt64, methodId :UInt16) with explicit registration via registerInterface({ id, methods }); no name-derived heuristic. - Identity preservation: WeakMap from local value to ExportId; weak FinalizingMap from ImportId to Presence; pass-back encoded as receiverHosted. - Promise pipelining via PromisedAnswer { questionId, transform } and the Tribble resolution rule (calls to P stay routed via R after P resolves), plus a 2-hop sender/receiver loopback Disembargo path. - Three-party handoff (Level 3) scaffold: Provide/Accept/vines and Disembargo accept/provide variants, with VatNetwork interface and default two-party network. - Trap mechanism reused from @endo/captp: SharedArrayBuffer + Atomics framing wrapping framed Cap'n Proto bytes. Tests cover the wire format, every Message variant round-trip, the interface registry, the payload codec, basic calls and exceptions, identity round-trip, promise pipelining, promise resolution, GC, abort, crosstalk, embargo bookkeeping, three-party encoding, the Trap re-export, and the Tribble 4-way race ordering invariant. 57 tests pass. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

@1

Verifies @endo/capn-proto wire format matches the upstream C++ Cap'n Proto 1.0.1 reference implementation byte-for-byte, in both directions. Schema fixes uncovered by the round-trip: - Call.methodId @3 packs into the 4-byte hole at byte 4 (was 16); Call.sendResultsTo discriminator is at byte 6 (was 18); Call.allowThirdPartyTailCall is at bit 128 = byte 16 bit 0 (was bit 32). - Disembargo.context group: discriminator at byte 4, value at byte 0 (had them reversed). - Return.releaseParamCaps, Finish.releaseResultCaps, and Finish.requireEarlyCancellationWorkaround all default to true and are stored XOR'd against 1; added writeBoolDefaultTrue / readBoolDefaultTrue helpers and applied them. - Message.abort @1 :Exception means the variant *is* an Exception, not a wrapper struct holding one; encodeAbort now writes the Exception directly into the Message's pointer slot. Same for Message.unimplemented @0 :Message (we encode an empty inner Message). Test/interop.test.js spawns the system `capnp` CLI: - For each Message variant we encode bytes and pipe them through `capnp decode rpc.capnp Message`, asserting on field values in the reference text output. - For each Message variant we feed text to `capnp encode rpc.capnp Message` and verify our decoder reads the resulting bytes correctly. The test gracefully skips when `capnp` is not installed. Adds .github/workflows/capn-proto-interop.yml that installs the `capnproto` Debian package and runs the interop test on every PR and master push, alongside the existing unit tests. The workflow runs on ubuntu-latest because the apt package is the simplest cross-CI way to get a stable reference Cap'n Proto C++ binary; the same test will also run anywhere capnp is on PATH. Vendors rpc.capnp and rpc-twoparty.capnp from the upstream Cap'n Proto project (MIT-licensed) so the interop test uses the canonical schema. All 78 tests pass locally (57 existing + 21 interop). https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

CI's `yarn install --immutable` step rejected the lockfile because the `@endo/capn-proto` workspace package was not registered. This caused nearly every job (lint, test, cover, viable-release, browser-tests, test-hermes, test-async-hooks, test-ocapn-python, check-action-pins, test262, test-xs, capn-proto interop) to fail in the install step. Adds the corresponding `@endo/capn-proto@workspace:packages/capn-proto` entry to yarn.lock, mirroring the structure of `@endo/captp`. Also adds the missing `@endo/captp` dependency declaration in capn-proto's package.json (it was already imported by src/trap.js). https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

Repo-wide prettier formatting on every capn-proto source and test file (restores `lint:prettier` CI gate). Also addresses the Copilot review on PR #3213: - src/proto/messages.js: consolidate imports at top of file (the earlier diff interleaved them with helper declarations, which would trip `import/first`); drop unused `readPtrAt`, `primitiveElementByteOffset`, `LIST_BYTE` imports and their `void` suppressions. - src/dispatch.js, src/three-party.js: drop unused `Fail` imports and trailing `void Fail;` markers. - src/dispatch.js: handleReturn now also `delete`s the QuestionEntry and releases the questionId after sending Finish, so the allocator can recycle ids instead of growing monotonically. - src/wire/struct.js: writeUint16 / writeUint32 / writeUint64 / writeBool now bounds-check against the struct's data section and Fail loudly instead of writing past the boundary. - src/wire/pointer.js: writePointer now validates each sub-field against its representable range (signed 30-bit offset, uint16 data/ptr words, uint29 elemCount, uint32 segmentId / cap index) and Fails on out-of-range input instead of silently masking. - src/wire/segment.js: new MessageBuilder.allocateInSegment(segId, words) primitive that allocates within a specific segment, growing it if needed. Used by allocStruct / allocList / allocCompositeList to place far-pointer landing pads next to their payloads. The previous code called `allocate(2)` and asserted segId match, which would Fail spuriously if an earlier segment had spare room. - src/three-party.js: handleDisembargoAccept tracks the real Provide questionId via a target-id → questionId map populated on initiateProvide, replacing the hard-coded `questionId: 0`. - README.md: replace the inaccurate Quick start (which referenced a non-existent `network` argument and a `getBootstrap(remoteVatId)` signature) with one that matches the actual `makeCapnp({ send, bootstrap })` API. - test/tribble-4way.test.js: rewrite to actually exercise four independent makeCapnp instances chained A→B→C→D and assert E-order at Bob, removing the dead `wireUp`/`inboxes`/`schedule` scaffolding the previous version had after collapsing to a 2-vat shortcut. All 78 tests pass locally. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

ESLint reported 10 errors on the previous commit; this addresses each: - src/dispatch.js: dropped `encodeAbort` and `encodeRelease` from the destructure since they are not used by any handler in this file (real callers are in connection.js). - src/connection.js: `bootstrap` is now `const` (we mutate `.value`, not the binding). - src/connection.js: removed the `id_` dangling-underscore variable and the dead `byIdEntries?.()` scan path. Method resolution now goes solely through `findMethodAcrossInterfaces`, which uses the registry's documented `iterate()` API. - src/connection.js: hoisted `findMethodAcrossInterfaces` to come before `makeRemoteHandlerForImport`, fixing the use-before-define. - src/connection.js: hoisted `describeForResolve` before `payloadCodec`. The `importRegistry` reference inside `payloadCodec` is now a forward-declared `let` whose value is assigned where the registry is constructed, again removing use-before-define hazards while preserving the existing initialisation order. - src/wire/list.js: corrected the `compositeElement` JSDoc parameter ordering (`@param {any} msg` was listed after `listLoc` but the actual signature is `(msg, listLoc, idx)`). All 78 tests still pass locally. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

The handler's applyMethod was wiring returnedP via two parallel paths: `HandledPromise.resolve(returnedP, pipelineP)` plus `answerPromise.then(... e => HandledPromise.resolve(returnedP, rej))`. Under SES with `LOCKDOWN_HARDEN_TAMING=unsafe` (the `unsafe` ses-ava config), an intermediate promise in that chain surfaced as an unhandled rejection — visible only because that mode reports them synchronously rather than waiting for FinalizationRegistry. Replaced the dual-resolution dance with the simpler CapTP-style pattern: build a single HandledPromise (`pipelineP`) seeded with our pipelineHandler, capture its `resolve` / `reject` from the executor, and settle it from `answerPromise`. The framework shortens `returnedP` onto whatever the handler returns, so the user-visible promise and the pipeline target are now the same object. The intermediate .then-chain that fanned out the rejection to a second copy of `returnedP` is gone, and a single defensive `.catch(() => {})` on `pipelineP` silences any leak before the framework's user-side shortening attaches a real handler. Verified by running ses-ava across all three configs (lockdown, unsafe, endo): 78 tests pass cleanly in each — 234 total — with no unhandled rejection diagnostics. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

ESLint reported 10 errors of the `no-void` form ("Expected 'undefined' and instead saw 'void'") at the `void X;` no-op suppressions I had been using to silence unused-variable lints. None of them needed suppression in the first place: the underscore-prefixed parameter convention (and just removing the side-effect statements where there were no other uses) is enough. - src/handler.js: removed every `void X` statement; renamed `args` / `returnedP` to `_args` / `_returnedP` in the function bodies that intentionally don't use them (`get`, `applyFunction`, `applyFunctionSendOnly`). - src/imports.js: replaced `void p; presence = captured;` with an explicit "we only need the side effect" comment plus `eslint-disable-next-line no-new` on the `new HandledPromise(...)` expression that we throw away. - src/proto/messages.js: removed the `void originalBytes` in encodeUnimplemented; the parameter is now `_arg` with a JSDoc note that originalBytes is intentionally ignored on the wire. Also addressed the 11 JSDoc warnings reported alongside the errors: - src/wire/struct.js: added `@param` declarations for byteIdx/value on writeUint8/16/32/64 and bitIdx/value on writeBool. - src/three-party.js: added `@param {{ ... }} msg` blocks on handleProvide and handleAccept. - src/proto/messages.js: added `@param` blocks on writeBoolDefaultTrue / readBoolDefaultTrue. - src/imports.js: added `@param {unknown} value` on importIdOf. - src/embargo.js: added `@param {() => void} onEcho` on open and `@param {number} id` on echo. Verified across all three ses-ava configs (lockdown, unsafe, endo): 234 tests pass cleanly with no unhandled rejections. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

CI's lint job reported 10 more errors after the previous round: - src/proto/messages.js: 5 `restrict-comparison-operands` errors at the `op.op === 'noop'` etc. comparisons in writePromisedAnswer and writePayload. Annotated `pa` and `payload` parameters with explicit JSDoc shapes and coerced `op.op` via `String(...)` so the rule no longer treats them as `unknown`. - src/pipeline.js: `no-continue` at line 33. Inverted the loop to drop the `continue`; 'noop' and unrecognised ops are now silently skipped by branching on the recognised op only. - src/payload-codec.js: 2 `no-undef` errors on `Buffer` references and 1 comparison-against-unknown. Added `/* global Buffer, atob, btoa */` and lifted `typeof Buffer !== 'undefined'` into a module constant `hasBuffer` so the lint rule sees a typed boolean. - src/loopback.js: `prefer-const` at line 45 on `let near; near = ...`. Restructured the mutual-reference setup with a small `farRef` indirection object; `near` and `far` are now both `const`, with `farRef.ref` patched once `far` exists. Also added remaining `@param {number} byteIdx` JSDoc on `readUint32` and `readUint64` in src/wire/struct.js (warnings). 234 tests pass cleanly across all three ses-ava configs. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

Round 3 of CI lint fixes for the 10 errors that remained: - src/wire/pointer.js:179-180: replaced `1 << 29` with `2 ** 29` to avoid `no-bitwise` on the SIGNED_30_MIN/MAX constants. Added JSDoc `@param` blocks to the check helpers (checkSignedOffset30, checkUint16, checkUint29, checkUint32) so their `value` parameter isn't typed as `unknown` when compared. - src/trap.js: added `/* global Buffer, atob, btoa */` and lifted `typeof Buffer !== 'undefined'` into a module-level `hasBuffer` constant so the comparison rule sees a typed boolean and the globals lookup is satisfied. Removed now-redundant `eslint-disable-next-line no-undef` comments. - src/payload-codec.js: added `@param` JSDoc to `toBase64(bytes)` and `fromBase64(s)` so their parameter types aren't `unknown`. - src/three-party.js: dropped unused destructured fields `importRegistry` and `findOrCreatePeerConnection` from the ctx destructure (they were leftovers from earlier scaffolding). - src/tables/four-tables.js: renamed local variable `exports_` to `exportsMap` to satisfy `no-underscore-dangle`. The public field name returned to callers is still `exports`. Also removed the unused `promiseToExportId` map that was kept for diagnostics. 234 tests pass cleanly across all three ses-ava configs. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

@PARAM

* connection.js: add `// eslint-disable-next-line no-use-before-define` on the `exportCap` call inside `describeForResolve`. The closure only runs once a senderPromise's resolver fires, by which point exportCap is fully initialised — but the lint rule fires statically on the forward reference. * embargo-trigger.test.js: hoist a no-op `await null` before the for-loop in `drainTicks` so the @jessie.js/no-nested-await rule sees an unnested top-level await as the function's FIRST await. Also added a missing @PARAM JSDoc for `sniffSenderPromiseId`.

…ion) Closes the L1 embargo loop: when a senderPromise resolves to a value reachable via a SHORTER path than the original (receiverHosted — the resolved cap is one of OUR exports), Cap'n Proto requires the recipient to emit `Disembargo { senderLoopback }` on the original path and gate the user-facing settle on the matching `receiverLoopback` echo. Without it, pipelined Calls already in flight at the peer (about to bounce back to our export) can be overtaken by post-resolve direct- local invocations on the now-settled Presence — out-of-order delivery. Before this change, the Disembargo emission half of the embargo mechanism was 50% wired: we could ECHO an incoming senderLoopback as receiverLoopback (existing path), and we could RECEIVE a receiverLoopback echo (firing `embargoTracker.echo` — also existing), but no code path ever called `embargoTracker.open()` to actually INITIATE an embargo on resolution. The `.open()` method is wired now. Three complementary cases, all matching capnp's rpc-test.c++ scenarios: • `receiverHosted` resolution + pipelined calls in flight → emit senderLoopback Disembargo, settle the user-facing presence only after the receiverLoopback echo arrives. • `receiverHosted` resolution + NO pipelined calls → no embargo (no race to guard against — local-direct after the resolve is safe because nothing was queued through B). • `senderHosted` / `senderPromise` resolution → no embargo even with pipelined calls in flight, because both routes still go through the same peer and the peer's in-order processing preserves order on its own. The L3 / `thirdPartyHosted` branch is unchanged (it has its own embargo handshake on the A↔C connection, landed previously). Test coverage * test/embargo-l1.test.js: three new tests covering the three cases above. Each uses a real A↔B loopback so the senderPromise emission runs for real, then hand-dispatches the Resolve to control the descriptor kind. 138 → 141 tests pass under `ava --config ../../ava-endo-lockdown.config.mjs`.

Documents the JS values that round-trip through each of the two encode paths (the JSON-payload codec and the schema-typed codec), with a side-by-side reference to the OCapN data model spec. Covers atoms (incl. the Symbol / Date / Map / Set gaps), containers (incl. the absence of a tagged-type extension point), reference values (senderHosted / senderPromise / receiverHosted / receiverAnswer / thirdPartyHosted vs OCapN's `desc:remote-object` / `remote-promise` / `sturdyref` / `handoff-give` / `handoff-receive`), error semantics (control-flow on the wire here, copy-data in OCapN), and identity / equality (same WeakMap discipline as OCapN within a connection; no sturdy refs or signed handoffs). Closes the comparison kumavis asked for in the PR review thread.

@cap

Drop the JSON-payload codec. Method params and results are now encoded as Cap'n Proto structs directly at the Payload.content AnyPointer slot — byte-compatible with capnp-C++/CF clients running the same .capnp schema. Bootstrap returns now place a cap pointer at the AnyPointer slot, matching capnp-C++'s wire output for a single-cap bootstrap. Required because the previous wrapping (content as a Data list of JSON-with-marker bytes) was not interpretable by any non-this-package client. A CF client doing `params.getContent().getAs<MyParams>()` would have failed because content was a List(UInt8), not a struct. Surface changes: • new schema/codec.js: encodeStructInto / decodeStructFrom write or read a struct directly at a caller-chosen pointer slot in an in-progress message. encodeRootStruct / decodeRootStruct become thin wrappers. • new proto/messages.js: writePayload takes an `encodeContent` callback that places any pointer kind (struct, cap, list, …) at Payload.content; readPayload returns a pointer slot the caller resolves with their schema. encodeCapContent / readCapContent helpers handle the bootstrap/Provide cap-only shape. • new MethodCodec contract: encode → { encodeContent, capTable }; decode ← { contentSlot, capTable }. registerInterface auto-derives these from the schema's Params/Results structs. • sendCall + handleCall hard-error when no methodCodec is registered for the called method; no JSON fallback. • src/payload-codec.js deleted; @cap:/@bigint:/@bytes:/@promise: markers removed. Test migration: • proto/messages.test.js + interop.test.js use the new encodeContent callback (with writeData) for proto-level round-trips against the real capnp CLI. • schema-rpc.test.js uses the new schema.structCodec(name) entrypoint in place of the removed encodePayload/decodePayload. • test/fixtures/json-codec.js — test-only generic codec (Data-at- AnyPointer-slot with JSON inside). Lets protocol-level tests (embargo, gc, lifecycle, three-party) keep their existing JS-shape args/results without designing per-test schemas. NOT CF-interop; not exported from src/. • ~14 tests register their interfaces via withJsonCodecs(...) instead of the bare { id, methods } literal.

…moval `MethodCodec` typedef was duplicated across types.js and interfaces.js and one branch still referenced the deleted `Payload` type — tsc flagged the dangling reference. Consolidate: types.js holds the canonical typedef (now using EncodedPayload / DecodedPayload), interfaces.js re-imports it. Drop a few stale "JSON-over-bytes payload codec" / "payload codec" mentions in module docstrings.

@0

Two errors flagged by `yarn lint:types`: 1. proto/messages.js writePayload doc-comment quoted the rpc.capnp grammar — `content @0 :AnyPointer` — and tsc parsed `@0` / `@1` as JSDoc tags (TS1003 'Identifier expected'). Reword to plain prose so tsc doesn't try to interpret them. 2. test/fixtures/json-codec.js#jsonCodecsFor returned `{}` (empty object literal type), which TS narrowed and refused to widen to the InterfaceDescriptor.methodCodecs Record. Added an explicit @type annotation.

Adds a "Comparison with @endo/ocapn" section to the README that goes beyond the value-bijection table in docs/serialization-model.md. Covers origin/lineage, wire format, protocol layering (Levels vs operations stack), capability descriptor mapping, identity / brand equality / persistence (the sturdy-ref + signed-handoff gap), promise pipelining, three-party handoff, error model, garbage collection, and transport. Closes with a "when to pick which" guide.

ESLint's @endo/recommended config doesn't expose `atob` / `btoa` as globals (the SES whitelist). Other call sites in the package (`src/trap.js:2`) use a `/* global atob, btoa */` directive; do the same in the new test fixture.

`yarn lint:prettier` (run from the monorepo root, separate from each workspace's lint:eslint + lint:types) flagged four files modified during the Payload-AnyPointer refactor: dispatch.js, promise-resolve test, proto/messages test, schema-rpc test. Re-run prettier on them.

ESLint's `import/no-duplicates` flagged proto/messages.js because the new `WORD_SIZE` import (added during the AnyPointer plumbing) lived in its own line below the existing `wire/segment.js` import. Merge them into a single import.

@cap

…cstring Two cosmetic JSDoc fixes flagged by `yarn lint:eslint` (jsdoc plugin): 1. types.js had `EncodedPayload` and `DecodedPayload` declared in a single comment block. Some JSDoc tooling mis-attributes the second typedef's properties to the first. Split into separate blocks. 2. test/fixtures/json-codec.js had `{"@cap": N}` etc. in the module docstring outside of backticks; the @-prefixed tokens risk being parsed as JSDoc tags. Drop the inline examples and point at the source-of-truth functions instead.

`@endo/restrict-comparison-operands` fired on `i < u8.length` because the `u8` parameter (and `b` in the inverse) had no JSDoc type, so the rule classified the comparison as "against unknown type". The file itself is `// @ts-nocheck` (test fixture), but the eslint rule still inspects JSDoc — add `@param` annotations so both helpers' loop bounds are typed.

Adds a node:net-based transport (`connectTcp` / `serveTcp`) and a streaming framed-message parser, plus end-to-end RPC interop tests against the upstream Cap'n Proto C++ implementation's `EzRpcServer`. Until now `@endo/capn-proto` had two transports: `makeLoopback` (in-process) and `makeTwoPartyVatNetwork` (the abstraction layer without a concrete byte-stream binding). Neither talks to anything outside the package. This commit lands the standard transport every other Cap'n Proto implementation supports — segment-table framed messages over TCP — so a Node peer can speak to a C++/Rust/Python peer running the same `.capnp` schema. Surface: • src/wire/streaming.js — `makeFramedStreamParser({ onMessage })`. Push raw socket chunks; emits complete framed messages. Handles arbitrary chunking (Cap'n Proto's segment-table header tells us how much to wait for). • src/transport/tcp.js — `connectTcp({ host, port, ... })` and `serveTcp({ host, port, bootstrap, ... })`. Each returns a `makeCapnp` instance bound to the socket, plus a `close()` helper. Records are deliberately unhardened: net.Server / net.Socket have mutable internal fields that harden() would freeze. • test/transport-tcp.test.js — node-to-node round-trip over loopback TCP, including a 50-call pipelined burst to exercise wire chunking. Interop fixture: • test/interop-rpc/echo.capnp — minimal Echo schema (ping + count). • test/interop-rpc/echo-server.c++ — capnp::EzRpcServer wrapping a handwritten `Echo::Server` impl. Modeled on the upstream calculator-server sample. • test/interop-rpc/build.sh — idempotent compile script (capnp schema → C++ + g++ link). Skips when up to date. • test/interop-rpc.test.js — spawns the C++ server, connects via `connectTcp`, and round-trips both methods plus a 32-call burst. Auto-skips when `capnp` or the built binary is absent. CI: • .github/workflows/capn-proto-interop.yml — adds a step that runs the build script before tests, plus an explicit invocation of test/interop-rpc.test.js so the live RPC interop result is surfaced separately from the byte-level interop. Smoke-tested locally: Node ping → C++ EzRpcServer → Node Return round-trips correctly, capabilities included.

Ports four "two-client" patterns from the upstream Cap'n Proto C++ test suite (rpc-test.c++ / rpc-twoparty-test.c++) as live interop tests against the same C++ EzRpcServer fixture. Schema (test/interop-rpc/echo.capnp): - Collapses Echo + CounterFactory + Pinger into a single TestSuite bootstrap (avoids C++ multiple-inheritance dispatchCall ambiguity). - Adds a Counter interface with per-instance state. - newCounter() returns a fresh Counter cap; callBack(target, msg) has the server invoke target.ping(msg) back into the caller. Server (echo-server.c++): - Single TestSuiteImpl handles all methods. - newCounter mints a fresh CounterImpl per call. - callBack uses kj's `target.pingRequest().send().then(...)` to invoke the supplied cap; the result feeds the outer response. Tests (test/interop-rpc-multi.test.js): 1. two independent Node clients each get isolated Counter state (matches the canonical RPC isolation guarantee — each connection sees its own server-side instance). 2. 50 pipelined inc()s on a returned Counter cap, in-order — the pipelining-on-a-returned-cap pattern from `TEST(Rpc, Pipelining)`. 3. cap-as-argument: Node passes its own bootstrap (a Node-hosted TestSuite cap) to the C++ server; the server invokes target.ping(msg) on it; the reply is the Node side's own response. Matches the cap-callback half of `TEST(Rpc, RetainAndRelease)`. 4. fan-out concurrency: 2 clients × 4 counters × 5 increments, fully interleaved — exercises wire chunking, parser independence per connection, and per-cap method-ordering. CI: workflow runs the new file alongside the existing live RPC test. Smoke-tested locally — all four scenarios pass against a real capnp::EzRpcServer.

Five separate review items, all in the same commit because they touch small surfaces: handler.js (3 review threads): • get(): the previous rejection embedded q(prop) (which adds quotes) and suggested syntax like `E(p).'foo'()` which isn't valid JS. Now uses String(prop) and explains the actual constraint — Cap'n Proto methods are addressed by ordinal, so callers need a method registered in the InterfaceRegistry. • applyFunctionRejection / fail: rewrote the message to describe the real requirement ("use E(target).method(args) on a method registered via an InterfaceRegistry") instead of the cryptic "first argument of E(target).method". • applyMethodSendOnly: was calling resolveMethod synchronously without a try/catch, so an unknown-method throw escaped as an uncaught exception out of the eventual-send dispatch loop. Now catches and console.warn's, mirroring applyMethod's promise-rejection behaviour so a single bad sendOnly call no longer takes down the session. trap.js: the non-Buffer fallback built a binary string via per-byte `bin += String.fromCharCode(...)`. The comment claimed O(n) but per-byte string concatenation in V8 is O(n^2) for any meaningful length. Now accumulates into an Array and joins once; the comment is corrected. cap-home-registry.js: the CapHome typedef said `importIdOf(presence)` is called against `hostConnection.interfaceRegistry`, but importIdOf lives on the import registry / connection, not on the InterfaceRegistry. Doc-only fix; corrects the type comment so future readers don't chase the wrong API. id-allocator.js: release(id) accepted any number, allowing double-free or release-of-unallocated-id to silently corrupt the four-tables state (the same id could be returned twice from alloc, breaking question/export/embargo bookkeeping). Now validates `0 <= id < next` and tracks freeSet membership; either condition Fails with a clear message instead of producing hard-to-debug state corruption. Also addresses Copilot's "no dedicated parser unit tests" concern with a new test/streaming-parser.test.js covering: • byte-by-byte delivery • header split across chunks • multiple messages glued in one chunk • partial trailing message buffered until completed • every possible single-split point of a 3-message stream • >512 segments rejection • Buffer-style typed-array views with non-zero byteOffset Smoke-tested locally; all scenarios round-trip identical bytes.

ESLint's `no-bitwise` rule (active under @endo/strict) flagged the `i & 0xff` sentinel-byte builder in `test/streaming-parser.test.js`. The intent is "byte-modulo i", so use the un-bitwise `i % 256` form that the existing test fixtures (e.g. four-tables tests) use.

`@endo/restrict-comparison-operands` (active under @endo/internal in CI's lint:eslint) flagged the loop bounds in synthesizeFramed and bytesEqual as "Comparison against unknown type" — same issue as the earlier json-codec.js fixture: an arrow-function parameter with no JSDoc type is unknown to the rule. Added `@param` annotations so the loop comparisons are typed.

Per rpc.capnp the four L3-related AnyPointer slots — - Provide.recipient - Accept.provision - ThirdPartyCapDescriptor.id - Return.acceptFromThirdParty (the thirdPartyCapId field) — hold AnyPointer values whose schema is chosen by the VatNetwork. Until now we wrote each as a Data list (List(UInt8)), which is wire- valid AnyPointer but not struct-shaped, so a CF / capnp-cpp peer doing `params.getRecipient().getAs<TestRecipientId>()` would fail to parse. Same fix as the earlier Payload.content refactor: the encoders take `encode*(msg, slot)` callbacks instead of opaque `Uint8Array`s, and the decoders return `*Slot` pointer locations the caller resolves with whatever schema the network agreed on. Surface changes: • encodeProvide({ ..., encodeRecipient }) instead of `recipient`. • encodeAccept({ ..., encodeProvision }) instead of `provision`. • CapDescriptor `thirdPartyHosted` now carries `encodeId` / `idSlot` instead of `thirdPartyCapId`. • Return result `{ kind: 'acceptFromThirdParty', encodeId / idSlot }` instead of `thirdPartyCapId`. • VatNetwork interface: legacy thirdPartyCapIdForHost / connectToThirdParty(bytes) / provisionIdForHandoff(bytes) / acceptIncomingProvide(...,bytes) / consumeProvision(bytes) becomes encodeThirdPartyCapId(host) -> encoder / connectToThirdParty(idSlot) / encodeProvisionForHandoff(idSlot) -> encoder / encodeRecipient(recipient) -> encoder / acceptIncomingProvide(qid, target, recipientSlot) / consumeProvision(provisionSlot) • connection.js#sendAccept(provision: Uint8Array) becomes sendAccept(encodeProvision: callback). • two-party.js's mock network updated to no-op encoders. • types.js's CapDescriptor union and VatNetwork typedef updated. Test fixtures: • New test/fixtures/l3-bytes-network.js bridges the legacy Uint8Array-style mocks to the new encoder-callback API. Existing unit tests (three-party, three-party-fallback, l3-accept, auto-provide, embargo-trigger) plug into bytesNetworkMock with the same legacy field names; the fixture translates by writing the bytes as a Data list at the AnyPointer slot. Wire-valid but not schema-shaped — adequate for unit tests of the L3 control flow. The live interop test (separate commit) uses a structured schema. Validates: • All L3 unit tests now construct callbacks via bytesAsDataEncoder + decode via decodeDataFromSlot. • proto/messages.test.js + interop.test.js Provide/Accept/ thirdPartyHosted round-trip tests use the existing contentAsData helper (same pattern as Payload.content). The next commit adds the live-L3-interop schema, the C++ fixture modeled on upstream's TestNetwork, and an interop test.

…ion field After the L3 AnyPointer refactor in 0298814, decodeMessage returns `provisionSlot` (a pointer-slot location) rather than `provision` bytes. embargo-trigger.test.js's "A emitted Accept on A↔C" assertion still read `acceptMsg.provision` and was undefined-comparing against the expected swissnum. Read via `decodeDataFromSlot(acceptMsg.provisionSlot)`.

These slipped into 339d012 alongside the embargo-trigger fix; they're half-implemented support for the live-L3 interop work and trigger no-bitwise / no-nested-ternary lint errors. Will reland when the custom C++ VatNetwork fixture lands together.

Catalog-pinned prettier (3.5.3) splits the long `if (!conn) throw Error(...)` line in auto-provide.test.js across two lines. My local prettier (3.8.3) was leaving it inlined, so the CI lint caught the difference. Apply the 3.5.3 form.

The L3 wire-format refactor (encoder-callback at every AnyPointer slot) landed in 2f7da6e9 with no consumer of the new shape exercised against `capnp decode`. This commit lands: • test/interop-rpc/l3.capnp — minimal OCapN-flavored schemas for the four L3 AnyPointer payloads: VatLocation { vatId :Text, transport :Text } TestRecipientId { recipient :VatLocation } TestProvisionId { swissNum :Data } TestThirdPartyCapId { host :VatLocation, swissNum :Data } The Cap'n Proto rpc.capnp spec leaves these schemas application- defined; we pick the simplest shape that captures location + swissnum without wading into signed-handoff territory. • test/interop.test.js — three new live byte-level interop tests: 1. Provide.recipient round-trips through `capnp decode rpc.capnp Message` (parent envelope byte-correct) AND through our `loadSchema(l3).structCodec('TestRecipientId').decode(...)` of the AnyPointer slot (inner struct byte-correct). 2. Accept.provision: same recipe with TestProvisionId. 3. ThirdPartyCapDescriptor.id (inside a Resolve): same recipe with TestThirdPartyCapId. All three use loadSchema's `structCodec(name).encode(value).encodeContent` to write the AnyPointer slot directly. The capnp CLI prints the slot as `<opaque pointer>` since rpc.capnp's `:AnyPointer` doesn't carry the application schema — that's the spec-correct behaviour. Smoke-tested locally against `capnp 1.0.1`; all three scenarios round- trip identical struct values and the framed parent messages decode cleanly via the reference CLI.

@2

…-dev The L3 VatNetwork API only exists in capnproto 2.0-dev (the installed 1.0.1 marks it as `// TODO(someday)`). Adopting that API requires matching its wire format. Wire changes (per the upstream rpc.capnp at the @0xb312981b2552a250 revision shipping in 2.0-dev): Accept: 1 ptr → 2 ptrs - embargo @2 widened from Bool to Data (ThirdPartyEmbargoId byte string). Empty / null = no embargo. A non-empty id requests the host park the Return until a matching Disembargo arrives. Disembargo: 1 ptr → 2 ptrs - accept @3 widened from Void to Data (same ThirdPartyEmbargoId). - provide @4 arm removed entirely. The forwarding side (B→C) now sends the same `accept` arm with target rewritten to promisedAnswer{provideQid}. Code changes follow the wire: - encodeAccept takes `embargoId :Uint8Array` instead of `embargo :Bool`. - encodeDisembargo accept arm carries an optional embargoId. - three-party.js mints a per-vat embargoId, threads it through sendAccept + outgoing Disembargo, and on the host (C) side keys the parked Return by provideQid (verifying the id on disembargo). - dispatch.js routes accept-arm Disembargo by target shape: importedCap → B-side forward, promisedAnswer → C-side unblock. Tests updated: - proto/messages, three-party, embargo-trigger: assert embargoId bytes flow end-to-end. - interop: vendored rpc.capnp upgraded to 2.0-dev; capnp 2.0-dev's Data print format (octal escapes for non-printable bytes, printable ASCII verbatim) handled by a small `expectedDataPrinted` helper. All 157 tests pass against capnp 2.0-dev built from upstream HEAD. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

Adds ThirdPartyToContact / ThirdPartyToAwait / ThirdPartyCompletion as the names rpc.capnp 2.0-dev gives the AnyPointer slots in ThirdPartyCapDescriptor.id, Provide.recipient, and Accept.provision. Each carries a swissnum byte string the live L3 fixture's custom C++ VatNetwork uses to match Accept↔Provide. Old TestRecipientId / TestProvisionId / TestThirdPartyCapId names kept as aliases (wire-compatible — same field offsets and types) so the existing byte-level interop tests don't churn. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

…cked) zizmor's pedantic persona flagged two issues introduced by the capn-proto-interop workflow: 1. excessive-permissions (high): `actions: write` at the workflow level was speculative and unused. The job only needs the default `contents: read` for `actions/checkout`. 2. artipacked (low): `actions/checkout` was not configured with `persist-credentials: false`, leaving the GitHub token in the `.git/config` of any subsequent step or uploaded artifact. Both fixed; `zizmor --persona pedantic --min-severity low` now reports "No findings to report." against this workflow. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

The setup-node SHA (6044e13b...) was an older v6 ref than master's (48b55a01...). zizmor's online stale-action-refs audit flagged the mismatch. Pinning to the same SHA the rest of the workflows use. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

The custom L3 VatNetwork the live interop test will use needs a concrete type for VatNetwork's first template parameter (peer identity). Reuses VatLocation under a fresh struct to match the rpc.capnp convention of dedicated SturdyRefHostId types. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

…apnp 2.0-dev" This reverts commit 91cca19.

Capn Proto's L3 machinery shifted substantially between 1.0.x and 2.0-dev: the C++ VatNetwork didn't expose L3 hooks at all in 1.0.x (only `// TODO(someday)`); 2.0-dev adds them but with breaking wire- format changes to Accept.embargo (Bool → Data) and Disembargo.context.accept (Void → Data, plus the `provide` arm removed). This package targets 1.0.x so the byte-level interop tests work against the apt-installable capnp CLI in CI; a custom C++ VatNetwork live-L3 test would require both a wire-format port to 2.0-dev and a build-from-source CI step. https://claude.ai/code/session_01NAEbfGZGkgvVRm49zjLeQC

kumavis marked this pull request as draft April 27, 2026 18:06

kumavis requested a review from Copilot April 27, 2026 18:06

Copilot started reviewing on behalf of kumavis April 27, 2026 18:07 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

kumavis requested a review from Copilot April 27, 2026 20:13

Copilot started reviewing on behalf of kumavis April 27, 2026 20:14 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

Comment thread packages/capn-proto/src/handler.js Outdated

Comment thread packages/capn-proto/src/handler.js Outdated

Comment thread packages/capn-proto/src/trap.js Outdated

Comment thread packages/capn-proto/src/trap.js Outdated

Comment thread packages/capn-proto/src/payload-codec.js Outdated

kumavis requested a review from Copilot April 29, 2026 06:09

Copilot started reviewing on behalf of kumavis April 29, 2026 06:10 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

Comment thread packages/capn-proto/src/wire/packed.js Outdated

Comment thread packages/capn-proto/src/three-party.js Outdated

Comment thread packages/capn-proto/src/wire/pointer.js

This comment was marked as outdated.

Sign in to view

kumavis force-pushed the claude/capnproto-eventual-send-z8GTA branch from 045a3a9 to 4b51fd7 Compare May 6, 2026 06:12

kumavis requested a review from Copilot May 6, 2026 06:26

Copilot AI reviewed May 6, 2026

View reviewed changes

Copilot started reviewing on behalf of kumavis May 6, 2026 06:35 View session

kumavis force-pushed the claude/capnproto-eventual-send-z8GTA branch from b18c56b to 906bffa Compare May 6, 2026 09:25

claude added 9 commits May 15, 2026 00:42

claude added 25 commits May 15, 2026 00:42

fix(capn-proto): declare atob/btoa globals in test fixture

19ba591

ESLint's @endo/recommended config doesn't expose `atob` / `btoa` as globals (the SES whitelist). Other call sites in the package (`src/trap.js:2`) use a `/* global atob, btoa */` directive; do the same in the new test fixture.

kumavis force-pushed the claude/capnproto-eventual-send-z8GTA branch from 1b841d8 to 61b05bf Compare May 15, 2026 00:42

claude added 4 commits May 15, 2026 00:47

Revert "refactor(capn-proto): Accept/Disembargo wire format for rpc.c…

9b5fa39

…apnp 2.0-dev" This reverts commit 91cca19.

Uh oh!

Conversation

kumavis commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Scope

What's implemented

Interop validation

A note on L3 wire-format compatibility

CI hygiene

Test plan

Uh oh!

changeset-bot Bot commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kumavis commented May 4, 2026

Uh oh!

This comment was marked as outdated.

kumavis commented May 6, 2026

Wire-type bijection

Brand equality

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kumavis commented Apr 27, 2026 •

edited

Loading

changeset-bot Bot commented Apr 27, 2026 •

edited

Loading