diff --git a/AGENTS.md b/AGENTS.md index 7489fde51..91f2dbe22 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,8 +7,11 @@ principle are wrong by definition — the principle wins until an ADR supersedes it. In particular: **we don't bundle the app's code, and we don't guess** — the framework never bundles/transforms your code, and assembles the deploy artifact only by documented, deterministic steps (no filename/depth guessing, no tree -laundering) -([ADR-0005](docs/design/90-decisions/ADR-0005-users-build-the-framework-assembles.md)). +laundering; symlinks are preserved as links only when their resolved target +stays inside the bundle, and never dereferenced) +([ADR-0005](docs/design/90-decisions/ADR-0005-users-build-the-framework-assembles.md), +amended by +[ADR-0047](docs/design/90-decisions/ADR-0047-compute-assembly-preserves-safe-runtime-topology.md)). For design work, also check: diff --git a/architecture.config.json b/architecture.config.json index 0878263cc..5ff5b6edf 100644 --- a/architecture.config.json +++ b/architecture.config.json @@ -6,6 +6,12 @@ "layer": "foundation", "plane": "shared" }, + { + "glob": "packages/0-framework/2-authoring/bundle-paths/src/**", + "domain": "framework", + "layer": "authoring", + "plane": "control" + }, { "glob": "packages/0-framework/1-core/core/src/*.ts", "domain": "framework", @@ -908,7 +914,7 @@ { "from": "packages/1-prisma-cloud/1-extensions/target/src/control/**", "to": "packages/9-public/composer-prisma-cloud/src/exports/local-target.ts", - "reason": "ADR-0041's lazy local-target reference (operator directive; naming, operator 2026-07-23): control/extension.ts's `localTarget` field is a dynamic import of its own published local-target subpath by bare specifier, so no local-target implementation code is bundled into, or loaded by, any deploy path. The specifier resolves at a CONSUMING app's runtime, never as a real build-time dependency between these two packages — verified: dist/control.mjs keeps it as a genuine external dynamic import, never inlined (target's invariant 7 test)." + "reason": "ADR-0041's lazy local-target reference (operator directive; naming, operator 2026-07-23): control/extension.ts's `localTarget` field is a dynamic import of its own published local-target subpath by bare specifier, so no local-target implementation code is bundled into, or loaded by, any deploy path. The specifier resolves at a CONSUMING app's runtime, never as a real build-time dependency between these two packages \u2014 verified: dist/control.mjs keeps it as a genuine external dynamic import, never inlined (target's invariant 7 test)." } ], "layerOrder": { diff --git a/docs/design/01-principles/architectural-principles.md b/docs/design/01-principles/architectural-principles.md index c0eb64796..2a9da5513 100644 --- a/docs/design/01-principles/architectural-principles.md +++ b/docs/design/01-principles/architectural-principles.md @@ -46,8 +46,13 @@ copied in exactly as the Next docs prescribe). What it must **never** do is *guess* or *launder*: no filename guessing (the wrapper's name is dictated), no monorepo-depth inference (the app's location in a standalone tree is *found* by locating `server.js`, not computed), no baking absolute paths into artifacts, and -a symlinked `node_modules` is a hard error, never dereferenced. See -[ADR-0005](../90-decisions/ADR-0005-users-build-the-framework-assembles.md); +a symlink is **never** dereferenced. A symlink survives packaging as a symlink +only after assembly resolves its real target and proves that target stays inside +the bundle; a link that escapes the bundle or dangles is a hard error naming the +link. Runtime files enter the bundle only by tracing the entry the author +declared — never by discovering one. See +[ADR-0005](../90-decisions/ADR-0005-users-build-the-framework-assembles.md) and +[ADR-0047](../90-decisions/ADR-0047-compute-assembly-preserves-safe-runtime-topology.md); every guessing/laundering violation has produced a real deploy failure. Do not relitigate. diff --git a/docs/design/90-decisions/ADR-0005-users-build-the-framework-assembles.md b/docs/design/90-decisions/ADR-0005-users-build-the-framework-assembles.md index 924525c38..75097abfb 100644 --- a/docs/design/90-decisions/ADR-0005-users-build-the-framework-assembles.md +++ b/docs/design/90-decisions/ADR-0005-users-build-the-framework-assembles.md @@ -1,5 +1,7 @@ # ADR-0005: Users build their app; the framework assembles deploy artifacts from built output +Superseded in part by [ADR-0047](ADR-0047-compute-assembly-preserves-safe-runtime-topology.md): assembly may trace the declared entry's runtime files and preserve symlinks whose targets remain inside the final bundle; it still never dereferences a link or guesses an entry. + ## Decision The framework never initiates or configures a user's build. The contract is: @@ -30,7 +32,11 @@ disciplines bound it — each was violated in the first real out-of-repo deploy: symlinked (non-hoisted) `node_modules` is a **hard error** at package time, never dereferenced — the user's to fix (a hoisted linker: npm, or pnpm/bun `node-linker=hoisted`), because that same non-flat install also crashes a Next - standalone server at boot. + standalone server at boot. *(Amended by + [ADR-0047](ADR-0047-compute-assembly-preserves-safe-runtime-topology.md): a + symlink whose resolved target stays inside the assembled bundle is preserved + as a symlink; only links that escape the bundle or dangle are hard errors. A + link is still never dereferenced.)* - **Code boundary, not runtime.** A plain `node()` service relies on the Compute runtime's `bun` auto-install for the dynamic requires its bundler missed (e.g. `pg/lib/*`); a `nextjs()` artifact *disables* auto-install (its `sharp` / @@ -105,6 +111,9 @@ declared location fails loudly — an error naming the resolved path and saying output, never *how* to produce it. - Any monorepo layout deploys — the app's deep location is found, not assumed; a non-hoisted (symlinked) `node_modules` fails fast with an actionable error. + *(Amended by [ADR-0047](ADR-0047-compute-assembly-preserves-safe-runtime-topology.md): + it fails only when a link's resolved target lands outside the bundle or is + missing.)* - Deploy never writes into `node_modules` or the user's build output; staging is deploy-owned, keyed by graph address. - The wrapper bundle resolves the user's own dependencies (the service module diff --git a/docs/design/90-decisions/ADR-0047-compute-assembly-preserves-safe-runtime-topology.md b/docs/design/90-decisions/ADR-0047-compute-assembly-preserves-safe-runtime-topology.md new file mode 100644 index 000000000..28db2214c --- /dev/null +++ b/docs/design/90-decisions/ADR-0047-compute-assembly-preserves-safe-runtime-topology.md @@ -0,0 +1,51 @@ +# ADR-0047: Compute assembly preserves safe runtime topology and proves routing + +## Decision + +Composer's Compute path turns declared built output into a self-contained artifact without rewriting application code: + +```text +declared entry ──trace imports──▶ staged runtime files +safe symlink ────────────────▶ archived as a symlink +missing traced target ─────────▶ staged from the declared trace root +escaping link ────────────────▶ hard error +promoted URL ──route probe───▶ deploy succeeds +``` + +The Node directory adapter traces the explicitly declared entry's static runtime file graph and stages those files beside the directory the author named. It does not choose an entry, run a build, or bundle the application. + +The Compute archive preserves a symlink as a tar symlink only after resolving its real target and proving that target remains inside the assembled bundle. Long targets use a POSIX PAX `linkpath` rather than flattening the package tree. Long entry paths ride a PAX `path` record the same way, with a `PaxEntries/` placeholder left in the legacy USTAR field; extraction is therefore correct only for a consumer that honours PAX records for both the entry path and the link target. It never dereferences the link. Next.js assembly handles one framework-output gap first: if pnpm's standalone tree contains an in-root traced link but omits its virtual-store target, Composer stages the exact corresponding target from Next's declared `outputFileTracingRoot`. A target unavailable there remains dangling and fails, as do links that escape the bundle. + +The generated bootstrap may install a narrowly scoped compatibility shim when Compute's JavaScript runtime differs from the Node behavior a framework relies on. Such a shim must be feature-gated to that runtime and must run before the application entry is imported. + +Compute also supplies `HOST=0.0.0.0` when the author did not configure a host. Framework servers must listen on Compute's network interface rather than a loopback-only default; an explicit author value remains authoritative. The default is applied wherever `ComputeService.run()` executes, so local development and integration-test harnesses bind to all interfaces too, not only the deployed Compute runtime. + +## Reasoning + +"Users build; Composer assembles" is a boundary between owning a build and manufacturing a deployment artifact. It does not require Composer to ignore the runtime topology recorded by a build. Astro's Node adapter, for example, emits server files that deliberately retain bare package imports. Copying only `dist/` preserves the bytes but not the runnable program. Tracing from the author-declared entry follows package metadata and import edges deterministically; it is file assembly, not a second application build. + +The same distinction applies to symlinks. Dereferencing a package-manager link can silently pull arbitrary deploy-machine files into an artifact, which remains forbidden. Preserving the link itself retains the build's topology. Resolving the target only for validation proves that the archived link cannot escape the artifact, including through a chain of links, without copying the target through the link. The pnpm/Next repair is narrower: its destination is the missing in-artifact target, its source is the same relative path in Next's trace root, and the source's real path must remain inside that root. + +Frameworks also exercise details of the Compute runtime that a plain HTTP server may not. A compatibility shim belongs in Composer's generated bootstrap because it is part of the hosting envelope, not the user's framework build. The shim is deliberately narrow: the current URL custom-inspect setter restores Node-compatible assignment semantics for Bun without patching SvelteKit output or changing unrelated globals. + +## Consequences + +- Node directory artifacts can carry runtime packages that a framework intentionally leaves external. +- Safe package-manager and framework symlinks remain links in both cloud and local artifacts, including peer-context targets longer than USTAR's fixed field; escaping or unresolved dangling links fail before upload. +- Runtime compatibility code is isolated in the generated bootstrap and covered by framework deployment tests. +- Framework servers receive a listen-all host default without overriding an author-configured host. +- These mechanisms are compatibility ownership, not permanent duplication. When the upstream Alchemy Compute provider supplies an equivalent archive, build staging, or runtime bootstrap guarantee, Composer deletes the corresponding local mechanism rather than keeping two implementations. + +## Alternatives considered + +**Require every build directory to be flat and fully self-contained.** Rejected: standard framework outputs do not all have that shape, and forcing every application to maintain post-build copy scripts moves hosting assembly into userland. + +**Dereference symlinks during packaging.** Rejected: it changes the build topology and can package files outside the declared artifact boundary. + +**Bundle the application entry again.** Rejected: that crosses the user-build boundary and creates a second framework compatibility surface. Static file tracing preserves the application's emitted code. + +## Related + +- [ADR-0005](ADR-0005-users-build-the-framework-assembles.md) — users own builds; Composer owns deterministic artifact assembly. +- [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) — Alchemy is the provisioning engine behind deploy. +- [Architectural principles](../01-principles/architectural-principles.md) — Composer does not bundle application code or guess build output. diff --git a/docs/design/90-decisions/README.md b/docs/design/90-decisions/README.md index b9e8edaf0..3e108f44d 100644 --- a/docs/design/90-decisions/README.md +++ b/docs/design/90-decisions/README.md @@ -26,7 +26,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0003](ADR-0003-deploy-derives-everything-from-the-root-node.md) — `prisma-composer deploy` derives everything from the root node; there is no deploy config file. - [ADR-0004](ADR-0004-paths-resolve-relative-to-the-authoring-file.md) — Paths resolve relative to the file that writes them; the build adapter carries the authoring module. -- [ADR-0005](ADR-0005-users-build-the-framework-assembles.md) — Users build the app's code; the framework assembles the artifact by documented, deterministic steps (validate, wrap, each app-type's documented deploy step — e.g. Next's static/public copy). No guessing (arithmetic/depth-inference/discovery), no laundering (symlink = hard error); read the build tool's own manifest (Next's `relativeAppDir`), don't walk or compute. +- [ADR-0005](ADR-0005-users-build-the-framework-assembles.md) — Users build the app's code; the framework assembles the artifact by documented, deterministic steps (validate, wrap, each app-type's documented deploy step — e.g. Next's static/public copy). No guessing (arithmetic/depth-inference/discovery), no laundering (symlink = hard error); read the build tool's own manifest (Next's `relativeAppDir`), don't walk or compute. *(Superseded in part by ADR-0047: a symlink whose resolved target stays inside the bundle is preserved as a link; the no-guessing and no-dereferencing rules stand.)* - [ADR-0006](ADR-0006-every-node-is-named.md) — Every node is named; the root's name names the application. - [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) — Deploy drives Alchemy through a generated, inspectable stack file. - [ADR-0008](ADR-0008-wrapper-inlines-everything-except-runtime-builtins.md) — The boot wrapper inlines everything except runtime built-ins. @@ -68,3 +68,4 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. - [ADR-0045](ADR-0045-deploy-state-lives-behind-the-platform-state-api.md) — Deploy state lives behind the platform state API (the Management API implements Alchemy's stock `HttpStateApi` wire contract per Branch; composer's state layer is Alchemy's stock HTTP client), and deploys hold a server-side per-`(stack, stage)` lease (TTL 60s, heartbeated, released on exit; contention fails fast naming the holder; state operations without a live lease fail 409). Supersedes ADR-0010 (lock → lease) and the storage half of ADR-0034 (Branch scoping and lifetime stand; the visible per-stage database is gone); closes ADR-0012 as obsolete. No migration: legacy stages are refused until destroyed or deleted. - [ADR-0046](ADR-0046-the-orm-facade-is-a-peer-dependency.md) — `@prisma/composer-prisma-cloud` takes the Prisma Next postgres facade (`@prisma/orm-postgres`) as a **peer** dependency at one exact version, not a regular dependency: Composer registers an extension pack against the application's copy of the target, and two copies of a shell in one tree means two codec/operation registries and two class identities — a value from one is rejected by the other, silently. As a peer, that combination fails at install instead. Every `@prisma/orm-*` spec in the workspace is one exact version and all name the same one (`scripts/lint-orm-pins.mjs`). `@prisma/orm-toolchain`, which Composer drives rather than extends, stays a regular dependency. Replaces ADR-0022's consequence bullet on how the ORM is installed. +- [ADR-0047](ADR-0047-compute-assembly-preserves-safe-runtime-topology.md) — Compute assembly traces runtime files from the author-declared Node entry without rebundling app code, preserves only symlinks whose resolved targets remain inside the staged bundle, and installs narrowly runtime-gated bootstrap compatibility when a framework needs Node semantics. Each local mechanism is removed once the upstream Alchemy Compute provider owns the equivalent guarantee. Supersedes ADR-0005's blanket ban on symlinks and its assumption that directory output is already self-contained. diff --git a/docs/guides/building-an-app.md b/docs/guides/building-an-app.md index 4354539c8..d7f843a2a 100644 --- a/docs/guides/building-an-app.md +++ b/docs/guides/building-an-app.md @@ -540,13 +540,20 @@ finds its siblings exactly where the build left them — resolve them against `import.meta.url`, not the working directory. Nothing is guessed: you name the directory and the entry, and that is what -ships. Two things to know: - -- The tree must contain no symlinks — the platform's packager rejects them, so - assembly fails early and names the link rather than shipping a broken - artifact. Have your build emit real files. +ships. Three things to know: + +- Symlinks are kept as symlinks, never followed and copied. A link whose target + resolves inside the built output ships as-is. A link that points outside it, + or at something that isn't there, fails the deploy with an error naming the + link, rather than shipping a broken artifact or packaging files from your + machine. +- The entry's runtime imports ship too. Deploy traces the file you named and + stages the packages it imports beside `dir`, so framework output that keeps + bare imports (Astro's Node adapter, for example) boots without you copying + `node_modules` into the build. - `entry` must be a file inside `dir`. Pointing it outside with `../` is an - error, not an escape hatch — only `dir` is copied. + error, not an escape hatch — only `dir` is copied verbatim; everything else + arrives through the trace. Without `dir` you get the single-file form above, unchanged. diff --git a/packages/0-framework/2-authoring/bundle-paths/package.json b/packages/0-framework/2-authoring/bundle-paths/package.json new file mode 100644 index 000000000..93b541db3 --- /dev/null +++ b/packages/0-framework/2-authoring/bundle-paths/package.json @@ -0,0 +1,22 @@ +{ + "name": "@internal/bundle-paths", + "version": "0.6.0", + "private": true, + "type": "module", + "description": "The path-containment predicate and bundle symlink validation shared by assembly and packaging (ADR-0047's boundary, defined once).", + "exports": { + ".": "./dist/index.mjs", + "./package.json": "./package.json" + }, + "scripts": { + "typecheck": "tsc --noEmit", + "build": "tsdown", + "clean": "rm -rf dist", + "test": "bun test" + }, + "devDependencies": { + "typescript": "^6.0.3", + "tsdown": "^0.22.7", + "@internal/tsdown-config": "workspace:0.6.0" + } +} diff --git a/packages/0-framework/2-authoring/bundle-paths/src/bundle-paths.test.ts b/packages/0-framework/2-authoring/bundle-paths/src/bundle-paths.test.ts new file mode 100644 index 000000000..9ebb97aab --- /dev/null +++ b/packages/0-framework/2-authoring/bundle-paths/src/bundle-paths.test.ts @@ -0,0 +1,46 @@ +import { describe, expect, test } from 'bun:test'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { assertBundleSymlinksStayInside, isWithin } from './bundle-paths.ts'; + +describe('isWithin', () => { + test('the root itself and descendants are within; siblings and parents are not', () => { + expect(isWithin('/a/b', '/a/b')).toBe(true); + expect(isWithin('/a/b', '/a/b/c/d')).toBe(true); + expect(isWithin('/a/b', '/a')).toBe(false); + expect(isWithin('/a/b', '/a/c')).toBe(false); + expect(isWithin('/a/b', '/a/b-evil')).toBe(false); + expect(isWithin('/a/b', '/a/b/../c')).toBe(false); + }); +}); + +describe('assertBundleSymlinksStayInside', () => { + const scratch = () => fs.mkdtempSync(path.join(os.tmpdir(), 'bundle-paths-')); + + test('accepts a bundle whose links resolve inside it', async () => { + const bundle = path.join(scratch(), 'bundle'); + fs.mkdirSync(path.join(bundle, 'real'), { recursive: true }); + fs.symlinkSync(path.join('.', 'real'), path.join(bundle, 'link')); + + await assertBundleSymlinksStayInside(bundle); + }); + + test('rejects a dangling link', async () => { + const bundle = path.join(scratch(), 'bundle'); + fs.mkdirSync(bundle, { recursive: true }); + fs.symlinkSync('./missing', path.join(bundle, 'link')); + + await expect(assertBundleSymlinksStayInside(bundle)).rejects.toThrow('dangling symlink'); + }); + + test('rejects a link whose target escapes the bundle', async () => { + const parent = scratch(); + const bundle = path.join(parent, 'bundle'); + fs.mkdirSync(path.join(parent, 'outside'), { recursive: true }); + fs.mkdirSync(bundle, { recursive: true }); + fs.symlinkSync('../outside', path.join(bundle, 'link')); + + await expect(assertBundleSymlinksStayInside(bundle)).rejects.toThrow('escapes the bundle'); + }); +}); diff --git a/packages/0-framework/2-authoring/bundle-paths/src/bundle-paths.ts b/packages/0-framework/2-authoring/bundle-paths/src/bundle-paths.ts new file mode 100644 index 000000000..332c8fef6 --- /dev/null +++ b/packages/0-framework/2-authoring/bundle-paths/src/bundle-paths.ts @@ -0,0 +1,48 @@ +/** + * The path-containment predicate and bundle-link validation shared by every + * assembly and packaging seam (node/nextjs adapters, the compute artifact + * writer, the local extractor). This predicate is the enforcement point of + * ADR-0047's boundary — a symlink may be preserved only while its target + * stays inside the assembled bundle — so it exists exactly once. + */ +import fs from 'node:fs'; +import path from 'node:path'; + +/** Lexical containment: `candidate` is `root` itself or below it. Both paths + * must already be absolute or share a resolution base; no filesystem access. */ +export function isWithin(root: string, candidate: string): boolean { + const relative = path.relative(root, candidate); + return ( + relative === '' || + (!relative.startsWith(`..${path.sep}`) && relative !== '..' && !path.isAbsolute(relative)) + ); +} + +/** Walks the assembled bundle and rejects a dangling symlink or one whose + * resolved target escapes the bundle root. Symlinked directories are not + * descended: their targets are validated, and their contents belong to the + * target's own location. */ +export async function assertBundleSymlinksStayInside(bundleDir: string): Promise { + const realRoot = await fs.promises.realpath(bundleDir); + const walk = async (directory: string): Promise => { + for (const entry of await fs.promises.readdir(directory, { withFileTypes: true })) { + const full = path.join(directory, entry.name); + if (entry.isSymbolicLink()) { + let realTarget: string; + try { + realTarget = await fs.promises.realpath(full); + } catch { + throw new Error(`the assembled bundle contains a dangling symlink: ${full}`); + } + if (!isWithin(realRoot, realTarget)) { + throw new Error( + `the assembled bundle contains a symlink whose target escapes the bundle: ${full} -> ${await fs.promises.readlink(full)}`, + ); + } + } else if (entry.isDirectory()) { + await walk(full); + } + } + }; + await walk(bundleDir); +} diff --git a/packages/0-framework/2-authoring/bundle-paths/src/exports/index.ts b/packages/0-framework/2-authoring/bundle-paths/src/exports/index.ts new file mode 100644 index 000000000..de844f72d --- /dev/null +++ b/packages/0-framework/2-authoring/bundle-paths/src/exports/index.ts @@ -0,0 +1,2 @@ +/** Public surface. Implementation lives in `../bundle-paths.ts`. */ +export { assertBundleSymlinksStayInside, isWithin } from '../bundle-paths.ts'; diff --git a/packages/0-framework/2-authoring/bundle-paths/tsconfig.json b/packages/0-framework/2-authoring/bundle-paths/tsconfig.json new file mode 100644 index 000000000..a16ed256c --- /dev/null +++ b/packages/0-framework/2-authoring/bundle-paths/tsconfig.json @@ -0,0 +1,7 @@ +{ + "extends": "../../../../tsconfig.base.json", + "compilerOptions": { + "types": ["bun-types"] + }, + "include": ["src"] +} diff --git a/packages/0-framework/2-authoring/bundle-paths/tsdown.config.ts b/packages/0-framework/2-authoring/bundle-paths/tsdown.config.ts new file mode 100644 index 000000000..b21e84a8f --- /dev/null +++ b/packages/0-framework/2-authoring/bundle-paths/tsdown.config.ts @@ -0,0 +1,7 @@ +import { defineConfig } from '@internal/tsdown-config'; + +export default defineConfig({ + entry: { + index: 'src/exports/index.ts', + }, +}); diff --git a/packages/0-framework/2-authoring/nextjs/package.json b/packages/0-framework/2-authoring/nextjs/package.json index e28e2bbd2..607995e65 100644 --- a/packages/0-framework/2-authoring/nextjs/package.json +++ b/packages/0-framework/2-authoring/nextjs/package.json @@ -14,6 +14,7 @@ "clean": "rm -rf dist" }, "dependencies": { + "@internal/bundle-paths": "workspace:0.6.0", "@internal/core": "workspace:0.6.0", "esbuild": "^0.28.1" }, diff --git a/packages/0-framework/2-authoring/nextjs/src/__tests__/assemble.test.ts b/packages/0-framework/2-authoring/nextjs/src/__tests__/assemble.test.ts index b878cc141..ea8245ead 100644 --- a/packages/0-framework/2-authoring/nextjs/src/__tests__/assemble.test.ts +++ b/packages/0-framework/2-authoring/nextjs/src/__tests__/assemble.test.ts @@ -34,6 +34,7 @@ function writeNextBuild(root: string): { appRel: string } { fs.writeFileSync(path.join(appOut, 'server.js'), '// standalone server\n'); fs.mkdirSync(path.join(standalone, 'node_modules', 'next'), { recursive: true }); fs.writeFileSync(path.join(standalone, 'node_modules', 'next', 'marker.txt'), 'next\n'); + fs.symlinkSync('next', path.join(standalone, 'node_modules', 'next-linked')); // Client assets — omitted from standalone by Next, at the app root. fs.mkdirSync(path.join(root, '.next', 'static'), { recursive: true }); fs.writeFileSync(path.join(root, '.next', 'static', 'chunk.js'), '// static asset\n'); @@ -42,7 +43,10 @@ function writeNextBuild(root: string): { appRel: string } { // Next's manifest — records the app's subpath within standalone (posix). fs.writeFileSync( path.join(root, '.next', 'required-server-files.json'), - JSON.stringify({ relativeAppDir: 'apps/web' }), + JSON.stringify({ + relativeAppDir: 'apps/web', + config: { outputFileTracingRoot: root }, + }), ); fs.writeFileSync( path.join(root, 'src', 'service.ts'), @@ -109,6 +113,9 @@ describe('assemble()', () => { expect(fs.existsSync(path.join(workDir, 'bundle', 'node_modules', 'next', 'marker.txt'))).toBe( true, ); + expect(fs.readlinkSync(path.join(workDir, 'bundle', 'node_modules', 'next-linked'))).toBe( + 'next', + ); // The documented copy: static + public placed beside the app's server.js. expect(fs.existsSync(path.join(bundleApp, '.next', 'static', 'chunk.js'))).toBe(true); expect(fs.existsSync(path.join(bundleApp, 'public', 'favicon.ico'))).toBe(true); @@ -124,4 +131,135 @@ describe('assemble()', () => { const server = standaloneServerPath(nextjs({ module: moduleUrl(root), appDir: '..' })); expect(server).toBe(path.join(root, '.next', 'standalone', 'apps', 'web', 'server.js')); }); + + test('stages a pnpm virtual-store target that Next omitted behind a traced link', async () => { + const root = makeAppRoot(); + writeNextBuild(root); + const standalone = path.join(root, '.next', 'standalone'); + const source = path.join( + root, + 'node_modules', + '.pnpm', + 'semver@6.3.1', + 'node_modules', + 'semver', + ); + fs.mkdirSync(source, { recursive: true }); + fs.writeFileSync(path.join(source, 'index.js'), 'module.exports = "6.3.1";\n'); + const linkDir = path.join(standalone, 'node_modules', '.pnpm', 'node_modules'); + fs.mkdirSync(linkDir, { recursive: true }); + fs.symlinkSync('../semver@6.3.1/node_modules/semver', path.join(linkDir, 'semver')); + + const cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'prisma-composer-nextjs-cwd-')); + tmpDirs.push(cwd); + const result = await assemble({ + address: 'storefront.web', + cwd, + build: nextjs({ module: moduleUrl(root), appDir: '..' }), + }); + + const bundleStore = path.join( + cwd, + '.prisma-composer', + 'artifacts', + 'storefront.web', + 'bundle', + 'node_modules', + '.pnpm', + ); + expect(fs.readlinkSync(path.join(bundleStore, 'node_modules', 'semver'))).toBe( + '../semver@6.3.1/node_modules/semver', + ); + expect( + fs.readFileSync( + path.join(bundleStore, 'semver@6.3.1', 'node_modules', 'semver', 'index.js'), + 'utf8', + ), + ).toContain('6.3.1'); + expect(result.watch).toContain(source); + }, 20_000); + + test('refuses a manifest whose app location escapes its tracing root', async () => { + const root = makeAppRoot(); + writeNextBuild(root); + const manifestPath = path.join(root, '.next', 'required-server-files.json'); + fs.writeFileSync(manifestPath, JSON.stringify({ relativeAppDir: '../evil', config: {} })); + + await expect( + assemble({ + address: 'storefront.web', + cwd: root, + build: nextjs({ module: moduleUrl(root), appDir: '..' }), + }), + ).rejects.toThrow(/escapes its tracing root/); + }, 20_000); + + test('names the missing manifest field when links need repair and no tracing root is recorded', async () => { + const root = makeAppRoot(); + writeNextBuild(root); + const standalone = path.join(root, '.next', 'standalone'); + const linkDir = path.join(standalone, 'node_modules', '.pnpm', 'node_modules'); + fs.mkdirSync(linkDir, { recursive: true }); + fs.symlinkSync('../semver@6.3.1/node_modules/semver', path.join(linkDir, 'semver')); + const manifestPath = path.join(root, '.next', 'required-server-files.json'); + fs.writeFileSync(manifestPath, JSON.stringify({ relativeAppDir: 'apps/web', config: {} })); + + await expect( + assemble({ + address: 'storefront.web', + cwd: root, + build: nextjs({ module: moduleUrl(root), appDir: '..' }), + }), + ).rejects.toThrow(/records no config\.outputFileTracingRoot/); + }, 20_000); + + test('fails at assemble when a staged store payload carries an escaping link', async () => { + const root = makeAppRoot(); + writeNextBuild(root); + const standalone = path.join(root, '.next', 'standalone'); + const source = path.join( + root, + 'node_modules', + '.pnpm', + 'semver@6.3.1', + 'node_modules', + 'semver', + ); + fs.mkdirSync(source, { recursive: true }); + fs.writeFileSync(path.join(source, 'index.js'), 'module.exports = "6.3.1";\n'); + fs.writeFileSync(path.join(root, 'outside-the-bundle.txt'), 'must not ship'); + // Copied verbatim into the bundle by staging, where it points outside. + fs.symlinkSync(path.join(root, 'outside-the-bundle.txt'), path.join(source, 'escaped.txt')); + const linkDir = path.join(standalone, 'node_modules', '.pnpm', 'node_modules'); + fs.mkdirSync(linkDir, { recursive: true }); + fs.symlinkSync('../semver@6.3.1/node_modules/semver', path.join(linkDir, 'semver')); + + const cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'prisma-composer-nextjs-cwd-')); + tmpDirs.push(cwd); + + await expect( + assemble({ + address: 'storefront.web', + cwd, + build: nextjs({ module: moduleUrl(root), appDir: '..' }), + }), + ).rejects.toThrow(/assembled bundle contains a symlink whose target escapes the bundle/); + }, 20_000); + + test('assembles a complete standalone build when its recorded tracing root is absent', async () => { + const root = makeAppRoot(); + writeNextBuild(root); + const manifestPath = path.join(root, '.next', 'required-server-files.json'); + const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); + manifest.config.outputFileTracingRoot = path.join(root, 'missing-build-machine-root'); + fs.writeFileSync(manifestPath, JSON.stringify(manifest)); + + const result = await assemble({ + address: 'storefront.web', + cwd: root, + build: nextjs({ module: moduleUrl(root), appDir: '..' }), + }); + + expect(fs.existsSync(path.join(result.dir, result.entry))).toBe(true); + }, 20_000); }); diff --git a/packages/0-framework/2-authoring/nextjs/src/control/build.ts b/packages/0-framework/2-authoring/nextjs/src/control/build.ts index d4f517a64..85dde5e78 100644 --- a/packages/0-framework/2-authoring/nextjs/src/control/build.ts +++ b/packages/0-framework/2-authoring/nextjs/src/control/build.ts @@ -11,10 +11,11 @@ * `outputFileTracingRoot` is the monorepo root) is *read from Next's own build * manifest* (`.next/required-server-files.json`'s `relativeAppDir`), never walked * for or computed from a hardcoded depth. It does not launder: node_modules is - * shipped exactly as `next build` produced it, so a symlinked (non-hoisted) - * node_modules is the packager's hard error — the same misconfiguration crashes - * the standalone server at boot, so it must be a flat install (npm, or pnpm/bun - * with a hoisted node-linker). + * shipped exactly as `next build` produced it. The packager preserves a link + * only after resolving its target inside the assembled bundle, and rejects + * escaping links. When Next records an in-root package link but omits its target + * from standalone (seen with pnpm's virtual store), assembly stages that exact + * target from `outputFileTracingRoot` so the artifact remains self-contained. * * Artifact layout: `/main.mjs` (our wrapper) + `/bundle/` * (the standalone tree, with static/public copied in). The packager adds @@ -27,6 +28,7 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { fileURLToPath } from 'node:url'; +import { assertBundleSymlinksStayInside, isWithin } from '@internal/bundle-paths'; import type { BuildAdapter } from '@internal/core'; import type { ExtensionDescriptor } from '@internal/core/config'; import type { AssembleInput, Bundle } from '@internal/core/deploy'; @@ -42,43 +44,159 @@ function isNextjsBuild(descriptor: BuildAdapter): descriptor is NextjsBuildAdapt ); } -/** - * The app's own subpath within `.next/standalone`, as an OS-relative path. Next - * mirrors the app's location under `outputFileTracingRoot` (deep, when that's the - * monorepo root); rather than walk the tree for `server.js`, we read where Next - * put it from `.next/required-server-files.json` — `relativeAppDir` is exactly - * that subpath. Older Next lacks the field; fall back to computing it from the - * same manifest's `config.outputFileTracingRoot`. - */ -function nextAppRel(appDir: string): string { +/** What `.next/required-server-files.json` tells us: where Next put the app + * inside the standalone tree, and the source root that tree mirrors. */ +interface ServerFilesManifest { + readonly path: string; + readonly relativeAppDir: string | undefined; + /** Absolute `config.outputFileTracingRoot`, or undefined when unrecorded. */ + readonly tracingRoot: string | undefined; +} + +function readServerFilesManifest(appDir: string): ServerFilesManifest { const manifestPath = path.join(appDir, '.next', 'required-server-files.json'); if (!fs.existsSync(manifestPath)) { throw new Error( `no ${path.join('.next', 'required-server-files.json')} under ${appDir} — run \`next build\` with output: "standalone" first.`, ); } - // JSON.parse is `any`; both fields we read are re-checked with `typeof` below. + // JSON.parse is `any`; both fields we read are re-checked with `typeof`. const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); const relativeAppDir: unknown = manifest?.relativeAppDir; const tracingRoot: unknown = manifest?.config?.outputFileTracingRoot; + return { + path: manifestPath, + relativeAppDir: typeof relativeAppDir === 'string' ? relativeAppDir : undefined, + tracingRoot: typeof tracingRoot === 'string' ? path.resolve(appDir, tracingRoot) : undefined, + }; +} + +/** + * The app's own subpath within `.next/standalone`, as an OS-relative path. Next + * mirrors the app's location under `outputFileTracingRoot` (deep, when that's the + * monorepo root); rather than walk the tree for `server.js`, we read where Next + * put it from `.next/required-server-files.json` — `relativeAppDir` is exactly + * that subpath. Older Next lacks the field; fall back to computing it from the + * same manifest's `config.outputFileTracingRoot`. + */ +function appRelFrom(manifest: ServerFilesManifest, appDir: string): string { const posixRel = - typeof relativeAppDir === 'string' - ? relativeAppDir - : typeof tracingRoot === 'string' - ? path.relative(tracingRoot, appDir).split(path.sep).join('/') - : undefined; + manifest.relativeAppDir ?? + (manifest.tracingRoot === undefined + ? undefined + : path.relative(manifest.tracingRoot, appDir).split(path.sep).join('/')); if (posixRel === undefined) { throw new Error( - `${manifestPath} records neither relativeAppDir nor config.outputFileTracingRoot — cannot locate the standalone server`, + `${manifest.path} records neither relativeAppDir nor config.outputFileTracingRoot — cannot locate the standalone server`, + ); + } + const rel = posixRel.split('/').join(path.sep); + if (path.isAbsolute(rel) || !isWithin(appDir, path.join(appDir, rel))) { + throw new Error( + `${manifest.path} records an app location that escapes its tracing root (${posixRel}) — refusing to stage outside the bundle`, + ); + } + return rel; +} + +async function lstatIfPresent(candidate: string): Promise { + try { + return await fs.promises.lstat(candidate); + } catch (error) { + if (error instanceof Error && Reflect.get(error, 'code') === 'ENOENT') return undefined; + throw error; + } +} + +/** Do not write through an existing link while repairing a missing target. */ +async function hasSymlinkAncestor(root: string, candidate: string): Promise { + const relative = path.relative(root, path.dirname(candidate)); + let cursor = root; + for (const segment of relative.split(path.sep).filter(Boolean)) { + cursor = path.join(cursor, segment); + const stat = await lstatIfPresent(cursor); + if (stat === undefined) return false; + if (stat.isSymbolicLink()) return true; + } + return false; +} + +async function collectSymlinks(root: string): Promise { + const links: string[] = []; + async function visit(directory: string): Promise { + for (const entry of await fs.promises.readdir(directory, { withFileTypes: true })) { + const entryPath = path.join(directory, entry.name); + if (entry.isSymbolicLink()) links.push(entryPath); + else if (entry.isDirectory()) await visit(entryPath); + } + } + await visit(root); + return links; +} + +/** In-bundle link targets that the standalone tree does not contain — the + * repairs staging has to make. */ +async function missingLinkTargets(bundleDir: string): Promise { + const missing: string[] = []; + for (const linkPath of await collectSymlinks(bundleDir)) { + const rawTarget = await fs.promises.readlink(linkPath); + if (path.isAbsolute(rawTarget)) continue; + const target = path.resolve(path.dirname(linkPath), rawTarget); + if (!isWithin(bundleDir, target) || (await lstatIfPresent(target)) !== undefined) continue; + if (await hasSymlinkAncestor(bundleDir, target)) continue; + missing.push(target); + } + return missing; +} + +/** + * pnpm can leave a traced hoist link in standalone while omitting the virtual- + * store directory it targets. The same target still exists at the corresponding + * path under outputFileTracingRoot. Stage only that exact, real in-root target; + * unsafe or genuinely unavailable links remain for the packager to reject. + */ +async function stageMissingStandaloneLinkTargets( + bundleDir: string, + manifest: ServerFilesManifest, +): Promise { + const tracingRoot = manifest.tracingRoot; + if (tracingRoot === undefined) { + if ((await missingLinkTargets(bundleDir)).length === 0) return []; + throw new Error( + `${manifest.path} records no config.outputFileTracingRoot, but the standalone tree contains symlinks whose targets Next omitted — the source root those targets must be staged from is unknown. Rebuild with a Next version that records config.outputFileTracingRoot, or set outputFileTracingRoot in next.config.`, ); } - return posixRel.split('/').join(path.sep); + // required-server-files.json records the build machine's absolute tracing + // root. A copied standalone build can be assembled elsewhere; if all of its + // links are already complete, no access to the original root is needed. + if ((await lstatIfPresent(tracingRoot)) === undefined) return []; + const tracedRootReal = await fs.promises.realpath(tracingRoot); + const stagedSources = new Set(); + let staged = true; + while (staged) { + staged = false; + for (const target of await missingLinkTargets(bundleDir)) { + const standaloneRelative = path.relative(bundleDir, target); + const source = path.resolve(tracingRoot, standaloneRelative); + const sourceStat = await lstatIfPresent(source); + if (sourceStat === undefined) continue; + const sourceReal = await fs.promises.realpath(source); + if (!isWithin(tracedRootReal, sourceReal)) continue; + + await fs.promises.mkdir(path.dirname(target), { recursive: true }); + await fs.promises.cp(source, target, { recursive: true, verbatimSymlinks: true }); + stagedSources.add(source); + staged = true; + } + } + return [...stagedSources]; } /** The built standalone server.js for a nextjs build — `appDir`'s standalone root plus the app subpath Next recorded. Single-sourced so `assemble()` (deploy) and the integration-test seam can't drift. */ export function standaloneServerPath(build: NextjsBuildAdapter): string { const appDir = path.resolve(path.dirname(fileURLToPath(build.module)), build.appDir); - return path.join(appDir, '.next', 'standalone', nextAppRel(appDir), 'server.js'); + const manifest = readServerFilesManifest(appDir); + return path.join(appDir, '.next', 'standalone', appRelFrom(manifest, appDir), 'server.js'); } export async function assemble(input: AssembleInput): Promise { @@ -101,16 +219,22 @@ export async function assemble(input: AssembleInput): Promise { } // The app's (possibly deep) location within the standalone tree — read from // Next's own build manifest, not searched for. - const appRel = nextAppRel(appDir); + const manifest = readServerFilesManifest(appDir); + const appRel = appRelFrom(manifest, appDir); const workDir = path.join(input.cwd, '.prisma-composer', 'artifacts', input.address); await fs.promises.rm(workDir, { recursive: true, force: true }); await fs.promises.mkdir(workDir, { recursive: true }); const bundleDir = path.join(workDir, 'bundle'); - // Ship the standalone tree as `next build` produced it (a symlinked - // node_modules stays symlinked → the packager rejects it, correctly). - await fs.promises.cp(standaloneRoot, bundleDir, { recursive: true }); + // Ship the standalone tree as `next build` produced it. Framework-emitted + // links stay links; the packager validates that every target remains inside + // the assembled bundle before emitting it into the archive. + await fs.promises.cp(standaloneRoot, bundleDir, { + recursive: true, + verbatimSymlinks: true, + }); + const stagedLinkTargets = await stageMissingStandaloneLinkTargets(bundleDir, manifest); // The documented copy: Next omits the client assets from standalone; place // them beside the app's server.js so it serves them (docs: `cp -r public @@ -118,13 +242,24 @@ export async function assemble(input: AssembleInput): Promise { const appOut = path.join(bundleDir, appRel); const staticSrc = path.join(appDir, '.next', 'static'); if (fs.existsSync(staticSrc)) { - await fs.promises.cp(staticSrc, path.join(appOut, '.next', 'static'), { recursive: true }); + await fs.promises.cp(staticSrc, path.join(appOut, '.next', 'static'), { + recursive: true, + verbatimSymlinks: true, + }); } const publicSrc = path.join(appDir, 'public'); if (fs.existsSync(publicSrc)) { - await fs.promises.cp(publicSrc, path.join(appOut, 'public'), { recursive: true }); + await fs.promises.cp(publicSrc, path.join(appOut, 'public'), { + recursive: true, + verbatimSymlinks: true, + }); } + // Fail here, at the cause, rather than in the packager: a dangling or + // escaping link left by the standalone tree or by a staged store payload is + // reported against the assembled bundle it came from. + await assertBundleSymlinksStayInside(bundleDir); + // Our wrapper, bundled to main.mjs at the working-dir root (unambiguously // ESM). run()'s `import("./bundle/")` resolves from here. const serviceModule = fileURLToPath(buildDescriptor.module); @@ -144,7 +279,7 @@ export async function assemble(input: AssembleInput): Promise { return { dir: workDir, entry: path.posix.join('bundle', appRel.split(path.sep).join('/'), 'server.js'), - watch: [standaloneRoot], + watch: [standaloneRoot, ...stagedLinkTargets], }; } diff --git a/packages/0-framework/2-authoring/node/package.json b/packages/0-framework/2-authoring/node/package.json index 045b48418..06cb17df2 100644 --- a/packages/0-framework/2-authoring/node/package.json +++ b/packages/0-framework/2-authoring/node/package.json @@ -14,7 +14,9 @@ "clean": "rm -rf dist" }, "dependencies": { + "@internal/bundle-paths": "workspace:0.6.0", "@internal/core": "workspace:0.6.0", + "@vercel/nft": "^1.10.2", "esbuild": "^0.28.1" }, "devDependencies": { diff --git a/packages/0-framework/2-authoring/node/src/__tests__/assemble.test.ts b/packages/0-framework/2-authoring/node/src/__tests__/assemble.test.ts index 7ec8c01e2..432a7045d 100644 --- a/packages/0-framework/2-authoring/node/src/__tests__/assemble.test.ts +++ b/packages/0-framework/2-authoring/node/src/__tests__/assemble.test.ts @@ -514,11 +514,7 @@ describe('assemble() — the directory form', () => { ).rejects.toThrow(/sits inside the build adapter's dir .* copy the artifact into itself/s); }); - test('rejects a tree containing a symlink, naming it — the packager rejects symlinks, and we ship what the build produced', async () => { - // Decided over dereferencing on copy: the artifact must be the tree the - // author's build produced (ADR-0005), and following a link could pull in - // files from outside dir that the author never named. Failing here beats - // failing in the packager, which reports it far from the cause. + test('rejects a tree symlink whose target is outside the final assembled bundle', async () => { const serviceDir = makeServiceDir(); writeTree(path.join(serviceDir, 'dist'), { 'server/start.js': 'export default "app-entry";\n', @@ -536,10 +532,10 @@ describe('assemble() — the directory form', () => { address: 'svc', cwd: makeCwd(), }), - ).rejects.toThrow(/contains symlinks.*server\/util\.js/s); + ).rejects.toThrow(/symlink whose target escapes the bundle.*bundle\/util\.js/s); }); - test('reports a symlinked directory without descending into it', async () => { + test('rejects an escaping directory symlink without descending into it', async () => { const serviceDir = makeServiceDir(); writeTree(path.join(serviceDir, 'dist'), { 'server/start.js': 'export default "app-entry";\n', @@ -557,7 +553,27 @@ describe('assemble() — the directory form', () => { address: 'svc', cwd: makeCwd(), }), - ).rejects.toThrow(/contains symlinks.*server\/vendor/s); + ).rejects.toThrow(/symlink whose target escapes the bundle.*bundle\/vendor/s); + }); + + test('preserves a relative directory symlink whose target stays inside the built tree', async () => { + const serviceDir = makeServiceDir(); + writeTree(path.join(serviceDir, 'dist', 'server'), { + 'start.js': 'export default "app-entry";\n', + 'node_modules/real/index.js': 'export const value = 1;\n', + }); + fs.symlinkSync('real', path.join(serviceDir, 'dist', 'server', 'node_modules', 'linked')); + writeServiceModule(serviceDir); + + const result = await assemble({ + build: node({ module: moduleUrl(serviceDir), dir: '../dist/server', entry: 'start.js' }), + address: 'svc', + cwd: makeCwd(), + }); + + const copied = path.join(result.dir, 'bundle', 'node_modules', 'linked'); + expect(fs.lstatSync(copied).isSymbolicLink()).toBe(true); + expect(fs.readlinkSync(copied)).toBe('real'); }); test('rejects a dir that is itself a symlink to a directory — hard-errors instead of dereferencing it and copying the target', async () => { @@ -579,7 +595,7 @@ describe('assemble() — the directory form', () => { address: 'svc', cwd: makeCwd(), }), - ).rejects.toThrow(/contains symlinks.*dist\/server/s); + ).rejects.toThrow(/dir .* is itself a symlink/s); }); test('rejects a dir that is itself a symlink to a FILE — the same hard error, not "not a directory"', async () => { @@ -602,6 +618,197 @@ describe('assemble() — the directory form', () => { address: 'svc', cwd: makeCwd(), }), - ).rejects.toThrow(/contains symlinks.*dist\/server/s); + ).rejects.toThrow(/dir .* is itself a symlink/s); }); + + test('stages bare runtime dependencies traced from a directory entry (Astro Node output shape)', async () => { + const serviceDir = makeServiceDir(); + const cwd = makeCwd(); + writeTree(path.join(serviceDir, 'dist'), { + 'server/entry.mjs': 'import { marker } from "runtime-fixture"; export default marker;\n', + }); + const marker = installFixturePackage(serviceDir, 'runtime-fixture'); + writeServiceModule(serviceDir); + + const result = await assemble({ + build: node({ + module: moduleUrl(serviceDir), + dir: '../dist', + entry: 'server/entry.mjs', + }), + address: 'astro', + cwd, + }); + + expect(result.entry).toBe('bundle/server/entry.mjs'); + expect( + fs.readFileSync( + path.join(result.dir, 'bundle', 'node_modules', 'runtime-fixture', 'index.js'), + 'utf8', + ), + ).toContain(marker); + }, 20_000); + + test('re-roots workspace package dependencies under a Node lookup path', async () => { + const workspaceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'prisma-composer-workspace-')); + tmpDirs.push(workspaceRoot); + const serviceDir = path.join(workspaceRoot, 'apps', 'web'); + fs.mkdirSync(path.join(serviceDir, 'src'), { recursive: true }); + writeTree(path.join(serviceDir, 'dist'), { + 'server/entry.mjs': 'import { marker } from "runtime-fixture"; export default marker;\n', + }); + const marker = 'WORKSPACE_RUNTIME_FIXTURE'; + const workspacePackage = path.join(workspaceRoot, 'packages', 'runtime-fixture'); + writeTree(workspacePackage, { + 'package.json': JSON.stringify({ + name: 'runtime-fixture', + version: '1.0.0', + type: 'module', + main: 'index.js', + }), + 'index.js': `export const marker = ${JSON.stringify(marker)};\n`, + }); + const serviceNodeModules = path.join(serviceDir, 'node_modules'); + fs.mkdirSync(serviceNodeModules, { recursive: true }); + fs.symlinkSync( + path.relative(serviceNodeModules, workspacePackage), + path.join(serviceNodeModules, 'runtime-fixture'), + ); + writeServiceModule(serviceDir); + + const result = await assemble({ + build: node({ + module: moduleUrl(serviceDir), + dir: '../dist', + entry: 'server/entry.mjs', + }), + address: 'astro', + cwd: workspaceRoot, + }); + + const stagedPackage = path.join( + result.dir, + 'bundle', + 'node_modules', + 'runtime-fixture', + 'index.js', + ); + expect(fs.lstatSync(path.dirname(stagedPackage)).isSymbolicLink()).toBe(true); + expect(fs.readFileSync(stagedPackage, 'utf8')).toContain(marker); + const loaded = await import(pathToFileURL(path.join(result.dir, result.entry)).href); + expect(loaded.default).toBe(marker); + }, 20_000); + + test('stages a dependency reachable only through a workspace-root virtual store, identically from any cwd', async () => { + // The pnpm shape: the app lives at repo/apps/web, but its dependency's real + // files sit in the store at repo/node_modules/.pnpm, above the app. A + // staging root picked from the deploy cwd (or from the app directory alone) + // excludes the store, and the trace then drops it — a bundle that boots + // straight into "cannot find module dep". The root must reach the store + // whichever directory the deploy was invoked from, so both runs here must + // produce byte-identical layouts. + const workspaceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'prisma-composer-pnpm-')); + tmpDirs.push(workspaceRoot); + const serviceDir = path.join(workspaceRoot, 'apps', 'web'); + fs.mkdirSync(path.join(serviceDir, 'src'), { recursive: true }); + writeTree(path.join(serviceDir, 'dist'), { + 'server/entry.mjs': 'import { marker } from "dep"; export default marker;\n', + }); + const marker = 'PNPM_STORE_FIXTURE'; + const storePackage = path.join( + workspaceRoot, + 'node_modules', + '.pnpm', + 'dep@1.0.0', + 'node_modules', + 'dep', + ); + writeTree(storePackage, { + 'package.json': JSON.stringify({ + name: 'dep', + version: '1.0.0', + type: 'module', + main: 'index.js', + }), + 'index.js': `export const marker = ${JSON.stringify(marker)};\n`, + }); + const serviceNodeModules = path.join(serviceDir, 'node_modules'); + fs.mkdirSync(serviceNodeModules, { recursive: true }); + fs.symlinkSync( + path.relative(serviceNodeModules, storePackage), + path.join(serviceNodeModules, 'dep'), + ); + writeServiceModule(serviceDir); + + const assembleFrom = (cwd: string) => + assemble({ + build: node({ module: moduleUrl(serviceDir), dir: '../dist', entry: 'server/entry.mjs' }), + address: 'astro', + cwd, + }); + + const first = await assembleFrom(makeCwd()); + const storeRelative = 'node_modules/.pnpm/dep@1.0.0/node_modules/dep'; + // The store is staged intact; node_modules/dep is the app-level link into + // it, so treeContents (which follows it) reports the same files twice. + expect(treeContents(path.join(first.dir, 'bundle'))).toEqual([ + `${storeRelative}/index.js`, + `${storeRelative}/package.json`, + 'node_modules/dep/index.js', + 'node_modules/dep/package.json', + 'server/entry.mjs', + ]); + // The app-level link survives as a link into the staged store, so Node's + // own resolution finds the dependency the same way it did before assembly. + const linked = path.join(first.dir, 'bundle', 'node_modules', 'dep'); + expect(fs.lstatSync(linked).isSymbolicLink()).toBe(true); + expect(fs.readlinkSync(linked)).toBe('.pnpm/dep@1.0.0/node_modules/dep'); + const loaded = await import(pathToFileURL(path.join(first.dir, first.entry)).href); + expect(loaded.default).toBe(marker); + + const second = await assembleFrom(workspaceRoot); + expect(treeContents(path.join(second.dir, 'bundle'))).toEqual( + treeContents(path.join(first.dir, 'bundle')), + ); + expect(fs.readlinkSync(path.join(second.dir, 'bundle', 'node_modules', 'dep'))).toBe( + fs.readlinkSync(linked), + ); + }, 30_000); + + test('rejects two traced packages of the same name that would collapse onto one bundle path', async () => { + // Staging keeps each file's path from its FIRST node_modules segment, so a + // hoisted /node_modules/dup and a nested + // /packages/lib/node_modules/dup both land on + // bundle/node_modules/dup. Silently keeping whichever was staged first + // ships an arbitrary version; assembly must name both instead. + const serviceDir = makeServiceDir(); + writeTree(path.join(serviceDir, 'dist'), { + 'server/entry.mjs': + 'import { marker } from "dup";\nimport { nested } from "lib";\nexport default marker + nested;\n', + }); + const dupPackage = (version: string, marker: string) => ({ + 'package.json': JSON.stringify({ name: 'dup', version, type: 'module', main: 'index.js' }), + 'index.js': `export const marker = ${JSON.stringify(marker)};\n`, + }); + writeTree(path.join(serviceDir, 'node_modules', 'dup'), dupPackage('1.0.0', 'HOISTED')); + const nestedLib = path.join(serviceDir, 'packages', 'lib'); + writeTree(nestedLib, { + 'package.json': JSON.stringify({ name: 'lib', version: '1.0.0', type: 'module' }), + 'index.js': 'export { marker as nested } from "dup";\n', + }); + writeTree(path.join(nestedLib, 'node_modules', 'dup'), dupPackage('2.0.0', 'NESTED')); + fs.symlinkSync( + path.relative(path.join(serviceDir, 'node_modules'), nestedLib), + path.join(serviceDir, 'node_modules', 'lib'), + ); + writeServiceModule(serviceDir); + + await expect( + assemble({ + build: node({ module: moduleUrl(serviceDir), dir: '../dist', entry: 'server/entry.mjs' }), + address: 'svc', + cwd: makeCwd(), + }), + ).rejects.toThrow(/stage to the same bundle path.*node_modules\/dup.*packages\/lib/s); + }, 20_000); }); diff --git a/packages/0-framework/2-authoring/node/src/control/build.ts b/packages/0-framework/2-authoring/node/src/control/build.ts index c4ad9b5cb..168f1ba35 100644 --- a/packages/0-framework/2-authoring/node/src/control/build.ts +++ b/packages/0-framework/2-authoring/node/src/control/build.ts @@ -8,8 +8,13 @@ * Two forms, chosen by the descriptor: without `dir`, `entry` is a single * self-contained file and only that file is copied. With `dir`, the whole * directory is copied verbatim and `entry` names the file inside it that boots. - * Neither form discovers anything — no tree-walking for an entry, no filename - * heuristics; the author states the paths and we copy exactly those. + * The directory form also follows the declared entry's static runtime dependency + * graph and stages those files beside the output. The staging root is derived + * from the declared paths and the traced files themselves — never from the + * deploy cwd — so the bundle's layout is the same whichever directory the + * deploy is invoked from. This is deterministic dependency assembly (not app + * bundling), and is what makes framework outputs such as Astro's Node adapter + * self-contained. * * The wrapper is a SEPARATE esbuild build of the service module (declarations * only, whose node carries run()/load()), emitted as `main.mjs` at the @@ -25,9 +30,11 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { fileURLToPath } from 'node:url'; +import { assertBundleSymlinksStayInside, isWithin } from '@internal/bundle-paths'; import type { BuildAdapter } from '@internal/core'; import type { ExtensionDescriptor } from '@internal/core/config'; import type { AssembleInput, Bundle } from '@internal/core/deploy'; +import { nodeFileTrace } from '@vercel/nft'; import { build } from 'esbuild'; import type { NodeBuildAdapter } from '../node.ts'; @@ -78,42 +85,6 @@ function resolveFile(entrySpec: string, moduleDir: string): BuiltRunnable { }; } -/** The shared "dir contains symlinks" error, reused by both the root-is-a-symlink case and the nested-symlink walk below — one message shape, never two to drift apart. */ -function symlinksFoundError(dirPath: string, found: readonly string[]): Error { - const listed = found.slice(0, 5).join(', '); - return new Error( - `the build adapter's dir ("${dirPath}") contains symlinks, which the platform's packager ` + - `rejects: ${listed}${found.length > 5 ? `, and ${found.length - 5} more` : ''}. The tree is ` + - 'copied verbatim, so make your build emit real files in dir (for example, a hoisted ' + - 'node_modules, or dereference the links into dir with cp -RL).', - ); -} - -/** - * Compute's packager rejects symlinks, so a tree containing one cannot deploy. - * We fail here, naming the links, rather than dereferencing them on the copy: - * the artifact must be what the author's build produced (ADR-0005), and - * following a link that points outside `dir` would pull in files the author - * never named. The walk reads dirents (lstat semantics), so a symlinked - * directory is reported and never descended into. Checks only `dirPath`'s - * children — the caller (`resolveDir`) checks `dirPath` itself before this - * runs, since that check also decides "not a directory" vs "is a symlink" - * and must happen before any dereferencing stat. - */ -async function assertNoSymlinks(dirPath: string): Promise { - const found: string[] = []; - const walk = async (current: string): Promise => { - for (const entry of await fs.promises.readdir(current, { withFileTypes: true })) { - const full = path.join(current, entry.name); - if (entry.isSymbolicLink()) found.push(full); - else if (entry.isDirectory()) await walk(full); - } - }; - await walk(dirPath); - - if (found.length > 0) throw symlinksFoundError(dirPath, found); -} - /** * The directory form: `dir` is the built tree, resolved against dirname(module) * (ADR-0004) and copied whole; `entry` resolves inside `dir` and names the file @@ -143,7 +114,9 @@ async function resolveDir( ); } if (dirLstat.isSymbolicLink()) { - throw symlinksFoundError(dirPath, [dirPath]); + throw new Error( + `the build adapter's dir ("${dirPath}") is itself a symlink — name the built directory directly. Nested links are preserved only after the final assembled bundle proves their targets stay inside it.`, + ); } if (!dirLstat.isDirectory()) { throw new Error( @@ -166,16 +139,191 @@ async function resolveDir( ); } - await assertNoSymlinks(dirPath); - return { source: dirPath, sourceField: 'dir', entry: path.relative(dirPath, entryPath).split(path.sep).join('/'), - copyInto: (bundleDir) => fs.promises.cp(dirPath, bundleDir, { recursive: true }), + copyInto: (bundleDir) => + fs.promises.cp(dirPath, bundleDir, { recursive: true, verbatimSymlinks: true }), }; } +function commonAncestor(left: string, right: string): string { + let candidate = path.resolve(left); + const resolvedRight = path.resolve(right); + while (!isWithin(candidate, resolvedRight)) { + const parent = path.dirname(candidate); + if (parent === candidate) return candidate; + candidate = parent; + } + return candidate; +} + +function isFilesystemRoot(candidate: string): boolean { + return path.parse(candidate).root === candidate; +} + +async function realPathOrSelf(target: string): Promise { + try { + return await fs.promises.realpath(target); + } catch { + return target; + } +} + +async function pathExists(target: string): Promise { + try { + await fs.promises.lstat(target); + return true; + } catch { + return false; + } +} + +/** Map traced packages to a node_modules directory the copied entry searches. + * Keep the complete suffix from the first node_modules segment so pnpm's + * virtual store and nested dependency topology remain intact. Non-package + * targets (for example a workspace package behind a node_modules symlink) keep + * their staging-root-relative location inside the bundle. */ +function stagedRuntimePath(source: string, stagingRoot: string, bundleDir: string): string { + const relative = path.relative(stagingRoot, source); + const segments = relative.split(path.sep); + const nodeModules = segments.indexOf('node_modules'); + const stagedSegments = nodeModules === -1 ? segments : segments.slice(nodeModules); + return path.join(bundleDir, ...stagedSegments); +} + +async function copyTracedEntry( + source: string, + destination: string, + stagingRoot: string, + bundleDir: string, + dirPath: string, +): Promise { + const stat = await fs.promises.lstat(source); + await fs.promises.mkdir(path.dirname(destination), { recursive: true }); + + if (stat.isSymbolicLink()) { + const realTarget = await fs.promises.realpath(source); + if (!isWithin(stagingRoot, realTarget)) { + throw new Error( + `the runtime dependency trace found a symlink outside its staging root: ${source} -> ${realTarget}`, + ); + } + const stagedTarget = isWithin(dirPath, realTarget) + ? path.join(bundleDir, path.relative(dirPath, realTarget)) + : stagedRuntimePath(realTarget, stagingRoot, bundleDir); + const linkTarget = path.relative(path.dirname(destination), stagedTarget); + await fs.promises.symlink(linkTarget, destination); + return; + } + if (stat.isDirectory()) { + await fs.promises.mkdir(destination, { recursive: true }); + return; + } + if (!stat.isFile()) { + throw new Error( + `the runtime dependency trace found an unsupported filesystem entry: ${source}`, + ); + } + await fs.promises.copyFile(source, destination); +} + +/** + * The staging root: the deepest directory containing the service module, the + * built dir, and every traced file (both the path the trace reported and, for a + * symlink, the path it resolves to). Derived only from those paths, never from + * the deploy cwd, so the same inputs always produce the same bundle layout. + * + * A root that walks all the way to the filesystem root is refused rather than + * used: every containment check below it would accept anything, and assembly + * would stage arbitrary reachable files. The path that widened it is named so + * the author can see which dependency caused it. + */ +function stagingRootFor( + moduleDir: string, + dirPath: string, + tracedPaths: readonly string[], +): string { + let root = commonAncestor(moduleDir, dirPath); + if (isFilesystemRoot(root)) { + throw new Error( + `the build adapter's dir ("${dirPath}") and the directory of its module ("${moduleDir}") share no ` + + 'common ancestor below the filesystem root, so runtime dependency staging has no root to work from.', + ); + } + for (const traced of tracedPaths) { + const widened = commonAncestor(root, traced); + if (isFilesystemRoot(widened)) { + throw new Error( + `the runtime dependency trace reached ${traced}, which shares no directory with the build ` + + `output ("${dirPath}") below the filesystem root — staging from there would sweep in ` + + 'arbitrary files. Keep the traced dependency inside the project that holds the build output.', + ); + } + root = widened; + } + return root; +} + +/** Stages the explicit entry's runtime file graph beside the copied build dir. + * `nodeFileTrace` follows import/require/package metadata; it does not rewrite + * the app. Files already supplied by `dir` remain the author's verbatim copy. + * + * The trace itself runs from the filesystem root so nothing it finds is dropped + * for sitting outside a narrower base — a pnpm virtual store at the workspace + * root is outside the app directory, and dropping it would silently ship a + * bundle missing its dependencies. */ +async function stageRuntimeDependencies(options: { + readonly entryPath: string; + readonly dirPath: string; + readonly moduleDir: string; + readonly bundleDir: string; +}): Promise { + const [moduleDir, entryPath, dirPath] = await Promise.all([ + fs.promises.realpath(options.moduleDir), + fs.promises.realpath(options.entryPath), + fs.promises.realpath(options.dirPath), + ]); + const traced = await nodeFileTrace([entryPath], { + base: path.parse(moduleDir).root, + processCwd: moduleDir, + }); + + const tracedEntries = await Promise.all( + [...traced.fileList].sort().map(async (relative) => { + const source = path.resolve(path.parse(moduleDir).root, relative); + return { source, origin: await realPathOrSelf(source) }; + }), + ); + + const stagingRoot = stagingRootFor( + moduleDir, + dirPath, + tracedEntries.flatMap(({ source, origin }) => [source, origin]), + ); + + const stagedFrom = new Map(); + for (const { source, origin } of tracedEntries) { + if (isWithin(dirPath, source)) continue; + const destination = stagedRuntimePath(source, stagingRoot, options.bundleDir); + const alreadyStaged = stagedFrom.get(destination); + if (alreadyStaged !== undefined) { + if (alreadyStaged === origin) continue; + throw new Error( + 'two runtime dependencies stage to the same bundle path ' + + `("${path.relative(options.bundleDir, destination).split(path.sep).join('/')}"): ` + + `${alreadyStaged} and ${origin}. Staging keeps each file's path from its first node_modules ` + + 'segment, so two packages of the same name installed at different depths collapse onto one ' + + 'location — deduplicate them in your install so only one version is reachable.', + ); + } + if (await pathExists(destination)) continue; + await copyTracedEntry(source, destination, stagingRoot, options.bundleDir, dirPath); + stagedFrom.set(destination, origin); + } +} + /** * The working dir is cleared on every assemble, so it must not overlap the copy * source: inside it, the rm would delete the source before the copy; the other @@ -232,7 +380,17 @@ export async function assemble(input: AssembleInput): Promise { throw new Error(`esbuild produced no main.mjs in ${workDir}`); } - await runnable.copyInto(path.join(workDir, 'bundle')); + const bundleDir = path.join(workDir, 'bundle'); + await runnable.copyInto(bundleDir); + if (buildDescriptor.dir !== undefined) { + await stageRuntimeDependencies({ + entryPath: path.join(runnable.source, ...runnable.entry.split('/')), + dirPath: runnable.source, + moduleDir, + bundleDir, + }); + } + await assertBundleSymlinksStayInside(bundleDir); return { dir: workDir, diff --git a/packages/1-prisma-cloud/0-lowering/local-target/package.json b/packages/1-prisma-cloud/0-lowering/local-target/package.json index 2152430ad..79d22c5c2 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/package.json +++ b/packages/1-prisma-cloud/0-lowering/local-target/package.json @@ -13,6 +13,7 @@ "clean": "rm -rf dist" }, "dependencies": { + "@internal/bundle-paths": "workspace:0.6.0", "@internal/core": "workspace:0.6.0", "@internal/dev-emulators": "workspace:0.6.0", "@internal/lowering": "workspace:0.6.0", diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/artifact-extract.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/artifact-extract.test.ts index edc807ae5..4331df770 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/artifact-extract.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/artifact-extract.test.ts @@ -40,7 +40,10 @@ describe('extractComputeArtifact', () => { const bundleDir = makeBundle({ 'main.js': 'export default { run: async () => {} };', 'nested/asset.txt': 'hello world', + 'nested/run.sh': '#!/bin/sh\nexit 0\n', }); + fs.chmodSync(path.join(bundleDir, 'nested', 'run.sh'), 0o755); + fs.symlinkSync('asset.txt', path.join(bundleDir, 'nested', 'asset-link.txt')); const artifact = packageComputeArtifact({ id: 'auth', bundleDir, @@ -55,9 +58,17 @@ describe('extractComputeArtifact', () => { const extracted = readAll(destDir); expect(extracted['main.js']).toBe('export default { run: async () => {} };'); expect(extracted['nested/asset.txt']).toBe('hello world'); + expect(extracted['nested/asset-link.txt']).toBe('hello world'); + expect(fs.readlinkSync(path.join(destDir, 'nested', 'asset-link.txt'))).toBe('asset.txt'); + expect(fs.statSync(path.join(destDir, 'nested', 'run.sh')).mode & 0o100).toBe(0o100); expect(extracted['bootstrap.js']).toContain( - 'await main.run("auth", () => import("./server.js"));', + 'await main.run(boot.address, () => import(boot.appEntrypoint));', ); + expect(JSON.parse(extracted['compute.bootstrap.json'] ?? '{}')).toEqual({ + moduleEntrypoint: './main.js', + appEntrypoint: './server.js', + address: 'auth', + }); expect(JSON.parse(extracted['compute.manifest.json'] ?? '{}')).toEqual({ manifestVersion: '1', entrypoint: 'bootstrap.js', @@ -95,4 +106,65 @@ describe('extractComputeArtifact', () => { expect(() => extractComputeArtifact(tmpGz, destDir)).toThrow(/has type "Directory"/); }); + + test('rejects a symlink entry with an absolute target', () => { + // Hand-build a one-entry ustar archive with typeflag '2' and an absolute + // linkname: the resolve check alone would accept one that points inside + // the extraction dir, which dangles after the rename into place. + const header = Buffer.alloc(512); + header.write('link', 0, 100, 'utf8'); + header.write('0000644\0', 100, 8, 'utf8'); + header.write('0000000\0', 108, 8, 'utf8'); + header.write('0000000\0', 116, 8, 'utf8'); + header.write('00000000000\0', 124, 12, 'utf8'); + header.write('00000000000\0', 136, 12, 'utf8'); + header.write(' ', 148, 8, 'utf8'); + header.write('2', 156, 1, 'utf8'); // typeflag: symlink + header.write('/etc/passwd', 157, 100, 'utf8'); + header.write('ustar\0', 257, 6, 'utf8'); + header.write('00', 263, 2, 'utf8'); + let sum = 0; + for (const b of header) sum += b; + header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'utf8'); + const tar = Buffer.concat([header, Buffer.alloc(1024)]); + const gz = zlib.gzipSync(tar); + + const tmpGz = path.join( + fs.mkdtempSync(path.join(os.tmpdir(), 'artifact-extract-abs-link-')), + 'abs-link.tar.gz', + ); + fs.writeFileSync(tmpGz, gz); + const destDir = path.join(path.dirname(tmpGz), 'dest'); + + expect(() => extractComputeArtifact(tmpGz, destDir)).toThrow(/escapes the extraction/); + }); + + test('round-trips a safe symlink target longer than USTAR through PAX', () => { + const longTarget = `.pnpm/${'next-with-peer-context-'.repeat(5)}/node_modules/next`; + const longPath = `assets/${'a'.repeat(140)}/${'b'.repeat(120)}/asset.txt`; + const bundleDir = makeBundle({ + 'main.js': 'export default {};', + [`node_modules/${longTarget}/index.js`]: '// real', + [longPath]: 'long-path asset', + }); + fs.symlinkSync(longTarget, path.join(bundleDir, 'node_modules', 'next')); + const artifact = packageComputeArtifact({ + id: 'long-link', + bundleDir, + appEntry: 'server.js', + address: 'long-link', + }); + const destDir = path.join( + fs.mkdtempSync(path.join(os.tmpdir(), 'artifact-extract-long-link-')), + 'dest', + ); + + extractComputeArtifact(artifact.path, destDir); + + expect(fs.readlinkSync(path.join(destDir, 'node_modules', 'next'))).toBe(longTarget); + expect(fs.readFileSync(path.join(destDir, 'node_modules', 'next', 'index.js'), 'utf8')).toBe( + '// real', + ); + expect(fs.readFileSync(path.join(destDir, longPath), 'utf8')).toBe('long-path asset'); + }); }); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/artifact-extract.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/artifact-extract.ts index 33420649d..5365aa3b4 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/artifact-extract.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/artifact-extract.ts @@ -5,18 +5,19 @@ * emulation). Reading tar is commodity even though the writer is a * deterministic subset we own (fixed mtimes, sorted entries): the maintained * `tar` package (dependency razor) does the parsing; this module keeps only - * the pinned entry filtering (regular files only, reject links/devices, - * reject path escapes) and the directory-level temp-then-rename. + * the pinned entry filtering (regular files plus safe relative symlinks, + * reject devices/path escapes) and the directory-level temp-then-rename. */ import * as crypto from 'node:crypto'; import * as fs from 'node:fs'; import * as path from 'node:path'; +import { isWithin } from '@internal/bundle-paths'; import * as tar from 'tar'; function unsupportedEntryError(entryPath: string, type: string): Error { return new Error( - `compute artifact entry "${entryPath}" has type "${type}" — only regular files are ` + - 'supported; this artifact was not produced by packageComputeArtifact.', + `compute artifact entry "${entryPath}" has type "${type}" — only regular files and ` + + 'safe symlinks are supported; this artifact was not produced by packageComputeArtifact.', ); } @@ -28,6 +29,7 @@ function pathEscapeError(entryPath: string): Error { } const REGULAR_FILE_TYPES = new Set(['File', 'OldFile', 'ContiguousFile']); +const SYMLINK_TYPE = 'SymbolicLink'; /** * Extracts `tarGzPath` (a `packageComputeArtifact` tar.gz) into `destDir`, @@ -52,13 +54,26 @@ export function extractComputeArtifact(tarGzPath: string, destDir: string): void // below still names the entry in a pinned error rather than leaving // tar's own generic warning as the only signal. onentry: (entry) => { - if (!REGULAR_FILE_TYPES.has(entry.type)) { - throw unsupportedEntryError(entry.path, entry.type); - } const resolved = path.resolve(tmpDir, entry.path); - if (resolved !== tmpDir && !resolved.startsWith(`${tmpDir}${path.sep}`)) { + if (!isWithin(tmpDir, resolved)) { throw pathEscapeError(entry.path); } + if (REGULAR_FILE_TYPES.has(entry.type)) return; + if (entry.type !== SYMLINK_TYPE) { + throw unsupportedEntryError(entry.path, entry.type); + } + if (entry.linkpath === undefined) { + throw unsupportedEntryError(entry.path, `${entry.type} without a target`); + } + // An absolute target could point inside tmpDir and survive the + // resolve check, then dangle once the tree is renamed into place. + if (path.isAbsolute(entry.linkpath)) { + throw pathEscapeError(`${entry.path} -> ${entry.linkpath}`); + } + const linkTarget = path.resolve(path.dirname(resolved), entry.linkpath); + if (!isWithin(tmpDir, linkTarget)) { + throw pathEscapeError(`${entry.path} -> ${entry.linkpath}`); + } }, }); fs.rmSync(destDir, { recursive: true, force: true }); diff --git a/packages/1-prisma-cloud/0-lowering/lowering/package.json b/packages/1-prisma-cloud/0-lowering/lowering/package.json index 600e6c4ca..e588c0e66 100644 --- a/packages/1-prisma-cloud/0-lowering/lowering/package.json +++ b/packages/1-prisma-cloud/0-lowering/lowering/package.json @@ -18,6 +18,7 @@ "clean": "rm -rf dist" }, "dependencies": { + "@internal/bundle-paths": "workspace:0.6.0", "@internal/core": "workspace:0.6.0", "@internal/foundation": "workspace:0.6.0", "@prisma/management-api-sdk": "^1.60.0", diff --git a/packages/1-prisma-cloud/0-lowering/lowering/src/__tests__/artifact.test.ts b/packages/1-prisma-cloud/0-lowering/lowering/src/__tests__/artifact.test.ts index 82020346e..bb1681047 100644 --- a/packages/1-prisma-cloud/0-lowering/lowering/src/__tests__/artifact.test.ts +++ b/packages/1-prisma-cloud/0-lowering/lowering/src/__tests__/artifact.test.ts @@ -15,32 +15,72 @@ function makeBundle(files: Record): string { return dir; } -/** Un-gzips and lists the tar entry names + reads one entry's content, without a tar library. */ -function readTar(gz: Buffer): { names: string[]; read: (name: string) => string } { +/** Un-gzips and inspects the deterministic tar subset, without a tar library. */ +function readTar(gz: Buffer): { + names: string[]; + read: (name: string) => string; + link: (name: string) => string | undefined; + mode: (name: string) => number | undefined; +} { const tar = zlib.gunzipSync(gz); const names: string[] = []; const contents = new Map(); + const links = new Map(); + const modes = new Map(); + let nextPax: Record = {}; let offset = 0; while (offset + 512 <= tar.length) { const header = tar.subarray(offset, offset + 512); if (header.every((b) => b === 0)) break; // end-of-archive block const rawName = header.subarray(0, 100).toString('utf8').replace(/\0.*$/s, ''); const rawPrefix = header.subarray(345, 500).toString('utf8').replace(/\0.*$/s, ''); - const name = rawPrefix.length > 0 ? `${rawPrefix}/${rawName}` : rawName; + const headerName = rawPrefix.length > 0 ? `${rawPrefix}/${rawName}` : rawName; const size = Number.parseInt( header.subarray(124, 136).toString('utf8').replace(/\0.*$/s, '').trim(), 8, ); + const mode = Number.parseInt( + header.subarray(100, 108).toString('utf8').replace(/\0.*$/s, '').trim(), + 8, + ); + const typeflag = header.subarray(156, 157).toString('utf8'); + const linkname = header.subarray(157, 257).toString('utf8').replace(/\0.*$/s, ''); offset += 512; - contents.set(name, tar.subarray(offset, offset + size).toString('utf8')); + const content = tar.subarray(offset, offset + size); + if (typeflag === 'x') { + let paxOffset = 0; + while (paxOffset < content.length) { + const separator = content.indexOf(0x20, paxOffset); + const length = Number.parseInt( + content.subarray(paxOffset, separator).toString('ascii'), + 10, + ); + const record = content.subarray(separator + 1, paxOffset + length - 1).toString('utf8'); + const equals = record.indexOf('='); + nextPax[record.slice(0, equals)] = record.slice(equals + 1); + paxOffset += length; + } + offset += Math.ceil(size / 512) * 512; + continue; + } + const name = nextPax['path'] ?? headerName; + contents.set(name, content.toString('utf8')); + modes.set(name, mode); + if (typeflag === '2') links.set(name, nextPax['linkpath'] ?? linkname); names.push(name); + nextPax = {}; offset += Math.ceil(size / 512) * 512; } - return { names, read: (name: string) => contents.get(name) ?? '' }; + return { + names, + read: (name: string) => contents.get(name) ?? '', + link: (name: string) => links.get(name), + mode: (name: string) => modes.get(name), + }; } describe('packageComputeArtifact', () => { - test('prints a bootstrap that statically imports only the wrapper, then dynamically imports the app entry', () => { + test('prints a constant bootstrap that reads data before dynamically importing the wrapper and app', () => { const bundleDir = makeBundle({ 'main.js': 'export default { run: async () => {} };' }); const artifact = packageComputeArtifact({ @@ -53,8 +93,70 @@ describe('packageComputeArtifact', () => { const bootstrap = read('bootstrap.js'); const importLines = bootstrap.split('\n').filter((line) => /^\s*import\b/.test(line)); - expect(importLines).toEqual(['import main from "./main.js";']); - expect(bootstrap).toContain('await main.run("auth", () => import("./server.js"));'); + expect(importLines).toEqual(['import { readFile } from "node:fs/promises";']); + expect(bootstrap).toContain('for (const constructor of [URL, URLSearchParams])'); + expect(bootstrap).toContain('Object.defineProperty(this, inspect'); + expect(bootstrap).toContain('const main = (await import(boot.moduleEntrypoint)).default;'); + expect(bootstrap).toContain('await main.run(boot.address, () => import(boot.appEntrypoint));'); + expect(JSON.parse(read('compute.bootstrap.json'))).toEqual({ + moduleEntrypoint: './main.js', + appEntrypoint: './server.js', + address: 'auth', + }); + }); + + test('executes the generated data-backed bootstrap under Bun', () => { + const bundleDir = makeBundle({ + 'main.js': + 'export default { run: async (address, boot) => { await boot(); console.log(`address:${address}`); } };', + 'server.js': 'console.log("server:booted");', + }); + const artifact = packageComputeArtifact({ + id: 'executable', + bundleDir, + appEntry: 'server.js', + address: 'auth', + }); + const archive = readTar(fs.readFileSync(artifact.path)); + const runtimeDir = fs.mkdtempSync(path.join(os.tmpdir(), 'artifact-bootstrap-run-')); + for (const name of ['bootstrap.js', 'compute.bootstrap.json', 'main.js', 'server.js']) { + fs.writeFileSync(path.join(runtimeDir, name), archive.read(name)); + } + + const child = Bun.spawnSync({ cmd: [process.execPath, 'bootstrap.js'], cwd: runtimeDir }); + const stdout = new TextDecoder().decode(child.stdout); + const stderr = new TextDecoder().decode(child.stderr); + + expect(child.exitCode, stderr).toBe(0); + expect(stdout).toContain('server:booted'); + expect(stdout).toContain('address:auth'); + }); + + test('keeps caller-provided entry and address strings out of executable JavaScript', () => { + const marker = 'globalThis.COMPROMISED = true'; + const bundleEntry = `main"; ${marker}; ".js`; + const appEntry = `server"; ${marker}; ".js`; + const address = `auth"); ${marker}; ("`; + const bundleDir = makeBundle({ + [bundleEntry]: 'export default {};', + [appEntry]: 'export default {};', + }); + + const artifact = packageComputeArtifact({ + id: 'hostile-data', + bundleDir, + bundleEntry, + appEntry, + address, + }); + const { read } = readTar(fs.readFileSync(artifact.path)); + + expect(read('bootstrap.js')).not.toContain(marker); + expect(JSON.parse(read('compute.bootstrap.json'))).toEqual({ + moduleEntrypoint: `./${bundleEntry}`, + appEntrypoint: `./${appEntry}`, + address, + }); }); test('writes compute.manifest.json with entrypoint bootstrap.js and the packaged address', () => { @@ -86,7 +188,7 @@ describe('packageComputeArtifact', () => { }); const { read } = readTar(fs.readFileSync(artifact.path)); - expect(read('bootstrap.js')).toContain('import main from "./main.mjs";'); + expect(JSON.parse(read('compute.bootstrap.json')).moduleEntrypoint).toBe('./main.mjs'); }); test('packaging twice with identical inputs yields an identical sha256 AND an identical path (redeploy noops)', () => { @@ -150,7 +252,7 @@ describe('packageComputeArtifact', () => { expect(artifact.path).toContain(`prisma-composer-compute-${String(os.userInfo().uid)}`); }); - test('a different address changes the hash (the bootstrap is address-specific)', () => { + test('a different address changes the hash (the bootstrap data is address-specific)', () => { const bundleDir = makeBundle({ 'main.js': 'export default {};' }); const a = packageComputeArtifact({ @@ -169,7 +271,7 @@ describe('packageComputeArtifact', () => { expect(a.sha256).not.toBe(b.sha256); }); - test('a different appEntry changes the hash (the bootstrap bakes in the boot import)', () => { + test('a different appEntry changes the hash (the bootstrap data names the boot import)', () => { const bundleDir = makeBundle({ 'main.js': 'export default {};' }); const a = packageComputeArtifact({ @@ -204,6 +306,7 @@ describe('packageComputeArtifact', () => { 'b.txt', 'bootstrap.js', 'bunfig.toml', + 'compute.bootstrap.json', 'compute.manifest.json', 'main.js', ]); @@ -221,18 +324,158 @@ describe('packageComputeArtifact', () => { expect(read('bunfig.toml')).toContain('auto = "disable"'); }); - test('a symlink in the bundle is a hard error naming the path and the fix (flat bundles only)', () => { + test('preserves a framework-produced symlink whose target stays inside the bundle', () => { const bundleDir = makeBundle({ 'main.js': 'export default {};', 'node_modules/real/index.js': '// real', }); - // A bun/pnpm-shaped relative dir-symlink, the kind a Next standalone tree - // is full of — the framework must reject it, not dereference it. + // A bun/Next-standalone-shaped relative directory symlink. fs.symlinkSync('real', path.join(bundleDir, 'node_modules', 'link')); + const artifact = packageComputeArtifact({ + id: 'auth', + bundleDir, + appEntry: 'server.js', + address: 'auth', + }); + const archive = readTar(fs.readFileSync(artifact.path)); + + expect(archive.names).toContain('node_modules/link'); + expect(archive.link('node_modules/link')).toBe('real'); + }); + + test('rewrites an absolute in-bundle symlink to a deploy-relative target', () => { + const bundleDir = makeBundle({ + 'main.js': 'export default {};', + 'node_modules/real/index.js': '// real', + }); + fs.symlinkSync( + path.join(bundleDir, 'node_modules', 'real'), + path.join(bundleDir, 'node_modules', 'link'), + ); + + const artifact = packageComputeArtifact({ + id: 'auth', + bundleDir, + appEntry: 'server.js', + address: 'auth', + }); + const archive = readTar(fs.readFileSync(artifact.path)); + + expect(archive.link('node_modules/link')).toBe('real'); + expect(archive.link('node_modules/link')).not.toContain(bundleDir); + }); + + test('preserves executable mode for staged runtime files', () => { + const bundleDir = makeBundle({ + 'main.js': 'export default {};', + 'node_modules/tool/bin/run': '#!/bin/sh\nexit 0\n', + }); + fs.chmodSync(path.join(bundleDir, 'node_modules', 'tool', 'bin', 'run'), 0o755); + + const artifact = packageComputeArtifact({ + id: 'auth', + bundleDir, + appEntry: 'server.js', + address: 'auth', + }); + const archive = readTar(fs.readFileSync(artifact.path)); + + expect(archive.mode('node_modules/tool/bin/run')).toBe(0o755); + expect(archive.mode('main.js')).toBe(0o644); + }); + + test('uses a PAX linkpath for a safe framework symlink target longer than USTAR allows', () => { + const longTarget = `.pnpm/${'next-with-peer-context-'.repeat(5)}/node_modules/next`; + const bundleDir = makeBundle({ + 'main.js': 'export default {};', + [`node_modules/${longTarget}/index.js`]: '// real', + }); + fs.symlinkSync(longTarget, path.join(bundleDir, 'node_modules', 'next')); + + const artifact = packageComputeArtifact({ + id: 'auth', + bundleDir, + appEntry: 'server.js', + address: 'auth', + }); + const archive = readTar(fs.readFileSync(artifact.path)); + + expect(Buffer.byteLength(longTarget, 'utf8')).toBeGreaterThan(100); + expect(archive.link('node_modules/next')).toBe(longTarget); + }); + + test('uses a PAX path for a bundle file whose path cannot fit USTAR fields', () => { + const longPath = `assets/${'a'.repeat(140)}/${'b'.repeat(120)}/asset.txt`; + const bundleDir = makeBundle({ + 'main.js': 'export default {};', + [longPath]: 'long-path asset', + }); + + const artifact = packageComputeArtifact({ + id: 'long-path', + bundleDir, + appEntry: 'server.js', + address: 'long-path', + }); + const archive = readTar(fs.readFileSync(artifact.path)); + + expect(Buffer.byteLength(longPath, 'utf8')).toBeGreaterThan(255); + expect(archive.read(longPath)).toBe('long-path asset'); + }); + + test('rejects a symlink whose real target escapes the assembled bundle', () => { + const parent = fs.mkdtempSync(path.join(os.tmpdir(), 'artifact-symlink-escape-')); + const bundleDir = path.join(parent, 'bundle'); + fs.mkdirSync(bundleDir); + fs.writeFileSync(path.join(bundleDir, 'main.js'), 'export default {};'); + fs.writeFileSync(path.join(parent, 'secret.txt'), 'must not ship'); + fs.symlinkSync('../secret.txt', path.join(bundleDir, 'escaped')); + + expect(() => + packageComputeArtifact({ id: 'auth', bundleDir, appEntry: 'server.js', address: 'auth' }), + ).toThrow(/symlink at escaped escapes the bundle root/); + }); + + test('rejects a link whose literal target leaves the bundle and re-enters through an outside alias', () => { + const parent = fs.mkdtempSync(path.join(os.tmpdir(), 'artifact-symlink-alias-')); + const bundleDir = path.join(parent, 'bundle'); + fs.mkdirSync(path.join(bundleDir, 'node_modules', 'pkg'), { recursive: true }); + fs.writeFileSync(path.join(bundleDir, 'main.js'), 'export default {};'); + fs.writeFileSync(path.join(bundleDir, 'node_modules', 'pkg', 'index.js'), '// real'); + // An alias OUTSIDE the bundle pointing back at it: the link below resolves + // (realpath) inside the bundle, but the literal target archived into the tar + // walks out through the alias — which every extractor rejects. + fs.symlinkSync(bundleDir, path.join(parent, 'alias')); + fs.symlinkSync( + path.join('..', '..', 'alias', 'node_modules', 'pkg'), + path.join(bundleDir, 'node_modules', 'aliased'), + ); + + expect(() => + packageComputeArtifact({ id: 'auth', bundleDir, appEntry: 'server.js', address: 'auth' }), + ).toThrow(/symlink at node_modules\/aliased has a target that leaves the bundle/); + }); + + test('rejects a dangling symlink instead of emitting an unusable artifact', () => { + const bundleDir = makeBundle({ 'main.js': 'export default {};' }); + fs.symlinkSync('missing.js', path.join(bundleDir, 'dangling')); + + expect(() => + packageComputeArtifact({ id: 'auth', bundleDir, appEntry: 'server.js', address: 'auth' }), + ).toThrow(/symlink at dangling is dangling/); + }); + + test('rejects a FIFO before attempting to read it', () => { + if (process.platform === 'win32') return; + const bundleDir = makeBundle({ 'main.js': 'export default {};' }); + const fifo = path.join(bundleDir, 'runtime.pipe'); + const created = Bun.spawnSync({ cmd: ['mkfifo', fifo] }); + expect(created.exitCode).toBe(0); + expect(() => packageComputeArtifact({ id: 'auth', bundleDir, appEntry: 'server.js', address: 'auth' }), - ).toThrow(/symlink at node_modules\/link .* deploy bundles must be flat/); + ).toThrow(/unsupported filesystem entry: runtime\.pipe/); }); test('a missing bundle dir (destroy before any build) returns a placeholder instead of throwing', () => { diff --git a/packages/1-prisma-cloud/0-lowering/lowering/src/compute/artifact.ts b/packages/1-prisma-cloud/0-lowering/lowering/src/compute/artifact.ts index de79c3eed..82c1dc368 100644 --- a/packages/1-prisma-cloud/0-lowering/lowering/src/compute/artifact.ts +++ b/packages/1-prisma-cloud/0-lowering/lowering/src/compute/artifact.ts @@ -11,6 +11,7 @@ import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; import * as zlib from 'node:zlib'; +import { isWithin } from '@internal/bundle-paths'; export interface PackageComputeArtifactOptions { /** The service's provision id — namespaces the temp output path. */ @@ -20,11 +21,11 @@ export interface PackageComputeArtifactOptions { /** The Prisma App wrapper file inside bundleDir. Defaults to main.js|main.mjs. */ readonly bundleEntry?: string; /** - * The app's own runnable inside bundleDir (e.g. "server.js") — baked into the - * bootstrap's boot import: `main.run(address, () => import("./"))`. + * The app's own runnable inside bundleDir (e.g. "server.js") — recorded in + * the bootstrap data consumed by the generated bootstrap. */ readonly appEntry: string; - /** The node's deployment address — baked into the printed bootstrap. */ + /** The node's deployment address — recorded as intrinsic artifact metadata. */ readonly address: string; } @@ -45,32 +46,82 @@ function resolveEntry(bundleDir: string, entry: string | undefined): string { return found; } -/** All files under `dir`, as dir-relative POSIX paths, in sorted order. A - * symlink is a hard error: deploy bundles must be flat (ADR-0005), and the - * user's build owns flattening — dereferencing here would relink the tree and - * risk packaging files from outside it. */ -function walkFiles(dir: string): string[] { - const out: string[] = []; +type BundleEntry = + | { readonly relPath: string; readonly type: 'file'; readonly executable: boolean } + | { readonly relPath: string; readonly type: 'symlink'; readonly linkname: string }; + +function compareArchivePaths(left: { relPath: string }, right: { relPath: string }): number { + return Buffer.compare(Buffer.from(left.relPath, 'utf8'), Buffer.from(right.relPath, 'utf8')); +} + +/** All files and safe symlinks under `dir`, as dir-relative POSIX paths, in + * sorted order. Symlinks are preserved as links — never dereferenced — after + * their real target is proven to remain inside the bundle root. This accepts + * framework-produced trees such as Next standalone while retaining ADR-0005's + * boundary against packaging arbitrary files from the deploy machine. */ +function walkEntries(dir: string): BundleEntry[] { + const out: BundleEntry[] = []; + const realRoot = fs.realpathSync(dir); const visit = (sub: string): void => { for (const entry of fs.readdirSync(path.join(dir, sub), { withFileTypes: true })) { const rel = sub.length > 0 ? `${sub}/${entry.name}` : entry.name; if (entry.isSymbolicLink()) { - throw new Error( - `bundle contains a symlink at ${rel} — deploy bundles must be flat; ` + - 'materialize links in your build (e.g. cp -RL) so the tree is self-contained.', - ); + const symlinkPath = path.join(dir, ...rel.split('/')); + const target = fs.readlinkSync(symlinkPath); + if (path.sep === '/' && target.includes('\\')) { + throw new Error( + `bundle symlink at ${rel} has an unsupported backslash target: ${target}`, + ); + } + let realTarget: string; + try { + realTarget = fs.realpathSync(path.resolve(path.dirname(symlinkPath), target)); + } catch { + throw new Error(`bundle symlink at ${rel} is dangling: ${target}`); + } + if (!isWithin(realRoot, realTarget)) { + throw new Error( + `bundle symlink at ${rel} escapes the bundle root: ${target} — deploy artifacts may only preserve links whose targets are inside the assembled bundle.`, + ); + } + const linkname = ( + path.isAbsolute(target) + ? path.relative(fs.realpathSync(path.dirname(symlinkPath)), realTarget) + : target + ) + .split(path.sep) + .join('/'); + // The realpath check above proves where the link points on THIS machine; + // the archived link is the literal string, which every extractor + // re-checks lexically against the unpack root. A target that leaves the + // bundle and re-enters through an out-of-bundle alias passes the first + // check and fails the second, so reject it here — at the cause. + const lexicalTarget = path.resolve(path.dirname(symlinkPath), ...linkname.split('/')); + if (!isWithin(dir, lexicalTarget)) { + throw new Error( + `bundle symlink at ${rel} has a target that leaves the bundle: ${linkname} — its resolved target is inside the bundle, but the link path itself walks outside and re-enters, which every extractor rejects. Point the link at the in-bundle path directly.`, + ); + } + out.push({ relPath: rel, type: 'symlink', linkname }); + continue; } if (entry.isDirectory()) visit(rel); - else out.push(rel); + else if (entry.isFile()) { + const mode = fs.statSync(path.join(dir, ...rel.split('/'))).mode; + out.push({ relPath: rel, type: 'file', executable: (mode & 0o100) !== 0 }); + } else { + throw new Error(`bundle contains an unsupported filesystem entry: ${rel}`); + } } }; visit(''); - return out.sort(); + return out.sort(compareArchivePaths); } -// ——— A minimal, deterministic USTAR writer: fixed mtime (epoch 0), fixed -// mode/uid/gid, sorted entries. gzip (node:zlib) is itself deterministic — -// its header carries no timestamp — so byte-identical inputs always hash +// ——— A minimal, deterministic USTAR + POSIX PAX writer: fixed mtime (epoch 0), +// fixed mode/uid/gid, sorted entries. PAX is used only when a path or symlink +// target exceeds USTAR's fixed fields. gzip (node:zlib) is itself deterministic +// — its header carries no timestamp — so byte-identical inputs always hash // identically. function octal(value: number, length: number): string { @@ -91,17 +142,47 @@ function splitUstarPath(relPath: string): { name: string; prefix: string } { throw new Error(`path too long for a ustar tar entry: ${relPath}`); } -function ustarHeader(relPath: string, size: number): Buffer { +function paxRecord(key: 'path' | 'linkpath', value: string): string { + const payload = ` ${key}=${value}\n`; + let length = Buffer.byteLength(payload, 'utf8') + 1; + while (true) { + const record = `${length}${payload}`; + const actualLength = Buffer.byteLength(record, 'utf8'); + if (actualLength === length) return record; + length = actualLength; + } +} + +function ustarPathOrPlaceholder(relPath: string): { path: string; paxPath?: string } { + try { + splitUstarPath(relPath); + return { path: relPath }; + } catch { + const digest = crypto.createHash('sha256').update(relPath).digest('hex').slice(0, 32); + return { path: `PaxEntries/${digest}`, paxPath: relPath }; + } +} + +function ustarHeader( + relPath: string, + size: number, + options: { + readonly mode: number; + readonly typeflag: '0' | '2' | 'x'; + readonly linkname?: string; + }, +): Buffer { const { name, prefix } = splitUstarPath(relPath); const buf = Buffer.alloc(512); buf.write(name, 0, 100, 'utf8'); - buf.write(octal(0o644, 8), 100, 8, 'utf8'); // mode + buf.write(octal(options.mode, 8), 100, 8, 'utf8'); buf.write(octal(0, 8), 108, 8, 'utf8'); // uid buf.write(octal(0, 8), 116, 8, 'utf8'); // gid buf.write(octal(size, 12), 124, 12, 'utf8'); buf.write(octal(0, 12), 136, 12, 'utf8'); // mtime: fixed at epoch 0 buf.write(' ', 148, 8, 'utf8'); // chksum placeholder (8 spaces) - buf.write('0', 156, 1, 'utf8'); // typeflag: regular file + buf.write(options.typeflag, 156, 1, 'utf8'); + if (options.linkname !== undefined) buf.write(options.linkname, 157, 100, 'utf8'); buf.write('ustar\0', 257, 6, 'utf8'); buf.write('00', 263, 2, 'utf8'); buf.write(prefix, 345, 155, 'utf8'); @@ -113,15 +194,59 @@ function ustarHeader(relPath: string, size: number): Buffer { } function createDeterministicTarGz( - entries: readonly { relPath: string; content: Buffer }[], + entries: readonly ( + | { + readonly relPath: string; + readonly type: 'file'; + readonly content: Buffer; + readonly mode: number; + } + | { readonly relPath: string; readonly type: 'symlink'; readonly linkname: string } + )[], ): Buffer { - const sorted = [...entries].sort((a, b) => a.relPath.localeCompare(b.relPath)); + const sorted = [...entries].sort(compareArchivePaths); const chunks: Buffer[] = []; for (const entry of sorted) { - chunks.push(ustarHeader(entry.relPath, entry.content.length)); - chunks.push(entry.content); - const pad = (512 - (entry.content.length % 512)) % 512; - if (pad > 0) chunks.push(Buffer.alloc(pad)); + const archivePath = ustarPathOrPlaceholder(entry.relPath); + const pax = [archivePath.paxPath === undefined ? '' : paxRecord('path', archivePath.paxPath)]; + if (entry.type === 'symlink' && Buffer.byteLength(entry.linkname, 'utf8') > 100) { + pax.push(paxRecord('linkpath', entry.linkname)); + } + const paxContent = Buffer.from(pax.join(''), 'utf8'); + if (paxContent.length > 0) { + const digest = crypto.createHash('sha256').update(entry.relPath).digest('hex').slice(0, 32); + chunks.push( + ustarHeader(`PaxHeaders/${digest}`, paxContent.length, { + mode: 0o644, + typeflag: 'x', + }), + ); + chunks.push(paxContent); + const paxPad = (512 - (paxContent.length % 512)) % 512; + if (paxPad > 0) chunks.push(Buffer.alloc(paxPad)); + } + if (entry.type === 'symlink') { + chunks.push( + ustarHeader(archivePath.path, 0, { + mode: 0o777, + typeflag: '2', + // A non-empty legacy field keeps libarchive/GNU tar applying the PAX + // linkpath override; this is the conventional long-link placeholder. + linkname: + Buffer.byteLength(entry.linkname, 'utf8') <= 100 ? entry.linkname : '././@LongSymLink', + }), + ); + } else { + chunks.push( + ustarHeader(archivePath.path, entry.content.length, { + mode: entry.mode, + typeflag: '0', + }), + ); + chunks.push(entry.content); + const pad = (512 - (entry.content.length % 512)) % 512; + if (pad > 0) chunks.push(Buffer.alloc(pad)); + } } chunks.push(Buffer.alloc(1024)); // end-of-archive: two zero blocks return zlib.gzipSync(Buffer.concat(chunks)); @@ -144,26 +269,89 @@ export function packageComputeArtifact(opts: PackageComputeArtifactOptions): Com } const entryFile = resolveEntry(opts.bundleDir, opts.bundleEntry); - const bootstrap = `import main from "./${entryFile}";\nawait main.run(${JSON.stringify(opts.address)}, () => import(${JSON.stringify(`./${opts.appEntry}`)}));\n`; - // `address` is intrinsic artifact metadata, not dev config — bootstrap.js - // above already bakes `main.run(address, …)`, so the manifest carrying it - // too is the same fact recorded twice: once for the boot path, once for a + // Keep all caller-provided strings in JSON data rather than interpolating + // them into executable JavaScript. The generated bootstrap is constant code, + // so an unusual but valid address or entry filename cannot become code. + const bootstrapData = `${JSON.stringify( + { + moduleEntrypoint: `./${entryFile}`, + appEntrypoint: `./${opts.appEntry}`, + address: opts.address, + }, + null, + 2, + )}\n`; + const bootstrap = `import { readFile } from "node:fs/promises"; + +const boot = JSON.parse( + await readFile(new URL("./compute.bootstrap.json", import.meta.url), "utf8"), +); + +// Compute currently boots JavaScript with Bun. Its URL and URLSearchParams +// implementations accept Object.defineProperty but reject assignment to +// Node's custom-inspect symbol. SvelteKit assigns that symbol while creating a +// tracked request URL, so install a narrow setter that materializes the same +// own property Node would. Remove this compatibility shim when the upstream +// Alchemy Compute runtime owns the equivalent normalization. +if (process.versions.bun !== undefined) { + const inspect = Symbol.for("nodejs.util.inspect.custom"); + for (const constructor of [URL, URLSearchParams]) { + const inherited = constructor.prototype[inspect]; + Object.defineProperty(constructor.prototype, inspect, { + configurable: true, + get() { return inherited; }, + set(value) { + Object.defineProperty(this, inspect, { configurable: true, value, writable: true }); + }, + }); + } +} + +const main = (await import(boot.moduleEntrypoint)).default; +await main.run(boot.address, () => import(boot.appEntrypoint)); +`; + // `address` is intrinsic artifact metadata, not dev config. It is recorded + // twice: bootstrap data drives the boot path, while the manifest serves a // reader that needs the address WITHOUT executing the artifact (the local // Deployment provider, which learns nothing else about dev — local-dev - // spec § 4). No version bump — no consumer needs protecting from a new - // field; the platform still reads only `entrypoint`. + // spec § 4). The platform still reads only `entrypoint` from the manifest. const manifest = `${JSON.stringify( { manifestVersion: MANIFEST_VERSION, entrypoint: 'bootstrap.js', address: opts.address }, null, 2, )}\n`; - const files = walkFiles(opts.bundleDir).map((relPath) => ({ - relPath, - content: fs.readFileSync(path.join(opts.bundleDir, relPath)), - })); - files.push({ relPath: 'bootstrap.js', content: Buffer.from(bootstrap, 'utf8') }); - files.push({ relPath: 'compute.manifest.json', content: Buffer.from(manifest, 'utf8') }); + const files: ( + | { relPath: string; type: 'file'; content: Buffer; mode: number } + | { relPath: string; type: 'symlink'; linkname: string } + )[] = walkEntries(opts.bundleDir).map((entry) => + entry.type === 'symlink' + ? entry + : { + relPath: entry.relPath, + type: 'file', + content: fs.readFileSync(path.join(opts.bundleDir, ...entry.relPath.split('/'))), + mode: entry.executable ? 0o755 : 0o644, + }, + ); + files.push({ + relPath: 'bootstrap.js', + type: 'file', + content: Buffer.from(bootstrap, 'utf8'), + mode: 0o644, + }); + files.push({ + relPath: 'compute.bootstrap.json', + type: 'file', + content: Buffer.from(bootstrapData, 'utf8'), + mode: 0o644, + }); + files.push({ + relPath: 'compute.manifest.json', + type: 'file', + content: Buffer.from(manifest, 'utf8'), + mode: 0o644, + }); // Disable bun's runtime auto-install for every Compute artifact. An app's // build produces a self-contained entry with its dependencies inlined // (ADR-0005), so nothing needs fetching at boot; this guards against a stray @@ -173,7 +361,9 @@ export function packageComputeArtifact(opts: PackageComputeArtifactOptions): Com // at boot. files.push({ relPath: 'bunfig.toml', + type: 'file', content: Buffer.from('[install]\nauto = "disable"\n', 'utf8'), + mode: 0o644, }); const gz = createDeterministicTarGz(files); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts index 1717c95d0..d149a9b0a 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts @@ -51,6 +51,17 @@ describe('bootstrapService', () => { expect(portAtBoot).toBe('4711'); }); + test('defaults HOST to the Compute listen-all address before boot', async () => { + const app = compute({ name: 'web', deps: {}, build }); + let hostAtBoot: string | undefined; + await withEnv({ HOST: undefined, PORT: undefined, COMPOSER_PORT: undefined }, () => + bootstrapService(app, { service: { port: 4711 }, inputs: {} }, async () => { + hostAtBoot = process.env['HOST']; + }), + ); + expect(hostAtBoot).toBe('0.0.0.0'); + }); + test('writes the input document row so input() reads it like a deployed boot', async () => { const app = compute({ name: 'web', diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts index c161ec9ef..610537c86 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts @@ -36,9 +36,15 @@ const build = { }; /** Sets env vars for the duration of `fn`, restoring whatever was there before. */ -async function withEnv(values: Record, fn: () => Promise | T): Promise { +async function withEnv( + values: Record, + fn: () => Promise | T, +): Promise { const previous = new Map(Object.keys(values).map((k) => [k, process.env[k]])); - for (const [k, v] of Object.entries(values)) process.env[k] = v; + for (const [k, v] of Object.entries(values)) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } try { return await fn(); } finally { @@ -404,6 +410,26 @@ describe('compute().run(address, boot) → load() — the round trip', () => { expect(portAtBoot).toBe('3000'); }); + test('run() binds framework servers to the Compute network unless the author set HOST', async () => { + const app = compute({ name: 'ingest', deps: {}, build }); + let defaultHost: string | undefined; + let explicitHost: string | undefined; + + await withEnv({ COMPOSER_INGEST_PORT: '', COMPOSER_PORT: '', HOST: undefined }, () => + app.run('ingest', async () => { + defaultHost = process.env['HOST']; + }), + ); + await withEnv({ COMPOSER_INGEST_PORT: '', COMPOSER_PORT: '', HOST: '127.0.0.1' }, () => + app.run('ingest', async () => { + explicitHost = process.env['HOST']; + }), + ); + + expect(defaultHost).toBe('0.0.0.0'); + expect(explicitHost).toBe('127.0.0.1'); + }); + // Reserved provider params (ADR-0031) are the provider-side counterpart of // a declared param: `run()` validates each one's address-scoped row against // its own schema — the same `coerce` a declared param takes — and re-stashes diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts index 235a3477f..5da0725c1 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts @@ -133,13 +133,13 @@ describe('invariant 4: environment touches are confined to the config serializer }); expect(hits.sort((a, b) => a.file.localeCompare(b.file))).toEqual([ - { file: 'compute.ts', count: 1 }, + { file: 'compute.ts', count: 2 }, { file: 'container.ts', count: 3 }, { file: 'control/extension.ts', count: 2 }, { file: 'local-target/preflight.ts', count: 2 }, { file: 'preflight.ts', count: 2 }, { file: 'serializer.ts', count: 12 }, - { file: 'testing.ts', count: 3 }, + { file: 'testing.ts', count: 4 }, ]); }); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/compute.ts b/packages/1-prisma-cloud/1-extensions/target/src/compute.ts index 4bb562792..b9aee38f7 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/compute.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/compute.ts @@ -105,6 +105,10 @@ export class ComputeService< // reserved `port` param the same way serialize does (descriptors/compute.ts). const port = config.service['port']; if (typeof port === 'number') process.env['PORT'] = String(port); + // Compute routes to the workload over its network interface, not loopback. + // Astro's Node adapter defaults HOST to localhost; preserve an explicit + // author value, otherwise supply the framework-neutral listen-all address. + process.env['HOST'] ??= '0.0.0.0'; return boot(); } diff --git a/packages/1-prisma-cloud/1-extensions/target/src/testing.ts b/packages/1-prisma-cloud/1-extensions/target/src/testing.ts index afdd87496..c8dcd2c00 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/testing.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/testing.ts @@ -76,6 +76,7 @@ export async function bootstrapService