Skip to content

Align publish flows with KA lifecycle#1275

Open
Jurij89 wants to merge 7 commits into
feat/api-agent-tooling-cleanupfrom
codex/1260-lifecycle-async-publish
Open

Align publish flows with KA lifecycle#1275
Jurij89 wants to merge 7 commits into
feat/api-agent-tooling-cleanupfrom
codex/1260-lifecycle-async-publish

Conversation

@Jurij89

@Jurij89 Jurij89 commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Align app-facing write/publish flows with the named Knowledge Asset lifecycle by adding POST /api/knowledge-assets/:name/vm/publish-async as the async VM publish entrypoint.
  • Remove product-facing loose SWM write/enqueue surfaces and migrate source-worker, clients, adapters, benchmarks, node-ui, docs, and devnet scripts to lifecycle routes.
  • Harden async publisher intent/recovery semantics so named KA jobs bind to current full-share state, reject stale/unsealed intent, and avoid protected stranded recovery states.
  • Keep SWM receive/gossip/ACK/catchup substrate internals intact while moving batch-rejection reporting to lifecycle-shaped KA persistence.

Related

Files changed

File What
packages/cli/src/daemon/routes/knowledge-assets.ts Adds named KA vm/publish-async and lifecycle-shaped batch rejection route.
packages/cli/src/daemon/routes/memory.ts Removes app-facing loose SWM write/conditional-write/report-batch-rejection behavior from the old URL surface.
packages/cli/src/daemon/routes/publisher.ts Removes raw public enqueue behavior while preserving job management/control-plane reads.
packages/agent/src/dkg-agent-publish.ts / packages/agent/src/dkg-agent.ts Resolves current finalized full-share publish intent and exposes stable share operation metadata.
packages/publisher/src/* Adds named KA job typing, intent/idempotency checks, lifecycle executor handling, and recovery hardening.
packages/cli/src/api-client.ts, source-worker runners, OpenClaw adapter, node-ui, network sim, benchmarks Migrates product callers to named KA lifecycle APIs.
scripts/*.sh and devnet helpers Migrates executable devnet flows off deleted legacy routes and preserves full-batch verification semantics.
packages/*/test/* Adds/updates route deletion, async publish, metadata, source-worker, adapter, node-ui, and devnet-helper regression coverage.
README.md, ARCHITECTURE.md, docs/**, agent-docs/** Updates product docs and includes the reviewed plan, decisions, implementation log, and lessons.

Test plan

  • pnpm --filter @origintrail-official/dkg-publisher run build
  • pnpm --filter @origintrail-official/dkg-agent run build
  • pnpm --filter @origintrail-official/dkg run build
  • pnpm --filter @origintrail-official/dkg-node-ui run build
  • pnpm --filter @origintrail-official/dkg-publisher test -- test/async-lift-publisher.test.ts test/metadata.test.ts --reporter verbose
  • pnpm --filter @origintrail-official/dkg test -- test/api-client.test.ts test/source-worker-daemon-client.test.ts test/source-worker-runner.test.ts --reporter verbose
  • pnpm --filter @origintrail-official/dkg test -- test/daemon-http-behavior-extra.test.ts --reporter verbose
  • pnpm --filter @origintrail-official/dkg test -- test/daemon-http-behavior-extra.test.ts test/issue-306-787-write-quad-validation.test.ts test/assertion-cli-smoke.test.ts test/memory-graph-events.test.ts --reporter verbose
  • pnpm --filter @origintrail-official/dkg-agent test -- test/e2e-memory-layers.test.ts -t 'skipSeal share stays unsealed' --reporter verbose
  • pnpm --filter @origintrail-official/dkg-node-ui test -- test/chat-memory.test.ts test/chat-memory-persistence-regression.test.ts --reporter verbose
  • bash -n scripts/*.sh
  • node --check scripts/testnet-publish-stress/publish-loop.mjs; node --check scripts/dkg-v10-implementation-time-accounting.mjs; node --check scripts/dkg-v10-decisions.mjs; node --check scripts/dkg-v10-decision-time-accounting.mjs
  • git diff --check
  • Post-rebase: pnpm --filter @origintrail-official/dkg run build
  • Post-rebase: pnpm --filter @origintrail-official/dkg-publisher test -- test/async-lift-publisher.test.ts test/metadata.test.ts --reporter verbose
  • Post-rebase: pnpm --filter @origintrail-official/dkg test -- test/api-client.test.ts test/daemon-http-behavior-extra.test.ts --reporter verbose
  • Post-rebase: targeted legacy-surface scan for active /api/shared-memory/* write/publish and raw /api/publisher/enqueue callers.

@Jurij89 Jurij89 force-pushed the codex/1260-lifecycle-async-publish branch 2 times, most recently from 1f13a36 to 8c9bec6 Compare June 21, 2026 20:51
@Jurij89 Jurij89 marked this pull request as ready for review June 21, 2026 20:54
Comment thread packages/agent/src/dkg-agent-publish.ts Outdated

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Verification reviewer failed: 2026-06-21T21:19:13.063828Z ERROR rmcp::transport::worker: worker quit with fatal: Transport channel closed, when UnexpectedServerResponse("HTTP 401: {\n "error": {\n "message": "Your authentication token has been invalidated. Please try signing in again.",\n "type": "invalid_request_error",\n "code": "token_invalidated",\n "param": null\n },\n "status": 401\n}")

Details omitted because the failure output was 1469 characters. Check the local Review Agent logs for the reviewer event and output files.

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Business logic reviewer failed: 2026-06-21T21:19:37.183386Z ERROR rmcp::transport::worker: worker quit with fatal: Transport channel closed, when UnexpectedServerResponse("HTTP 401: {\n "error": {\n "message": "Your authentication token has been invalidated. Please try signing in again.",\n "type": "invalid_request_error",\n "code": "token_invalidated",\n "param": null\n },\n "status": 401\n}")

Details omitted because the failure output was 1471 characters. Check the local Review Agent logs for the reviewer event and output files.

Comment thread packages/cli/src/daemon/routes/memory.ts Outdated
Comment thread packages/cli/src/daemon/lifecycle.ts Outdated
Comment thread packages/cli/src/daemon/routes/memory.ts Outdated
Comment thread packages/publisher/test/async-lift-publisher.test.ts
Comment thread agent-docs/todo.md Outdated
Comment thread packages/agent/src/dkg-agent-publish.ts
Comment thread packages/agent/src/dkg-agent-publish.ts Outdated
Comment thread packages/cli/src/daemon/routes/knowledge-assets.ts
Comment thread scripts/devnet-test-curator-converge.sh
Comment thread scripts/devnet-publish-helpers.sh
Comment thread scripts/devnet-test-rfc38-prereg-bytecap-stress.sh
Comment thread packages/cli/src/daemon/routes/memory.ts Outdated
Comment thread packages/agent/src/dkg-agent-publish.ts Outdated
Comment thread packages/agent/src/dkg-agent-publish.ts
Comment thread packages/publisher/src/lift-job-types.ts
Comment thread packages/agent/src/dkg-agent-publish.ts
Comment thread packages/agent/src/dkg-agent-publish.ts Outdated
Comment thread packages/agent/test/e2e-memory-layers.test.ts
Comment thread packages/cli/src/daemon/routes/memory.ts
Comment thread packages/cli/src/source-worker-daemon-client.ts
Comment thread scripts/devnet-test.sh Outdated

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Verification reviewer failed: retry_exhausted

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Business logic reviewer failed: retry_exhausted

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Business logic reviewer failed: retry_exhausted

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Business logic reviewer failed: retry_exhausted

@Jurij89 Jurij89 force-pushed the feat/api-agent-tooling-cleanup branch 2 times, most recently from b6fc66e to 52cedc1 Compare June 24, 2026 18:39
@Jurij89 Jurij89 force-pushed the codex/1260-lifecycle-async-publish branch from 939a8af to cedf002 Compare June 24, 2026 18:45
@Jurij89 Jurij89 force-pushed the codex/1260-lifecycle-async-publish branch from cedf002 to 28d2f74 Compare June 24, 2026 18:51

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational Notice: Review Agent could not complete this review.

Business logic reviewer failed: retry_exhausted

// let `resolveKaUal` throw (failing the update) — which is the
// correct behavior when chain-truth is required but unavailable.
const labelMeta = this.graphManager.metaGraphUri(contextGraphId);
const labelMeta = contextGraphMetaUri(contextGraphId, options.subGraphName);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug: Sub-graph updates now pass options.subGraphName into contextGraphMetaUri, but that helper's second argument is a context/on-chain partition id and builds <cg>/context/<value>/_meta, not the KA label metadata graph used by publish/history. As a result, a queued sub-graph update can write the refreshed UAL/root/merkle metadata into an unqueried graph while the existing root _meta metadata remains stale; the prior-root lookup above has the same issue. Keep the label metadata graph aligned with publish, for example this.graphManager.metaGraphUri(contextGraphId) or an explicit target meta graph, and use subGraphName only for the data graph.

sealMerkleRoot: `0x${'12'.repeat(32)}`,
intentKey: `sha256:${'ab'.repeat(32)}`,
}),
preflightKnowledgeAssetVmPublishSnapshot: async () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug: This is the only negative coverage for stale/missing async VM publish snapshots, but it stubs preflightKnowledgeAssetVmPublishSnapshot to throw, so a regression in the real snapshot preflight would still pass. Add a test that uses the real store/snapshot path with a missing or stale shareOperationId and asserts /vm/publish-async returns 409 without enqueuing.

}

export interface LiftRequest {
readonly jobType?: 'lift' | 'knowledge-asset-vm-publish';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: LiftRequest is being widened with an optional jobType plus an optional knowledgeAssetVmPublish payload, which creates an invalid intermediate state (jobType: 'knowledge-asset-vm-publish' with no payload, or a payload on a normal lift). The implementation already compensates with scattered jobType branches and knowledgeAssetVmPublish null checks across the async publisher. This should be modeled as a discriminated union, e.g. RawLiftRequest | KnowledgeAssetVmPublishLiftRequest, with the KA payload required in the KA branch and per-job handling dispatched from that type. That would make the boundary explicit and remove several of the special-case checks this PR adds.

.option('--transition-type <value>', 'Transition type (CREATE|MUTATE|REVOKE)', 'CREATE')
.option('--authority-type <value>', 'Authority type (owner|multisig|quorum|capability)', 'owner')
.option('--prior-version <value>', 'Prior version reference for MUTATE/REVOKE flows')
.command('publish-async <context-graph> <name>')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: This new user-facing replacement for publisher enqueue has no CLI invocation coverage; the tests only check the suggested text and the lower-level API client/route. Add a CLI smoke or command test that runs dkg publisher publish-async <cg> <name> with options like --sub-graph/--publish-epochs and verifies it calls the expected endpoint and prints the accepted job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants