diff --git a/AGENTS.md b/AGENTS.md index ebcf3584..061e6441 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -112,7 +112,7 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import). ## Current Stats (v0.9.16) - 53 MCP tools (8 visible by default, `AGENTMEMORY_TOOLS=all` for all) -- 124 REST endpoints +- 132 REST endpoints - 6 MCP resources, 3 MCP prompts - 12 hooks, 4 skills - 50+ iii functions diff --git a/README.md b/README.md index 2c29969d..877af5a0 100644 --- a/README.md +++ b/README.md @@ -1332,7 +1332,7 @@ Create `~/.agentmemory/.env`:

API

-124 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer ` when `AGENTMEMORY_SECRET` is set, and mesh sync endpoints require `AGENTMEMORY_SECRET` on both peers. +132 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer ` when `AGENTMEMORY_SECRET` is set, and mesh sync endpoints require `AGENTMEMORY_SECRET` on both peers.
Key endpoints diff --git a/docs/superpowers/plans/2026-05-27-hermes-memory-fusion.md b/docs/superpowers/plans/2026-05-27-hermes-memory-fusion.md new file mode 100644 index 00000000..192800ec --- /dev/null +++ b/docs/superpowers/plans/2026-05-27-hermes-memory-fusion.md @@ -0,0 +1,884 @@ +# Hermes Memory Fusion Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a policy-driven, shadow-first memory metacognition layer to agentmemory by borrowing the strongest ideas from RayShark/hermes-patches without importing its incompatible runtime or database architecture. + +**Architecture:** Keep agentmemory's iii-engine Function/Trigger/StateKV model as the authority. Add new module-local `mem::*` functions for policy expansion, shadow write candidates, readback verification, and lesson-to-policy suggestions; wire them through the existing composition root, REST surface, and viewer only after tests prove the internal functions. Defer hard preflight blocking and automatic writes until shadow telemetry proves precision. + +**Tech Stack:** TypeScript ESM, iii-sdk functions/triggers, StateKV scopes, Vitest, existing SearchIndex/VectorIndex/HybridSearch, existing viewer HTML, optional REST endpoints. No direct SQLite/PostgreSQL access and no standalone persistence. + +--- + +## Source Baseline + +External research target: + +- Repository: `https://github.com/RayShark/hermes-patches` +- Inspected commit: `1034e61a81088b0129678e60e28a6c9fa7896a36` +- Relevant source files: + - `agent/memory_metacognition.py` + - `agent/memory_write_pipeline.py` + - `agent/memory_semantic_classifier.py` + - `agent/memory_graph/db/models.py` + - `agent/memory_graph/services/search.py` + - `agent/request_context.py` + - `agent/shadow_write_logger.py` + - `memory_policy.default.yaml` + - `memory_write_config.yaml` + +Current agentmemory anchor files: + +- Composition root: `src/index.ts` +- State scopes: `src/state/schema.ts` +- State wrapper: `src/state/kv.ts` +- Core types: `src/types.ts` +- Ingestion: `src/functions/observe.ts` +- Durable manual memory: `src/functions/remember.ts` +- Search and indexes: `src/functions/search.ts`, `src/state/hybrid-search.ts` +- Query expansion: `src/functions/query-expansion.ts` +- Context injection: `src/functions/context.ts` +- Lessons: `src/functions/lessons.ts`, `src/functions/reflect.ts` +- Verification: `src/functions/verify.ts` +- Governance and audit: `src/functions/governance.ts`, `src/functions/audit.ts` +- REST surface: `src/triggers/api.ts` +- Viewer: `src/viewer/server.ts`, `src/viewer/index.html` + +## What To Borrow + +### Borrow 1: Shadow Write Pipeline + +Hermes has a conservative write pipeline: + +```text +conversation turn + -> candidate extraction + -> importance/type/conflict/dedup/review gates + -> optional write + -> readback verification + -> repair queue when readback fails +``` + +agentmemory should borrow this as a shadow-first path. The first shipped version must not automatically call `mem::remember`. It should only produce `MemoryWriteCandidate` rows that a user or later policy can review. + +Why this fits agentmemory: + +- `mem::observe` already captures raw and compressed observations. +- `mem::remember` already handles durable memory persistence, supersession, BM25 indexing, vector indexing, and cascade update. +- `mem::verify` already checks evidence citations, but it does not prove that a newly saved memory can be found by future recall queries. + +The missing layer is candidate generation plus readback. + +### Borrow 2: Policy-Driven Query Expansion + +Hermes uses a YAML policy to map user phrasing to stable recall terms. agentmemory already has `mem::expand-query`, but it is LLM-based and currently not fully integrated into the recall path. + +agentmemory should add a cheap policy expansion pass: + +```text +input query + -> policy expansions from KV + -> existing LLM expansion when enabled + -> HybridSearch.searchWithExpansion() +``` + +This should improve recall for stable project vocabulary without adding model cost to every search. + +### Borrow 3: Readback Verification + +Hermes generates future-oriented readback queries for a candidate and verifies that the written memory appears in top search results. + +agentmemory should make readback a first-class function: + +```text +memory or candidate + -> generated queries + -> mem::search / mem::smart-search + -> top-k hit check + -> pass/fail with repair suggestion +``` + +This is distinct from `mem::verify`. `mem::verify` answers "what evidence supports this memory?" Readback answers "will this memory be retrievable later?" + +### Borrow 4: Lesson To Policy Suggestion + +Hermes classifies lessons into policy patch suggestions. agentmemory already stores lessons with confidence, reinforcement, decay, project scope, and context injection. The next useful step is turning repeated lessons into reviewable suggestions: + +- query expansion rule +- context disclosure trigger +- preflight warning rule +- durable slot content +- ordinary lesson only + +No policy file should be modified automatically in Phase 1. + +### Borrow 5: Namespace Zero-Default Principle + +Hermes requires a non-empty namespace for user-private memory writes. agentmemory has `agentId` tagging and `AGENTMEMORY_AGENT_SCOPE=isolated`, but not a general namespace model. + +agentmemory should not copy Hermes' Telegram-specific namespace logic. It should borrow the principle: + +- do not silently write private user memory into a shared/global namespace; +- make the scope explicit in candidate rows; +- block automatic writes to shared scope until a reviewer approves. + +## What Not To Borrow + +- Do not port Hermes' SQLAlchemy/PostgreSQL Memory Graph. agentmemory must continue using iii-engine StateKV. +- Do not introduce direct SQLite, direct Postgres, or out-of-band file persistence for core state. +- Do not copy Hermes' Telegram/user-path-specific regexes into core. +- Do not ship hard preflight blocking in the first version. +- Do not add MCP tools in Phase 1 unless the implementation also updates every required registry/count/docs file listed in `AGENTS.md`. +- Do not write policy changes from lessons without an explicit review step. + +## Target Design + +### New KV Scopes + +Add these scopes in `src/state/schema.ts`: + +```ts +memoryPolicy: "mem:policy", +writeCandidates: "mem:write-candidates", +readbackResults: "mem:readback", +policySuggestions: "mem:policy-suggestions", +``` + +The exact names should be short and stable because they become part of export/import and viewer state. + +### New Types + +Add these types in `src/types.ts`: + +```ts +export interface MemoryPolicy { + id: "default"; + updatedAt: string; + queryExpansions: QueryExpansionRule[]; + writePolicy: MemoryWritePolicy; + preflightRules: PreflightRule[]; +} + +export interface PreflightRule { + id: string; + tool: string; + taskType: string; + triggerPatterns: string[]; + decision: "allow" | "warn" | "block"; + enabled: boolean; + createdAt: string; + updatedAt: string; +} + +export interface QueryExpansionRule { + id: string; + trigger: string; + expansions: string[]; + scope?: "global" | "project"; + project?: string; + enabled: boolean; + createdAt: string; + updatedAt: string; +} + +export interface MemoryWritePolicy { + mode: "shadow" | "limited_auto" | "disabled"; + autoWriteThreshold: number; + allowedAutoTypes: MemoryWriteCandidate["memoryType"][]; + neverAutoWriteShared: boolean; +} + +export interface MemoryWriteCandidate { + id: string; + sessionId?: string; + observationId?: string; + project?: string; + agentId?: string; + scope: "global" | "project" | "agent"; + createdAt: string; + sourceText: string; + evidenceQuote: string; + subject: string; + predicate: string; + value: string; + memoryType: + | "fact" + | "preference" + | "architecture" + | "bug" + | "workflow" + | "lesson" + | "procedural_rule" + | "credential_route" + | "temporary" + | "ignore"; + confidence: number; + importance: number; + target: "memory" | "lesson" | "slot" | "review" | "ignore"; + requiresReview: boolean; + reason: string; + readbackQueries: string[]; + status: "shadow" | "approved" | "rejected" | "written" | "readback_failed"; +} + +export interface ReadbackResult { + id: string; + candidateId?: string; + memoryId?: string; + createdAt: string; + queries: Array<{ + query: string; + topIds: string[]; + matched: boolean; + }>; + passed: boolean; + failureReason?: string; +} + +export interface PolicySuggestion { + id: string; + lessonId?: string; + createdAt: string; + type: "query_expansion" | "preflight" | "context_disclosure" | "slot" | "memory_only"; + confidence: number; + scope: "global" | "project"; + project?: string; + proposal: Record; + status: "proposed" | "approved" | "rejected" | "applied"; + reasoning: string; +} +``` + +### New Function Modules + +Create focused files: + +- `src/functions/memory-policy.ts` + - `mem::policy-get` + - `mem::policy-update` + - `mem::policy-expand-query` +- `src/functions/write-candidates.ts` + - `mem::write-candidates-generate` + - `mem::write-candidates-list` + - `mem::write-candidates-review` +- `src/functions/readback.ts` + - `mem::readback-verify` + - `mem::readback-list` +- `src/functions/policy-suggestions.ts` + - `mem::policy-suggest-from-lesson` + - `mem::policy-suggestions-list` + - `mem::policy-suggestions-review` + +Register these in `src/index.ts` near the existing lessons/verify/retention registrations. + +### REST Endpoints + +Phase 1 REST should be enough for viewer and manual testing. Add endpoints in `src/triggers/api.ts`: + +- `GET /agentmemory/policy` +- `POST /agentmemory/policy` +- `POST /agentmemory/policy/expand-query` +- `POST /agentmemory/write-candidates/generate` +- `GET /agentmemory/write-candidates` +- `POST /agentmemory/write-candidates/review` +- `POST /agentmemory/readback/verify` +- `GET /agentmemory/readback` + +Update the REST endpoint count in `src/index.ts` and `README.md` when these are added. + +Policy-suggestions endpoints belong to Phase 2 because their underlying functions are introduced there. + +### MCP Tools + +Do not add MCP tools in Phase 1. This avoids changing public tool count while the feature is experimental. + +If MCP tools are added later, update all required files from `AGENTS.md`: + +- `src/mcp/tools-registry.ts` +- `src/mcp/server.ts` +- `src/triggers/api.ts` +- `src/index.ts` +- `test/mcp-standalone.test.ts` +- `README.md` +- `plugin/.claude-plugin/plugin.json` + +## Phase 1: Shadow Write And Readback + +### Task 1: Add Types And KV Scopes + +**Files:** + +- Modify: `src/state/schema.ts` +- Modify: `src/types.ts` +- Test: `test/memory-policy-types.test.ts` + +- [ ] **Step 1: Add a schema test** + +Create `test/memory-policy-types.test.ts`: + +```ts +import { describe, expect, it } from "vitest"; +import { KV } from "../src/state/schema.js"; + +describe("memory metacognition KV scopes", () => { + it("defines stable scopes for policy, candidates, readback, and suggestions", () => { + expect(KV.memoryPolicy).toBe("mem:policy"); + expect(KV.writeCandidates).toBe("mem:write-candidates"); + expect(KV.readbackResults).toBe("mem:readback"); + expect(KV.policySuggestions).toBe("mem:policy-suggestions"); + }); +}); +``` + +- [ ] **Step 2: Run the failing test** + +Run: + +```bash +npm test -- test/memory-policy-types.test.ts +``` + +Expected: FAIL because the new `KV` keys do not exist. + +- [ ] **Step 3: Add the KV scopes and exported interfaces** + +Modify `src/state/schema.ts` and `src/types.ts` with the fields from the Target Design section. + +- [ ] **Step 4: Run the type test** + +Run: + +```bash +npm test -- test/memory-policy-types.test.ts +``` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add src/state/schema.ts src/types.ts test/memory-policy-types.test.ts +git commit -m "feat: add memory metacognition state types" +``` + +### Task 2: Add Policy Get, Update, And Query Expansion + +**Files:** + +- Create: `src/functions/memory-policy.ts` +- Modify: `src/index.ts` +- Test: `test/memory-policy.test.ts` + +- [ ] **Step 1: Write tests for default policy and rule expansion** + +Create `test/memory-policy.test.ts` with cases for: + +- default policy exists when no KV row has been saved; +- disabled query expansion rules are ignored; +- project-scoped rules only apply to that project; +- expansion output deduplicates original query and rule expansions. + +Example assertion shape: + +```ts +expect(result.expansion.original).toBe("改配置"); +expect(result.expansion.reformulations).toContain("config.yaml"); +expect(result.expansion.reformulations).toContain("provider"); +``` + +- [ ] **Step 2: Run the failing test** + +```bash +npm test -- test/memory-policy.test.ts +``` + +Expected: FAIL because `registerMemoryPolicyFunction` does not exist. + +- [ ] **Step 3: Implement `src/functions/memory-policy.ts`** + +Required behavior: + +- `mem::policy-get` returns a default policy when `KV.memoryPolicy/default` is absent. +- `mem::policy-update` validates mode, thresholds, enabled flags, and expansion arrays before writing. +- `mem::policy-expand-query` accepts `{ query, project?, maxQueries? }`, returns `{ success, expansion }`. +- Rule expansion must not call the LLM. +- Rule expansion must return the original query plus deduplicated expansions capped by `maxQueries`. + +- [ ] **Step 4: Register the function** + +Modify `src/index.ts`: + +```ts +registerMemoryPolicyFunction(sdk, kv); +``` + +Place it near `registerQueryExpansionFunction` and `registerSmartSearchFunction`. + +- [ ] **Step 5: Run the tests** + +```bash +npm test -- test/memory-policy.test.ts test/memory-policy-types.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/functions/memory-policy.ts src/index.ts test/memory-policy.test.ts +git commit -m "feat: add policy-driven query expansion" +``` + +### Task 3: Add Shadow Write Candidate Generation + +**Files:** + +- Create: `src/functions/write-candidates.ts` +- Modify: `src/index.ts` +- Test: `test/write-candidates.test.ts` + +- [ ] **Step 1: Write candidate generation tests** + +Create tests for these inputs: + +- `"以后遇到这种报错，先查之前的修复记录再动手"` creates a `procedural_rule` or `workflow` candidate requiring review. +- `"我更喜欢简洁直接的回答"` creates a `preference` candidate with confidence at least `0.75`. +- `"哈哈可以"` creates no write candidate or an `ignore` candidate. +- `"我的 API key 是 sk-test"` never stores the raw secret in `sourceText`, `evidenceQuote`, or `value`. +- a generated candidate is persisted to `KV.writeCandidates` with status `shadow`. + +- [ ] **Step 2: Run the failing test** + +```bash +npm test -- test/write-candidates.test.ts +``` + +Expected: FAIL because `registerWriteCandidatesFunction` does not exist. + +- [ ] **Step 3: Implement conservative extraction** + +Implement `mem::write-candidates-generate` with deterministic, low-risk rules first: + +- explicit preference patterns; +- explicit correction patterns; +- workflow/procedural-memory patterns; +- temporary/noise suppression; +- secret redaction before persistence. + +Do not call `mem::remember`. + +- [ ] **Step 4: Add list and review operations** + +Implement: + +- `mem::write-candidates-list` with filters `{ status?, project?, agentId?, limit? }`; +- `mem::write-candidates-review` with `{ candidateId, decision, reason? }`; +- allowed decisions: `approve`, `reject`; +- approving only changes candidate status to `approved`; it does not write memory in Phase 1. + +- [ ] **Step 5: Register the function** + +Modify `src/index.ts`: + +```ts +registerWriteCandidatesFunction(sdk, kv); +``` + +- [ ] **Step 6: Run targeted tests** + +```bash +npm test -- test/write-candidates.test.ts +``` + +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add src/functions/write-candidates.ts src/index.ts test/write-candidates.test.ts +git commit -m "feat: add shadow memory write candidates" +``` + +### Task 4: Add Readback Verification + +**Files:** + +- Create: `src/functions/readback.ts` +- Modify: `src/index.ts` +- Test: `test/readback.test.ts` + +- [ ] **Step 1: Write readback tests** + +Cover: + +- candidate readback generates at least two queries; +- memory readback checks whether the target memory id appears in top search results; +- failed readback stores a `ReadbackResult` in `KV.readbackResults`; +- readback does not mutate memories or candidates except optional candidate status `readback_failed`. + +- [ ] **Step 2: Run the failing test** + +```bash +npm test -- test/readback.test.ts +``` + +Expected: FAIL because `registerReadbackFunction` does not exist. + +- [ ] **Step 3: Implement `mem::readback-verify`** + +Behavior: + +- accepts `{ candidateId?; memoryId?; queries?; limit?; mode?: "search" | "smart-search" }`; +- if `candidateId` is provided, load the candidate and use its `readbackQueries`; +- if `memoryId` is provided, generate queries from title/content/concepts/files; +- call existing `mem::search` or `mem::smart-search`; +- consider readback passed when target id appears in the top `limit` result IDs for any query; +- persist a `ReadbackResult`. + +- [ ] **Step 4: Register the function** + +Modify `src/index.ts`: + +```ts +registerReadbackFunction(sdk, kv); +``` + +- [ ] **Step 5: Run targeted tests** + +```bash +npm test -- test/readback.test.ts test/write-candidates.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/functions/readback.ts src/index.ts test/readback.test.ts +git commit -m "feat: add memory readback verification" +``` + +### Task 5: Add REST Endpoints + +**Files:** + +- Modify: `src/triggers/api.ts` +- Modify: `src/index.ts` +- Modify: `README.md` +- Test: `test/api-memory-metacognition.test.ts` + +- [ ] **Step 1: Write endpoint tests** + +Test: + +- auth is enforced when `AGENTMEMORY_SECRET` is set; +- request bodies are whitelisted before `sdk.trigger`; +- invalid query/candidate/review payloads return 400; +- successful calls trigger the matching `mem::*` function. + +- [ ] **Step 2: Run the failing test** + +```bash +npm test -- test/api-memory-metacognition.test.ts +``` + +Expected: FAIL because the endpoints are absent. + +- [ ] **Step 3: Implement REST handlers** + +Add the Phase 1 endpoints from the REST Endpoints section. Follow the existing API pattern: + +- parse `req.body` as `Record`; +- validate strings, numbers, arrays, and enums; +- construct a whitelisted payload object; +- call `sdk.trigger({ function_id, payload })`; +- return 400 for invalid input and 200/201 for success. + +- [ ] **Step 4: Update endpoint counts** + +Update the boot log endpoint count in `src/index.ts` and the REST endpoint counts in `README.md`. + +- [ ] **Step 5: Run targeted tests** + +```bash +npm test -- test/api-memory-metacognition.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/triggers/api.ts src/index.ts README.md test/api-memory-metacognition.test.ts +git commit -m "feat: expose memory metacognition REST endpoints" +``` + +## Phase 2: Policy Suggestions + +### Task 6: Add Lesson-To-Policy Suggestions + +**Files:** + +- Create: `src/functions/policy-suggestions.ts` +- Modify: `src/index.ts` +- Test: `test/policy-suggestions.test.ts` + +- [ ] **Step 1: Write suggestion tests** + +Cover: + +- a lesson mentioning "search", "关键词", or "recall" becomes a `query_expansion` suggestion; +- a lesson mentioning "must", "before", "检查", or "执行前" becomes a `preflight` suggestion; +- private indicators such as token, key, password, chat id, or user id force project/private scope and review; +- approving a suggestion does not mutate policy until an explicit apply function exists. + +- [ ] **Step 2: Implement suggestion classification** + +Use deterministic scoring inspired by Hermes, but keep terms generic and project-safe. + +- [ ] **Step 3: Register and test** + +```bash +npm test -- test/policy-suggestions.test.ts +``` + +Expected: PASS. + +- [ ] **Step 4: Commit** + +```bash +git add src/functions/policy-suggestions.ts src/index.ts test/policy-suggestions.test.ts +git commit -m "feat: suggest policy changes from lessons" +``` + +## Phase 3: Viewer Review UI + +### Task 7: Add Viewer Panels For Candidates And Suggestions + +**Files:** + +- Modify: `src/viewer/index.html` +- Test: existing viewer tests or add `test/viewer-memory-metacognition.test.ts` + +- [ ] **Step 1: Add API client calls in the viewer script** + +Add calls for: + +- `GET /agentmemory/write-candidates` +- `POST /agentmemory/write-candidates/review` +- `GET /agentmemory/readback` +- `GET /agentmemory/policy-suggestions` +- `POST /agentmemory/policy-suggestions/review` + +- [ ] **Step 2: Add a review-focused UI** + +UI requirements: + +- show candidate type, confidence, target, evidence quote, readback status; +- show approve/reject buttons; +- never display raw secret-like values; +- show policy suggestions separately from write candidates; +- keep the panel behind an existing tab or a new "Review" tab. + +- [ ] **Step 3: Verify in browser** + +Start a local dev instance after implementation and verify: + +```bash +npm run build +npm run dev +``` + +Then open the viewer port and confirm the panel renders with seeded API data or fixture state. + +- [ ] **Step 4: Commit** + +```bash +git add src/viewer/index.html test/viewer-memory-metacognition.test.ts +git commit -m "feat: add memory review panels to viewer" +``` + +## Phase 4: Preflight Warning + +### Task 8: Add Warn-Only Preflight Function + +**Files:** + +- Create: `src/functions/preflight-policy.ts` +- Modify: `src/index.ts` +- Test: `test/preflight-policy.test.ts` + +- [ ] **Step 1: Write tests** + +Cover: + +- destructive command rules return `decision: "warn"` by default; +- no rule returns `decision: "allow"`; +- block rules are ignored unless `writePolicy.mode` or a dedicated preflight flag enables blocking; +- memory recall checks are best-effort and never throw. + +- [ ] **Step 2: Implement warn-only function** + +Implement `mem::preflight-check`: + +```ts +type PreflightDecision = "allow" | "warn" | "block"; +``` + +Phase 4 must return `warn` instead of `block` unless a separate future PR explicitly enables blocking. + +- [ ] **Step 3: Do not wire it into hooks by default** + +Expose it only as REST or internal function. Hook-level blocking needs a separate risk review. + +- [ ] **Step 4: Commit** + +```bash +git add src/functions/preflight-policy.ts src/index.ts test/preflight-policy.test.ts +git commit -m "feat: add warn-only memory preflight checks" +``` + +## Acceptance Criteria + +Phase 1 is complete when: + +- `npm test` passes. +- `npm run build` passes. +- policy query expansion works without LLM calls. +- write candidates are generated and persisted in shadow mode. +- no candidate generation path calls `mem::remember`. +- readback verification stores pass/fail results. +- REST handlers whitelist fields and enforce auth consistently. +- README endpoint counts are correct. +- no MCP tool count changes are introduced. + +Phase 2 is complete when: + +- lessons can generate reviewable policy suggestions. +- suggestions are persisted and reviewable. +- suggestions do not mutate policy automatically. + +Phase 3 is complete when: + +- viewer exposes candidate and suggestion review. +- secret-like content remains redacted. +- UI works on desktop and narrow viewport. + +Phase 4 is complete when: + +- preflight returns allow/warn/block data structures. +- default behavior remains warn-only. +- hooks do not block tools by default. + +## Test Commands + +Targeted tests during development: + +```bash +npm test -- test/memory-policy-types.test.ts +npm test -- test/memory-policy.test.ts +npm test -- test/write-candidates.test.ts +npm test -- test/readback.test.ts +npm test -- test/api-memory-metacognition.test.ts +npm test -- test/policy-suggestions.test.ts +npm test -- test/preflight-policy.test.ts +``` + +Full verification before PR: + +```bash +npm test +npm run build +git diff --check +``` + +If a PR adds Python helper scripts or docs-check-covered exported functions, add docstrings/JSDoc before pushing. The previous docstring coverage warning should be treated as a release gate even if it is only a warning. + +## Risk Register + +### Risk: Memory Pollution + +Automatic writes can corrupt long-term memory. Phase 1 prevents this by keeping candidate generation shadow-only. + +Mitigation: + +- default `MemoryWritePolicy.mode = "shadow"`; +- no `mem::remember` calls from candidate generation; +- review status required before write; +- readback failure recorded instead of silently accepted. + +### Risk: Search Semantics Drift + +Policy expansion may change recall results. + +Mitigation: + +- policy expansion is explicit and inspectable; +- disabled rules are ignored; +- project-scoped rules only apply to matching projects; +- original query is always preserved. + +### Risk: Privacy Leak Across Agents Or Users + +agentmemory currently has `agentId` isolation but not a full user namespace model. + +Mitigation: + +- candidate rows carry `agentId` and `project`; +- automatic write to shared/global scope is disabled; +- viewer and REST list endpoints must support `agentId` filters consistently with existing memory/session endpoints. + +### Risk: Public API Churn + +Adding MCP tools would require public tool count updates and compatibility maintenance. + +Mitigation: + +- Phase 1 uses internal functions plus REST only; +- MCP tools are deferred until the feature has stable semantics. + +### Risk: Overfitting To Hermes + +Hermes contains Telegram/CJK/user-specific rules and some experimental paths. + +Mitigation: + +- borrow only generic mechanisms; +- keep policy data configurable; +- avoid copying hardcoded trigger phrases except in tests where they prove generic behavior. + +## Open Design Decisions + +These must be answered before implementation starts: + +1. Should Phase 1 include viewer UI, or should it stop at REST plus internal functions? +2. Should readback verification use `mem::search`, `mem::smart-search`, or both by default? +3. Should approved write candidates be manually converted through existing `memory_save`, or should a later phase add `mem::write-candidates-apply`? +4. Should policy be project-scoped by default, or global with optional project filters? + +Recommended defaults: + +- Phase 1 includes REST and internal functions; viewer is Phase 3. +- readback runs both `mem::search` and `mem::smart-search` when available, and passes if either finds the target. +- approved candidates remain approved-only until a later apply function is reviewed. +- policy is global by default, with project filters on individual rules. + +## PR Strategy + +Use small PRs: + +1. `feat: add memory metacognition state types` +2. `feat: add policy-driven query expansion` +3. `feat: add shadow memory write candidates` +4. `feat: add memory readback verification` +5. `feat: expose memory metacognition REST endpoints` +6. `feat: suggest policy changes from lessons` +7. `feat: add memory review panels to viewer` +8. `feat: add warn-only memory preflight checks` + +Each PR should include: + +- focused tests; +- no unrelated viewer or tool-count churn; +- updated endpoint counts when REST endpoints change; +- explicit note that Hermes code was used as inspiration, not vendored. diff --git a/src/functions/memory-policy.ts b/src/functions/memory-policy.ts new file mode 100644 index 00000000..3280fb3e --- /dev/null +++ b/src/functions/memory-policy.ts @@ -0,0 +1,249 @@ +import type { ISdk } from "iii-sdk"; +import type { + MemoryPolicy, + MemoryWriteCandidate, + MemoryWritePolicy, + PreflightRule, + QueryExpansion, + QueryExpansionRule, +} from "../types.js"; +import { KV } from "../state/schema.js"; +import type { StateKV } from "../state/kv.js"; +import { recordAudit } from "./audit.js"; + +const POLICY_ID = "default"; +const DEFAULT_WRITE_POLICY: MemoryWritePolicy = { + mode: "shadow", + autoWriteThreshold: 0.85, + allowedAutoTypes: ["preference", "workflow"], + neverAutoWriteShared: true, +}; +const VALID_CANDIDATE_TYPES = new Set([ + "fact", + "preference", + "architecture", + "bug", + "workflow", + "lesson", + "procedural_rule", + "credential_route", + "temporary", + "ignore", +]); + +function nowIso(): string { + return new Date().toISOString(); +} + +function defaultPolicy(timestamp = nowIso()): MemoryPolicy { + return { + id: POLICY_ID, + updatedAt: timestamp, + queryExpansions: [], + writePolicy: { ...DEFAULT_WRITE_POLICY }, + preflightRules: [], + }; +} + +async function readPolicy(kv: StateKV): Promise { + const stored = await kv.get(KV.memoryPolicy, POLICY_ID); + if (!stored) return defaultPolicy(); + return normalizePolicy(stored); +} + +function asString(value: unknown, max = 256): string | null { + if (typeof value !== "string") return null; + const trimmed = value.trim(); + if (!trimmed) return null; + return trimmed.slice(0, max); +} + +function uniqueStrings(values: unknown, maxItems: number, maxLen = 128): string[] { + if (!Array.isArray(values)) return []; + const seen = new Set(); + const result: string[] = []; + for (const value of values) { + const text = asString(value, maxLen); + if (!text) continue; + const key = text.toLowerCase(); + if (seen.has(key)) continue; + seen.add(key); + result.push(text); + if (result.length >= maxItems) break; + } + return result; +} + +function normalizeQueryRule(raw: unknown, timestamp: string): QueryExpansionRule | null { + if (!raw || typeof raw !== "object") return null; + const r = raw as Partial; + const id = asString(r.id, 96); + const trigger = asString(r.trigger, 128); + if (!id || !trigger) return null; + const scope = + r.scope === "global" || r.scope === "project" ? r.scope : "global"; + const rule: QueryExpansionRule = { + id, + trigger, + expansions: uniqueStrings(r.expansions, 20), + scope, + enabled: r.enabled !== false, + createdAt: asString(r.createdAt, 64) ?? timestamp, + updatedAt: timestamp, + }; + if (scope === "project") { + const project = asString(r.project, 256); + if (!project) return null; + rule.project = project; + } + return rule; +} + +function normalizeWritePolicy(raw: unknown): MemoryWritePolicy { + if (!raw || typeof raw !== "object") return { ...DEFAULT_WRITE_POLICY }; + const r = raw as Partial; + const mode = + r.mode === "disabled" || r.mode === "limited_auto" || r.mode === "shadow" + ? r.mode + : DEFAULT_WRITE_POLICY.mode; + const threshold = + typeof r.autoWriteThreshold === "number" && + Number.isFinite(r.autoWriteThreshold) + ? Math.max(0, Math.min(1, r.autoWriteThreshold)) + : DEFAULT_WRITE_POLICY.autoWriteThreshold; + const allowedAutoTypes = uniqueStrings(r.allowedAutoTypes, 20).filter( + (type): type is MemoryWriteCandidate["memoryType"] => + VALID_CANDIDATE_TYPES.has(type as MemoryWriteCandidate["memoryType"]), + ); + return { + mode, + autoWriteThreshold: threshold, + allowedAutoTypes: + allowedAutoTypes.length > 0 + ? allowedAutoTypes + : [...DEFAULT_WRITE_POLICY.allowedAutoTypes], + neverAutoWriteShared: r.neverAutoWriteShared !== false, + }; +} + +function normalizePreflightRule(raw: unknown, timestamp: string): PreflightRule | null { + if (!raw || typeof raw !== "object") return null; + const r = raw as Partial; + const id = asString(r.id, 96); + const tool = asString(r.tool, 128); + const taskType = asString(r.taskType, 128); + if (!id || !tool || !taskType) return null; + const decision = + r.decision === "block" || r.decision === "warn" || r.decision === "allow" + ? r.decision + : "warn"; + return { + id, + tool, + taskType, + triggerPatterns: uniqueStrings(r.triggerPatterns, 20), + decision, + enabled: r.enabled !== false, + createdAt: asString(r.createdAt, 64) ?? timestamp, + updatedAt: timestamp, + }; +} + +function normalizePolicy(raw: Partial): MemoryPolicy { + const timestamp = nowIso(); + const queryExpansions = Array.isArray(raw.queryExpansions) + ? raw.queryExpansions + .map((rule) => normalizeQueryRule(rule, timestamp)) + .filter((rule): rule is QueryExpansionRule => rule !== null) + : []; + const preflightRules = Array.isArray(raw.preflightRules) + ? raw.preflightRules + .map((rule) => normalizePreflightRule(rule, timestamp)) + .filter((rule): rule is PreflightRule => rule !== null) + : []; + return { + id: POLICY_ID, + updatedAt: timestamp, + queryExpansions, + writePolicy: normalizeWritePolicy(raw.writePolicy), + preflightRules, + }; +} + +function matchesRule(rule: QueryExpansionRule, query: string, project?: string): boolean { + if (!rule.enabled) return false; + if (rule.scope === "project" && rule.project !== project) return false; + return query.toLowerCase().includes(rule.trigger.toLowerCase()); +} + +function expandFromPolicy( + policy: MemoryPolicy, + query: string, + project: string | undefined, + maxQueries: number, +): QueryExpansion { + const seen = new Set([query.toLowerCase()]); + const reformulations: string[] = []; + for (const rule of policy.queryExpansions) { + if (!matchesRule(rule, query, project)) continue; + for (const expansion of rule.expansions) { + const key = expansion.toLowerCase(); + if (seen.has(key)) continue; + seen.add(key); + reformulations.push(expansion); + if (reformulations.length >= maxQueries) { + return { + original: query, + reformulations, + temporalConcretizations: [], + entityExtractions: [], + }; + } + } + } + return { + original: query, + reformulations, + temporalConcretizations: [], + entityExtractions: [], + }; +} + +export function registerMemoryPolicyFunction(sdk: ISdk, kv: StateKV): void { + sdk.registerFunction("mem::policy-get", async () => ({ + success: true, + policy: await readPolicy(kv), + })); + + sdk.registerFunction( + "mem::policy-update", + async (data: Partial | undefined) => { + const policy = normalizePolicy(data ?? {}); + await kv.set(KV.memoryPolicy, POLICY_ID, policy); + await recordAudit(kv, "policy_update", "mem::policy-update", [POLICY_ID], { + queryExpansionRules: policy.queryExpansions.length, + preflightRules: policy.preflightRules.length, + mode: policy.writePolicy.mode, + }); + return { success: true, policy }; + }, + ); + + sdk.registerFunction( + "mem::policy-expand-query", + async (data: { query?: string; project?: string; maxQueries?: number }) => { + const query = asString(data?.query, 500); + if (!query) return { success: false, error: "query is required" }; + const rawMax = Number(data?.maxQueries); + const maxQueries = Number.isFinite(rawMax) + ? Math.max(1, Math.min(20, Math.floor(rawMax))) + : 8; + const project = asString(data?.project, 256) ?? undefined; + const policy = await readPolicy(kv); + return { + success: true, + expansion: expandFromPolicy(policy, query, project, maxQueries), + }; + }, + ); +} diff --git a/src/functions/readback.ts b/src/functions/readback.ts new file mode 100644 index 00000000..c5fba42e --- /dev/null +++ b/src/functions/readback.ts @@ -0,0 +1,244 @@ +import type { ISdk } from "iii-sdk"; +import type { + Memory, + MemoryWriteCandidate, + ReadbackResult, +} from "../types.js"; +import { KV, generateId } from "../state/schema.js"; +import type { StateKV } from "../state/kv.js"; +import { recordAudit } from "./audit.js"; + +type ReadbackMode = "search" | "smart-search"; + +interface ReadbackInput { + candidateId?: string; + memoryId?: string; + queries?: unknown; + limit?: number; + mode?: ReadbackMode; +} + +interface SearchLikeResult { + results?: unknown[]; +} + +function nowIso(): string { + return new Date().toISOString(); +} + +function asString(value: unknown, max = 500): string | undefined { + if (typeof value !== "string") return undefined; + const trimmed = value.trim(); + if (!trimmed) return undefined; + return trimmed.slice(0, max); +} + +function uniqueStrings(values: unknown, maxItems: number, maxLen = 200): string[] { + if (!Array.isArray(values)) return []; + const seen = new Set(); + const result: string[] = []; + for (const value of values) { + const text = asString(value, maxLen); + if (!text) continue; + const key = text.toLowerCase(); + if (seen.has(key)) continue; + seen.add(key); + result.push(text); + if (result.length >= maxItems) break; + } + return result; +} + +function normalizeLimit(value: unknown): number { + if (!Number.isInteger(value)) return 10; + return Math.max(1, Math.min(value as number, 50)); +} + +function normalizeMode(value: unknown): ReadbackMode { + return value === "smart-search" ? "smart-search" : "search"; +} + +function normalizeQueries(queries: unknown): string[] { + return uniqueStrings(queries, 10, 240); +} + +function makeMemoryQueries(memory: Memory): string[] { + const candidates = [ + `${memory.title} ${memory.content.slice(0, 160)}`, + memory.concepts.join(" "), + memory.files.join(" "), + memory.title, + memory.content.slice(0, 200), + ]; + const queries = uniqueStrings(candidates, 5, 240); + if (queries.length >= 2) return queries; + if (memory.title && !queries.includes(memory.title)) queries.push(memory.title); + const content = memory.content.slice(0, 120).trim(); + if (content && !queries.includes(content)) queries.push(content); + return queries.slice(0, 5); +} + +function makeCandidateQueries(candidate: MemoryWriteCandidate): string[] { + const stored = normalizeQueries(candidate.readbackQueries); + if (stored.length > 0) return stored; + return uniqueStrings( + [ + `${candidate.subject} ${candidate.predicate}`, + `${candidate.subject} ${candidate.value}`, + `${candidate.memoryType} ${candidate.value}`, + ], + 5, + 240, + ); +} + +function extractResultId(result: unknown): string | null { + if (!result || typeof result !== "object") return null; + const record = result as Record; + const direct = asString(record.id, 160) ?? asString(record.obsId, 160) ?? + asString(record.observationId, 160) ?? asString(record.memoryId, 160); + if (direct) return direct; + const observation = record.observation; + if (observation && typeof observation === "object") { + return asString((observation as Record).id, 160) ?? null; + } + return null; +} + +function topIdsFromResult(result: unknown, limit: number): string[] { + const results = (result as SearchLikeResult | null)?.results; + if (!Array.isArray(results)) return []; + const ids: string[] = []; + const seen = new Set(); + for (const item of results) { + const id = extractResultId(item); + if (!id || seen.has(id)) continue; + seen.add(id); + ids.push(id); + if (ids.length >= limit) break; + } + return ids; +} + +async function persistReadback( + kv: StateKV, + readback: ReadbackResult, +): Promise { + await kv.set(KV.readbackResults, readback.id, readback); + await recordAudit( + kv, + "readback_verify", + "mem::readback-verify", + [readback.memoryId, readback.candidateId].filter( + (id): id is string => typeof id === "string", + ), + { + passed: readback.passed, + queries: readback.queries.length, + failureReason: readback.failureReason, + }, + ); + return readback; +} + +export function registerReadbackFunction(sdk: ISdk, kv: StateKV): void { + sdk.registerFunction( + "mem::readback-verify", + async (data: ReadbackInput | undefined) => { + const candidateId = asString(data?.candidateId, 160); + const memoryId = asString(data?.memoryId, 160); + const explicitQueries = normalizeQueries(data?.queries); + const limit = normalizeLimit(data?.limit); + const mode = normalizeMode(data?.mode); + + if (!candidateId && !memoryId) { + return { success: false, error: "candidateId or memoryId is required" }; + } + + let targetMemoryId = memoryId; + let queries = explicitQueries; + let candidate: MemoryWriteCandidate | null = null; + + if (candidateId) { + candidate = await kv.get( + KV.writeCandidates, + candidateId, + ); + if (!candidate) return { success: false, error: "candidate not found" }; + if (queries.length === 0) queries = makeCandidateQueries(candidate); + targetMemoryId = + targetMemoryId ?? + asString((candidate as MemoryWriteCandidate & { memoryId?: unknown }).memoryId, 160); + if (!targetMemoryId) { + const readback: ReadbackResult = { + id: generateId("readback"), + candidateId, + createdAt: nowIso(), + queries: queries.map((query) => ({ query, topIds: [], matched: false })), + passed: false, + failureReason: "candidate has no durable memoryId yet", + }; + return { + success: true, + readback: await persistReadback(kv, readback), + }; + } + } + + const memory = await kv.get(KV.memories, targetMemoryId!); + if (!memory) return { success: false, error: "memory not found" }; + if (queries.length === 0) queries = makeMemoryQueries(memory); + + const queryResults = await Promise.all( + queries.map(async (query) => { + const searchResult = await sdk.trigger({ + function_id: mode === "smart-search" ? "mem::smart-search" : "mem::search", + payload: { query, limit }, + }); + const topIds = topIdsFromResult(searchResult, limit); + return { + query, + topIds, + matched: topIds.includes(memory.id), + }; + }), + ); + + const passed = queryResults.some((query) => query.matched); + const readback: ReadbackResult = { + id: generateId("readback"), + ...(candidateId ? { candidateId } : {}), + memoryId: memory.id, + createdAt: nowIso(), + queries: queryResults, + passed, + ...(passed ? {} : { failureReason: "target not found in top results" }), + }; + return { success: true, readback: await persistReadback(kv, readback) }; + }, + ); + + sdk.registerFunction( + "mem::readback-list", + async (data?: { + candidateId?: string; + memoryId?: string; + limit?: number; + }) => { + const candidateId = asString(data?.candidateId, 160); + const memoryId = asString(data?.memoryId, 160); + const limit = normalizeLimit(data?.limit); + let results = await kv.list(KV.readbackResults); + if (candidateId) { + results = results.filter((result) => result.candidateId === candidateId); + } + if (memoryId) { + results = results.filter((result) => result.memoryId === memoryId); + } + results.sort( + (a, b) => new Date(b.createdAt).getTime() - new Date(a.createdAt).getTime(), + ); + return { success: true, readbacks: results.slice(0, limit) }; + }, + ); +} diff --git a/src/functions/write-candidates.ts b/src/functions/write-candidates.ts new file mode 100644 index 00000000..df30deba --- /dev/null +++ b/src/functions/write-candidates.ts @@ -0,0 +1,290 @@ +import type { ISdk } from "iii-sdk"; +import type { MemoryWriteCandidate } from "../types.js"; +import { KV, generateId } from "../state/schema.js"; +import type { StateKV } from "../state/kv.js"; +import { recordAudit } from "./audit.js"; + +const SECRET_PATTERN = + /(sk-[A-Za-z0-9_-]{6,}|github_pat_[A-Za-z0-9_]+|ghp_[A-Za-z0-9_]+|xox[baprs]-[A-Za-z0-9-]+|AKIA[0-9A-Z]{16}|api[_ -]?key\s*(?:是|=|:)\s*\S+|token\s*(?:是|=|:)\s*\S+)/gi; + +function nowIso(): string { + return new Date().toISOString(); +} + +function asString(value: unknown, max = 500): string | undefined { + if (typeof value !== "string") return undefined; + const trimmed = value.trim(); + if (!trimmed) return undefined; + return trimmed.slice(0, max); +} + +function redactSecrets(text: string): string { + return text.replace(SECRET_PATTERN, "[REDACTED_SECRET]"); +} + +function isLowSignal(text: string): boolean { + return /^(哈哈|嗯|好的|可以|ok|行|好|继续)+$/i.test(text.trim()); +} + +function resolveScope(data: { + agentId?: string; + project?: string; +}): MemoryWriteCandidate["scope"] { + if (data.agentId) return "agent"; + if (data.project) return "project"; + return "global"; +} + +function makeReadbackQueries(candidate: { + subject: string; + predicate: string; + value: string; + memoryType: string; +}): string[] { + const candidates = [ + `${candidate.subject} ${candidate.predicate}`, + `${candidate.subject} ${candidate.value.slice(0, 48)}`, + `${candidate.memoryType} ${candidate.value.slice(0, 48)}`, + ]; + const seen = new Set(); + return candidates + .map((q) => q.replace(/\s+/g, " ").trim()) + .filter((q) => { + if (!q || seen.has(q)) return false; + seen.add(q); + return true; + }) + .slice(0, 5); +} + +function baseCandidate(data: { + sourceText: string; + sessionId?: string; + observationId?: string; + project?: string; + agentId?: string; +}): Omit< + MemoryWriteCandidate, + | "id" + | "createdAt" + | "subject" + | "predicate" + | "value" + | "memoryType" + | "confidence" + | "importance" + | "target" + | "requiresReview" + | "reason" + | "readbackQueries" + | "status" +> { + const sourceText = redactSecrets(data.sourceText).slice(0, 1200); + return { + ...(data.sessionId ? { sessionId: data.sessionId } : {}), + ...(data.observationId ? { observationId: data.observationId } : {}), + ...(data.project ? { project: data.project } : {}), + ...(data.agentId ? { agentId: data.agentId } : {}), + scope: resolveScope(data), + sourceText, + evidenceQuote: sourceText.slice(0, 500), + }; +} + +function completeCandidate( + base: ReturnType, + fields: Pick< + MemoryWriteCandidate, + | "subject" + | "predicate" + | "value" + | "memoryType" + | "confidence" + | "importance" + | "target" + | "requiresReview" + | "reason" + >, +): MemoryWriteCandidate { + const candidate: MemoryWriteCandidate = { + id: generateId("cand"), + createdAt: nowIso(), + ...base, + ...fields, + value: redactSecrets(fields.value).slice(0, 500), + status: "shadow", + readbackQueries: [], + }; + candidate.readbackQueries = makeReadbackQueries(candidate); + return candidate; +} + +function extractCandidates(data: { + sourceText: string; + sessionId?: string; + observationId?: string; + project?: string; + agentId?: string; +}): MemoryWriteCandidate[] { + const text = data.sourceText.trim(); + if (!text || isLowSignal(text)) return []; + const base = baseCandidate(data); + + if (/(api[_ -]?key|token|凭据|credential|not logged in|登录)/i.test(text)) { + return [ + completeCandidate(base, { + subject: "tool_credential_route", + predicate: "derived_from_user_signal", + value: base.evidenceQuote, + memoryType: "credential_route", + confidence: 0.86, + importance: 0.9, + target: "review", + requiresReview: true, + reason: "Credential-related memory must be reviewed and redacted", + }), + ]; + } + + if (/(以后|下次|遇到|先|必须|一定要|不要|别).{0,80}(报错|错误|修复|检查|验证|记录|动手|执行)/i.test(text)) { + return [ + completeCandidate(base, { + subject: "agent_memory_workflow", + predicate: "procedural_rule", + value: base.evidenceQuote, + memoryType: "procedural_rule", + confidence: 0.88, + importance: 0.9, + target: "review", + requiresReview: true, + reason: "Future workflow instruction should be reviewed before durable write", + }), + ]; + } + + if (/(我|用户).{0,20}(更喜欢|喜欢|偏好|讨厌|不喜欢|关心|在意)/.test(text)) { + return [ + completeCandidate(base, { + subject: "user", + predicate: "preference", + value: base.evidenceQuote, + memoryType: "preference", + confidence: 0.82, + importance: 0.8, + target: "memory", + requiresReview: false, + reason: "Explicit user preference", + }), + ]; + } + + return []; +} + +function validStatus(value: unknown): value is MemoryWriteCandidate["status"] { + return ( + value === "shadow" || + value === "approved" || + value === "rejected" || + value === "written" || + value === "readback_failed" + ); +} + +export function registerWriteCandidatesFunction(sdk: ISdk, kv: StateKV): void { + sdk.registerFunction( + "mem::write-candidates-generate", + async (data: { + sourceText?: string; + sessionId?: string; + observationId?: string; + project?: string; + agentId?: string; + }) => { + const sourceText = asString(data?.sourceText, 5000); + if (!sourceText) return { success: false, error: "sourceText is required" }; + const payload = { + sourceText, + sessionId: asString(data.sessionId, 128), + observationId: asString(data.observationId, 128), + project: asString(data.project, 256), + agentId: asString(data.agentId, 128), + }; + const candidates = extractCandidates(payload); + await Promise.all( + candidates.map((candidate) => + kv.set(KV.writeCandidates, candidate.id, candidate), + ), + ); + if (candidates.length > 0) { + await recordAudit( + kv, + "write_candidate", + "mem::write-candidates-generate", + candidates.map((candidate) => candidate.id), + { generated: candidates.length }, + ); + } + return { success: true, candidates }; + }, + ); + + sdk.registerFunction( + "mem::write-candidates-list", + async (data?: { + status?: MemoryWriteCandidate["status"]; + project?: string; + agentId?: string; + limit?: number; + }) => { + const requestedLimit = data?.limit; + const limit = Number.isFinite(requestedLimit) + ? Math.max(1, Math.min(200, Math.floor(requestedLimit as number))) + : 50; + let candidates = await kv.list(KV.writeCandidates); + if (validStatus(data?.status)) { + candidates = candidates.filter((candidate) => candidate.status === data.status); + } + const project = asString(data?.project, 256); + if (project) candidates = candidates.filter((candidate) => candidate.project === project); + const agentId = asString(data?.agentId, 128); + if (agentId) candidates = candidates.filter((candidate) => candidate.agentId === agentId); + candidates.sort( + (a, b) => new Date(b.createdAt).getTime() - new Date(a.createdAt).getTime(), + ); + return { success: true, candidates: candidates.slice(0, limit) }; + }, + ); + + sdk.registerFunction( + "mem::write-candidates-review", + async (data: { candidateId?: string; decision?: string; reason?: string }) => { + const candidateId = asString(data?.candidateId, 128); + if (!candidateId) return { success: false, error: "candidateId is required" }; + if (data.decision !== "approve" && data.decision !== "reject") { + return { success: false, error: "decision must be approve or reject" }; + } + const candidate = await kv.get( + KV.writeCandidates, + candidateId, + ); + if (!candidate) return { success: false, error: "candidate not found" }; + const reviewed: MemoryWriteCandidate = { + ...candidate, + status: data.decision === "approve" ? "approved" : "rejected", + }; + await kv.set(KV.writeCandidates, reviewed.id, reviewed); + await recordAudit( + kv, + "write_candidate", + "mem::write-candidates-review", + [reviewed.id], + { + decision: data.decision, + reason: asString(data.reason, 500), + }, + ); + return { success: true, candidate: reviewed }; + }, + ); +} diff --git a/src/index.ts b/src/index.ts index 3428139f..b98b666e 100644 --- a/src/index.ts +++ b/src/index.ts @@ -83,6 +83,9 @@ import { registerWorkingMemoryFunctions } from "./functions/working-memory.js"; import { registerSkillExtractFunctions } from "./functions/skill-extract.js"; import { registerSlidingWindowFunction } from "./functions/sliding-window.js"; import { registerQueryExpansionFunction } from "./functions/query-expansion.js"; +import { registerMemoryPolicyFunction } from "./functions/memory-policy.js"; +import { registerWriteCandidatesFunction } from "./functions/write-candidates.js"; +import { registerReadbackFunction } from "./functions/readback.js"; import { registerTemporalGraphFunctions } from "./functions/temporal-graph.js"; import { registerRetentionFunctions } from "./functions/retention.js"; import { registerCompressFileFunction } from "./functions/compress-file.js"; @@ -326,6 +329,9 @@ async function main() { registerCascadeFunction(sdk, kv); registerSlidingWindowFunction(sdk, kv, provider); + registerMemoryPolicyFunction(sdk, kv); + registerWriteCandidatesFunction(sdk, kv); + registerReadbackFunction(sdk, kv); registerQueryExpansionFunction(sdk, provider); registerTemporalGraphFunctions(sdk, kv, provider); registerRetentionFunctions(sdk, kv); @@ -516,7 +522,7 @@ async function main() { `Ready. ${embeddingProvider ? "Triple-stream (BM25+Vector+Graph)" : "BM25+Graph"} search active.`, ); bootLog( - `REST API: 124 endpoints at http://localhost:${config.restPort}/agentmemory/*`, + `REST API: 132 endpoints at http://localhost:${config.restPort}/agentmemory/*`, ); bootLog( `MCP surface (opt-in via \`npx @agentmemory/mcp\`): ${getAllTools().length} tools · 6 resources · 3 prompts`, diff --git a/src/state/schema.ts b/src/state/schema.ts index 8d0d90f0..9a485076 100644 --- a/src/state/schema.ts +++ b/src/state/schema.ts @@ -47,6 +47,10 @@ export const KV = { globalSlots: "mem:slots:global", state: "mem:state", commits: "mem:commits", + memoryPolicy: "mem:policy", + writeCandidates: "mem:write-candidates", + readbackResults: "mem:readback", + policySuggestions: "mem:policy-suggestions", } as const; export const STREAM = { diff --git a/src/triggers/api.ts b/src/triggers/api.ts index 06a29282..56d4f819 100644 --- a/src/triggers/api.ts +++ b/src/triggers/api.ts @@ -1,5 +1,11 @@ import type { ISdk, ApiRequest } from "iii-sdk"; -import type { Session, CompressedObservation, HookPayload, CommitLink } from "../types.js"; +import type { + Session, + CompressedObservation, + HookPayload, + CommitLink, + MemoryWriteCandidate, +} from "../types.js"; import { withKeyedLock } from "../state/keyed-mutex.js"; import { KV } from "../state/schema.js"; import { StateKV } from "../state/kv.js"; @@ -115,6 +121,186 @@ function parseOptionalPositiveInt(value: unknown): number | undefined | null { return parsed; } +function badRequest(error: string): Response { + return { status_code: 400, body: { error } }; +} + +function requestBody(req: ApiRequest): Record { + if (!req.body || typeof req.body !== "object" || Array.isArray(req.body)) { + return {}; + } + return req.body as Record; +} + +function addOptionalString( + payload: Record, + field: string, + value: unknown, +): Response | null { + if (value === undefined || value === null || value === "") return null; + const text = asNonEmptyString(value); + if (!text) return badRequest(`${field} must be a non-empty string`); + payload[field] = text; + return null; +} + +function parseStringArrayField( + value: unknown, + field: string, +): string[] | Response | undefined { + if (value === undefined) return undefined; + if (!Array.isArray(value)) return badRequest(`${field} must be an array of strings`); + const strings: string[] = []; + for (const item of value) { + const text = asNonEmptyString(item); + if (!text) return badRequest(`${field} must be an array of non-empty strings`); + strings.push(text); + } + return strings; +} + +function isResponse(value: unknown): value is Response { + return !!value && typeof value === "object" && "status_code" in value; +} + +function parsePolicyUpdatePayload(body: Record): Record | Response { + const payload: Record = {}; + + if (body.queryExpansions !== undefined) { + if (!Array.isArray(body.queryExpansions)) { + return badRequest("queryExpansions must be an array"); + } + const rules: Record[] = []; + for (const raw of body.queryExpansions) { + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + return badRequest("queryExpansions must contain objects"); + } + const rule = raw as Record; + const id = asNonEmptyString(rule.id); + const trigger = asNonEmptyString(rule.trigger); + if (!id || !trigger) { + return badRequest("query expansion id and trigger are required strings"); + } + const expansions = parseStringArrayField(rule.expansions ?? [], "expansions"); + if (isResponse(expansions)) return expansions; + const sanitized: Record = { id, trigger, expansions }; + if (rule.scope !== undefined) { + if (rule.scope !== "global" && rule.scope !== "project") { + return badRequest("query expansion scope must be global or project"); + } + sanitized.scope = rule.scope; + } + const project = asNonEmptyString(rule.project); + if (project) sanitized.project = project; + if (rule.enabled !== undefined) { + if (typeof rule.enabled !== "boolean") { + return badRequest("query expansion enabled must be boolean"); + } + sanitized.enabled = rule.enabled; + } + rules.push(sanitized); + } + payload.queryExpansions = rules; + } + + if (body.writePolicy !== undefined) { + if (!body.writePolicy || typeof body.writePolicy !== "object" || Array.isArray(body.writePolicy)) { + return badRequest("writePolicy must be an object"); + } + const raw = body.writePolicy as Record; + const writePolicy: Record = {}; + if (raw.mode !== undefined) { + if (raw.mode !== "shadow" && raw.mode !== "limited_auto" && raw.mode !== "disabled") { + return badRequest("writePolicy.mode must be shadow, limited_auto, or disabled"); + } + writePolicy.mode = raw.mode; + } + const threshold = parseOptionalFiniteNumber(raw.autoWriteThreshold); + if (threshold === null) { + return badRequest("writePolicy.autoWriteThreshold must be a finite number"); + } + if (threshold !== undefined) writePolicy.autoWriteThreshold = threshold; + const allowedAutoTypes = parseStringArrayField( + raw.allowedAutoTypes, + "writePolicy.allowedAutoTypes", + ); + if (isResponse(allowedAutoTypes)) return allowedAutoTypes; + if (allowedAutoTypes !== undefined) writePolicy.allowedAutoTypes = allowedAutoTypes; + if (raw.neverAutoWriteShared !== undefined) { + if (typeof raw.neverAutoWriteShared !== "boolean") { + return badRequest("writePolicy.neverAutoWriteShared must be boolean"); + } + writePolicy.neverAutoWriteShared = raw.neverAutoWriteShared; + } + payload.writePolicy = writePolicy; + } + + if (body.preflightRules !== undefined) { + if (!Array.isArray(body.preflightRules)) { + return badRequest("preflightRules must be an array"); + } + const rules: Record[] = []; + for (const raw of body.preflightRules) { + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + return badRequest("preflightRules must contain objects"); + } + const rule = raw as Record; + const id = asNonEmptyString(rule.id); + const tool = asNonEmptyString(rule.tool); + const taskType = asNonEmptyString(rule.taskType); + if (!id || !tool || !taskType) { + return badRequest("preflight rule id, tool, and taskType are required strings"); + } + const triggerPatterns = parseStringArrayField( + rule.triggerPatterns ?? [], + "triggerPatterns", + ); + if (isResponse(triggerPatterns)) return triggerPatterns; + const sanitized: Record = { + id, + tool, + taskType, + triggerPatterns, + }; + if (rule.decision !== undefined) { + if ( + rule.decision !== "allow" && + rule.decision !== "warn" && + rule.decision !== "block" + ) { + return badRequest("preflight rule decision must be allow, warn, or block"); + } + sanitized.decision = rule.decision; + } + if (rule.enabled !== undefined) { + if (typeof rule.enabled !== "boolean") { + return badRequest("preflight rule enabled must be boolean"); + } + sanitized.enabled = rule.enabled; + } + rules.push(sanitized); + } + payload.preflightRules = rules; + } + + return payload; +} + +function parseListPayload( + params: Record, + fields: string[], +): Record | Response { + const payload: Record = {}; + for (const field of fields) { + const err = addOptionalString(payload, field, params[field]); + if (err) return err; + } + const limit = parseOptionalPositiveInt(params.limit); + if (limit === null) return badRequest("limit must be a positive integer"); + if (limit !== undefined) payload.limit = limit; + return payload; +} + export function registerApiTriggers( sdk: ISdk, kv: StateKV, @@ -816,6 +1002,234 @@ export function registerApiTriggers( config: { api_path: "/agentmemory/observations", http_method: "GET" }, }); + sdk.registerFunction("api::policy-get", async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const result = await sdk.trigger({ function_id: "mem::policy-get", payload: {} }); + return { status_code: 200, body: result }; + }); + sdk.registerTrigger({ + type: "http", + function_id: "api::policy-get", + config: { api_path: "/agentmemory/policy", http_method: "GET" }, + }); + + sdk.registerFunction("api::policy-update", async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const payload = parsePolicyUpdatePayload(requestBody(req)); + if (isResponse(payload)) return payload; + const result = await sdk.trigger({ function_id: "mem::policy-update", payload }); + return { status_code: 200, body: result }; + }); + sdk.registerTrigger({ + type: "http", + function_id: "api::policy-update", + config: { api_path: "/agentmemory/policy", http_method: "POST" }, + }); + + sdk.registerFunction( + "api::policy-expand-query", + async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const body = requestBody(req); + const query = asNonEmptyString(body.query); + if (!query) { + return badRequest("query is required and must be a non-empty string"); + } + const payload: Record = { query }; + const projectErr = addOptionalString(payload, "project", body.project); + if (projectErr) return projectErr; + const maxQueries = parseOptionalPositiveInt(body.maxQueries); + if (maxQueries === null) return badRequest("maxQueries must be a positive integer"); + if (maxQueries !== undefined) payload.maxQueries = maxQueries; + const result = await sdk.trigger({ + function_id: "mem::policy-expand-query", + payload, + }); + return { status_code: 200, body: result }; + }, + ); + sdk.registerTrigger({ + type: "http", + function_id: "api::policy-expand-query", + config: { + api_path: "/agentmemory/policy/expand-query", + http_method: "POST", + }, + }); + + sdk.registerFunction( + "api::write-candidates-generate", + async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const body = requestBody(req); + const sourceText = asNonEmptyString(body.sourceText); + if (!sourceText) { + return badRequest("sourceText is required and must be a non-empty string"); + } + const payload: Record = { sourceText }; + for (const field of ["sessionId", "observationId", "project", "agentId"]) { + const err = addOptionalString(payload, field, body[field]); + if (err) return err; + } + const result = await sdk.trigger({ + function_id: "mem::write-candidates-generate", + payload, + }); + return { status_code: 200, body: result }; + }, + ); + sdk.registerTrigger({ + type: "http", + function_id: "api::write-candidates-generate", + config: { + api_path: "/agentmemory/write-candidates/generate", + http_method: "POST", + }, + }); + + sdk.registerFunction( + "api::write-candidates-list", + async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const params = req.query_params || {}; + const payload = parseListPayload(params, ["project", "agentId"]); + if (isResponse(payload)) return payload; + const status = params.status as unknown; + if (status !== undefined && status !== "") { + if ( + status !== "shadow" && + status !== "approved" && + status !== "rejected" && + status !== "written" && + status !== "readback_failed" + ) { + return badRequest( + "status must be shadow, approved, rejected, written, or readback_failed", + ); + } + payload.status = status satisfies MemoryWriteCandidate["status"]; + } + const result = await sdk.trigger({ + function_id: "mem::write-candidates-list", + payload, + }); + return { status_code: 200, body: result }; + }, + ); + sdk.registerTrigger({ + type: "http", + function_id: "api::write-candidates-list", + config: { + api_path: "/agentmemory/write-candidates", + http_method: "GET", + }, + }); + + sdk.registerFunction( + "api::write-candidates-review", + async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const body = requestBody(req); + const candidateId = asNonEmptyString(body.candidateId); + if (!candidateId) { + return badRequest("candidateId is required and must be a non-empty string"); + } + if (body.decision !== "approve" && body.decision !== "reject") { + return badRequest("decision must be approve or reject"); + } + const payload: Record = { + candidateId, + decision: body.decision, + }; + const reasonErr = addOptionalString(payload, "reason", body.reason); + if (reasonErr) return reasonErr; + const result = await sdk.trigger({ + function_id: "mem::write-candidates-review", + payload, + }); + return { status_code: 200, body: result }; + }, + ); + sdk.registerTrigger({ + type: "http", + function_id: "api::write-candidates-review", + config: { + api_path: "/agentmemory/write-candidates/review", + http_method: "POST", + }, + }); + + sdk.registerFunction( + "api::readback-verify", + async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const body = requestBody(req); + const payload: Record = {}; + const candidateErr = addOptionalString(payload, "candidateId", body.candidateId); + if (candidateErr) return candidateErr; + const memoryErr = addOptionalString(payload, "memoryId", body.memoryId); + if (memoryErr) return memoryErr; + if (!payload.candidateId && !payload.memoryId) { + return badRequest("candidateId or memoryId is required"); + } + const queries = parseStringArrayField(body.queries, "queries"); + if (isResponse(queries)) return queries; + if (queries !== undefined) payload.queries = queries; + const limit = parseOptionalPositiveInt(body.limit); + if (limit === null) return badRequest("limit must be a positive integer"); + if (limit !== undefined) payload.limit = limit; + if (body.mode !== undefined) { + if (body.mode !== "search" && body.mode !== "smart-search") { + return badRequest("mode must be search or smart-search"); + } + payload.mode = body.mode; + } + const result = await sdk.trigger({ + function_id: "mem::readback-verify", + payload, + }); + return { status_code: 200, body: result }; + }, + ); + sdk.registerTrigger({ + type: "http", + function_id: "api::readback-verify", + config: { + api_path: "/agentmemory/readback/verify", + http_method: "POST", + }, + }); + + sdk.registerFunction( + "api::readback-list", + async (req: ApiRequest): Promise => { + const authErr = checkAuth(req, secret); + if (authErr) return authErr; + const payload = parseListPayload(req.query_params || {}, [ + "candidateId", + "memoryId", + ]); + if (isResponse(payload)) return payload; + const result = await sdk.trigger({ + function_id: "mem::readback-list", + payload, + }); + return { status_code: 200, body: result }; + }, + ); + sdk.registerTrigger({ + type: "http", + function_id: "api::readback-list", + config: { api_path: "/agentmemory/readback", http_method: "GET" }, + }); + sdk.registerFunction("api::file-context", async ( req: ApiRequest<{ sessionId: string; files: string[] }>, diff --git a/src/types.ts b/src/types.ts index 97a7457b..65817a46 100644 --- a/src/types.ts +++ b/src/types.ts @@ -234,6 +234,108 @@ export interface MemorySlot { updatedAt: string; } +export interface QueryExpansionRule { + id: string; + trigger: string; + expansions: string[]; + scope?: "global" | "project"; + project?: string; + enabled: boolean; + createdAt: string; + updatedAt: string; +} + +export interface MemoryWritePolicy { + mode: "shadow" | "limited_auto" | "disabled"; + autoWriteThreshold: number; + allowedAutoTypes: MemoryWriteCandidate["memoryType"][]; + neverAutoWriteShared: boolean; +} + +export interface PreflightRule { + id: string; + tool: string; + taskType: string; + triggerPatterns: string[]; + decision: "allow" | "warn" | "block"; + enabled: boolean; + createdAt: string; + updatedAt: string; +} + +export interface MemoryPolicy { + id: "default"; + updatedAt: string; + queryExpansions: QueryExpansionRule[]; + writePolicy: MemoryWritePolicy; + preflightRules: PreflightRule[]; +} + +export interface MemoryWriteCandidate { + id: string; + sessionId?: string; + observationId?: string; + project?: string; + agentId?: string; + scope: "global" | "project" | "agent"; + createdAt: string; + sourceText: string; + evidenceQuote: string; + subject: string; + predicate: string; + value: string; + memoryType: + | "fact" + | "preference" + | "architecture" + | "bug" + | "workflow" + | "lesson" + | "procedural_rule" + | "credential_route" + | "temporary" + | "ignore"; + confidence: number; + importance: number; + target: "memory" | "lesson" | "slot" | "review" | "ignore"; + requiresReview: boolean; + reason: string; + readbackQueries: string[]; + status: "shadow" | "approved" | "rejected" | "written" | "readback_failed"; +} + +export interface ReadbackResult { + id: string; + candidateId?: string; + memoryId?: string; + createdAt: string; + queries: Array<{ + query: string; + topIds: string[]; + matched: boolean; + }>; + passed: boolean; + failureReason?: string; +} + +export interface PolicySuggestion { + id: string; + lessonId?: string; + createdAt: string; + type: + | "query_expansion" + | "preflight" + | "context_disclosure" + | "slot" + | "memory_only"; + confidence: number; + scope: "global" | "project"; + project?: string; + proposal: Record; + status: "proposed" | "approved" | "rejected" | "applied"; + reasoning: string; +} + export interface EmbeddingProvider { name: string; dimensions: number; @@ -555,7 +657,10 @@ export interface AuditEntry { | "slot_replace" | "slot_create" | "slot_delete" - | "slot_reflect"; + | "slot_reflect" + | "policy_update" + | "write_candidate" + | "readback_verify"; userId?: string; functionId: string; targetIds: string[]; diff --git a/test/api-memory-metacognition.test.ts b/test/api-memory-metacognition.test.ts new file mode 100644 index 00000000..9578d3d7 --- /dev/null +++ b/test/api-memory-metacognition.test.ts @@ -0,0 +1,249 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import type { ApiRequest } from "iii-sdk"; +import { registerApiTriggers } from "../src/triggers/api.js"; +import { mockKV, mockSdk } from "./helpers/mocks.js"; + +vi.mock("../src/logger.js", () => ({ + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})); + +describe("memory metacognition REST API", () => { + let sdk: ReturnType; + + beforeEach(() => { + sdk = mockSdk(); + }); + + function register(secret = "test-secret") { + registerApiTriggers(sdk as never, mockKV() as never, secret); + } + + function request( + body?: Record, + query_params: Record = {}, + headers: Record = { authorization: "Bearer test-secret" }, + ): ApiRequest { + return { body, query_params, headers } as unknown as ApiRequest; + } + + it("enforces auth for policy reads", async () => { + register(); + const result = await sdk.trigger( + "api::policy-get", + request(undefined, {}, {}), + ); + + expect(result).toEqual({ + status_code: 401, + body: { error: "unauthorized" }, + }); + }); + + it("registers exactly the Phase 1 endpoint paths", () => { + register(); + + expect(sdk.registerTrigger).toHaveBeenCalledWith( + expect.objectContaining({ + type: "http", + function_id: "api::policy-get", + config: expect.objectContaining({ + api_path: "/agentmemory/policy", + http_method: "GET", + }), + }), + ); + expect(sdk.registerTrigger).toHaveBeenCalledWith( + expect.objectContaining({ + type: "http", + function_id: "api::readback-list", + config: expect.objectContaining({ + api_path: "/agentmemory/readback", + http_method: "GET", + }), + }), + ); + }); + + it("whitelists policy update fields before triggering mem::policy-update", async () => { + let payload: unknown; + sdk.registerFunction("mem::policy-update", async (data) => { + payload = data; + return { success: true, policy: data }; + }); + register(); + + const result = await sdk.trigger( + "api::policy-update", + request({ + queryExpansions: [ + { + id: "rule_1", + trigger: "报错", + expansions: ["error", "fix"], + scope: "project", + project: "agentmemory", + enabled: true, + ignored: "drop me", + }, + ], + writePolicy: { + mode: "shadow", + autoWriteThreshold: 0.9, + allowedAutoTypes: ["preference"], + neverAutoWriteShared: true, + ignored: "drop me", + }, + preflightRules: [], + ignored: "drop me", + }), + ); + + expect(result).toMatchObject({ status_code: 200 }); + expect(payload).toEqual({ + queryExpansions: [ + { + id: "rule_1", + trigger: "报错", + expansions: ["error", "fix"], + scope: "project", + project: "agentmemory", + enabled: true, + }, + ], + writePolicy: { + mode: "shadow", + autoWriteThreshold: 0.9, + allowedAutoTypes: ["preference"], + neverAutoWriteShared: true, + }, + preflightRules: [], + }); + }); + + it("rejects invalid expand-query requests before triggering memory functions", async () => { + const mem = vi.fn(); + sdk.registerFunction("mem::policy-expand-query", async (data) => mem(data)); + register(); + + const result = await sdk.trigger( + "api::policy-expand-query", + request({ maxQueries: 3 }), + ); + + expect(result).toEqual({ + status_code: 400, + body: { error: "query is required and must be a non-empty string" }, + }); + expect(mem).not.toHaveBeenCalled(); + }); + + it("whitelists write candidate generation and review payloads", async () => { + const calls: unknown[] = []; + sdk.registerFunction("mem::write-candidates-generate", async (data) => { + calls.push(data); + return { success: true, candidates: [] }; + }); + sdk.registerFunction("mem::write-candidates-review", async (data) => { + calls.push(data); + return { success: true, candidate: data }; + }); + register(); + + await sdk.trigger( + "api::write-candidates-generate", + request({ + sourceText: "我更喜欢短句", + sessionId: "s1", + observationId: "o1", + project: "agentmemory", + agentId: "codex", + ignored: "drop me", + }), + ); + await sdk.trigger( + "api::write-candidates-review", + request({ + candidateId: "cand_1", + decision: "approve", + reason: "explicit preference", + ignored: "drop me", + }), + ); + + expect(calls).toEqual([ + { + sourceText: "我更喜欢短句", + sessionId: "s1", + observationId: "o1", + project: "agentmemory", + agentId: "codex", + }, + { + candidateId: "cand_1", + decision: "approve", + reason: "explicit preference", + }, + ]); + }); + + it("rejects invalid write candidate review decisions", async () => { + const mem = vi.fn(); + sdk.registerFunction("mem::write-candidates-review", async (data) => mem(data)); + register(); + + const result = await sdk.trigger( + "api::write-candidates-review", + request({ candidateId: "cand_1", decision: "maybe" }), + ); + + expect(result).toEqual({ + status_code: 400, + body: { error: "decision must be approve or reject" }, + }); + expect(mem).not.toHaveBeenCalled(); + }); + + it("maps readback verify and list to whitelisted payloads", async () => { + const calls: unknown[] = []; + sdk.registerFunction("mem::readback-verify", async (data) => { + calls.push(data); + return { success: true, readback: data }; + }); + sdk.registerFunction("mem::readback-list", async (data) => { + calls.push(data); + return { success: true, readbacks: [] }; + }); + register(); + + await sdk.trigger( + "api::readback-verify", + request({ + candidateId: "cand_1", + memoryId: "mem_1", + queries: ["短句偏好"], + limit: 5, + mode: "smart-search", + ignored: "drop me", + }), + ); + await sdk.trigger( + "api::readback-list", + request(undefined, { + candidateId: "cand_1", + memoryId: "mem_1", + limit: "5", + }), + ); + + expect(calls).toEqual([ + { + candidateId: "cand_1", + memoryId: "mem_1", + queries: ["短句偏好"], + limit: 5, + mode: "smart-search", + }, + { candidateId: "cand_1", memoryId: "mem_1", limit: 5 }, + ]); + }); +}); diff --git a/test/memory-policy-types.test.ts b/test/memory-policy-types.test.ts new file mode 100644 index 00000000..356c4c33 --- /dev/null +++ b/test/memory-policy-types.test.ts @@ -0,0 +1,11 @@ +import { describe, expect, it } from "vitest"; +import { KV } from "../src/state/schema.js"; + +describe("memory metacognition KV scopes", () => { + it("defines stable scopes for policy, candidates, readback, and suggestions", () => { + expect(KV.memoryPolicy).toBe("mem:policy"); + expect(KV.writeCandidates).toBe("mem:write-candidates"); + expect(KV.readbackResults).toBe("mem:readback"); + expect(KV.policySuggestions).toBe("mem:policy-suggestions"); + }); +}); diff --git a/test/memory-policy.test.ts b/test/memory-policy.test.ts new file mode 100644 index 00000000..3cc152b6 --- /dev/null +++ b/test/memory-policy.test.ts @@ -0,0 +1,140 @@ +import { beforeEach, describe, expect, it } from "vitest"; +import { registerMemoryPolicyFunction } from "../src/functions/memory-policy.js"; +import { KV } from "../src/state/schema.js"; +import type { MemoryPolicy } from "../src/types.js"; +import { mockKV, mockSdk } from "./helpers/mocks.js"; + +describe("Memory Policy", () => { + let sdk: ReturnType; + let kv: ReturnType; + + beforeEach(() => { + sdk = mockSdk(); + kv = mockKV(); + registerMemoryPolicyFunction(sdk as never, kv as never); + }); + + it("returns a conservative default policy when no policy is saved", async () => { + const result = (await sdk.trigger("mem::policy-get", {})) as { + success: boolean; + policy: MemoryPolicy; + }; + + expect(result.success).toBe(true); + expect(result.policy.id).toBe("default"); + expect(result.policy.writePolicy.mode).toBe("shadow"); + expect(result.policy.writePolicy.neverAutoWriteShared).toBe(true); + expect(result.policy.queryExpansions).toEqual([]); + expect(result.policy.preflightRules).toEqual([]); + }); + + it("updates policy with validated query expansion rules", async () => { + const result = (await sdk.trigger("mem::policy-update", { + queryExpansions: [ + { + id: "cfg", + trigger: "配置", + expansions: ["config.yaml", "provider", ""], + scope: "project", + project: "agentmemory", + enabled: true, + }, + ], + writePolicy: { + mode: "shadow", + autoWriteThreshold: 0.9, + allowedAutoTypes: ["preference", "workflow", "bad-type"], + neverAutoWriteShared: true, + }, + preflightRules: [], + })) as { success: boolean; policy: MemoryPolicy }; + + expect(result.success).toBe(true); + expect(result.policy.queryExpansions[0]).toMatchObject({ + id: "cfg", + trigger: "配置", + expansions: ["config.yaml", "provider"], + scope: "project", + project: "agentmemory", + enabled: true, + }); + expect(result.policy.writePolicy.allowedAutoTypes).toEqual([ + "preference", + "workflow", + ]); + expect(await kv.get(KV.memoryPolicy, "default")).toEqual(result.policy); + }); + + it("expands queries using enabled global and matching project rules", async () => { + await sdk.trigger("mem::policy-update", { + queryExpansions: [ + { + id: "global-config", + trigger: "配置", + expansions: ["config.yaml", "provider"], + scope: "global", + enabled: true, + }, + { + id: "project-gateway", + trigger: "配置", + expansions: ["gateway", "service"], + scope: "project", + project: "agentmemory", + enabled: true, + }, + ], + }); + + const result = (await sdk.trigger("mem::policy-expand-query", { + query: "改配置", + project: "agentmemory", + maxQueries: 6, + })) as { success: boolean; expansion: { original: string; reformulations: string[] } }; + + expect(result.success).toBe(true); + expect(result.expansion.original).toBe("改配置"); + expect(result.expansion.reformulations).toEqual([ + "config.yaml", + "provider", + "gateway", + "service", + ]); + }); + + it("ignores disabled and non-matching project rules while deduplicating expansions", async () => { + await sdk.trigger("mem::policy-update", { + queryExpansions: [ + { + id: "disabled", + trigger: "配置", + expansions: ["disabled-term"], + enabled: false, + }, + { + id: "other-project", + trigger: "配置", + expansions: ["other-project-term"], + scope: "project", + project: "other", + enabled: true, + }, + { + id: "global", + trigger: "配置", + expansions: ["config.yaml", "config.yaml", "provider"], + enabled: true, + }, + ], + }); + + const result = (await sdk.trigger("mem::policy-expand-query", { + query: "配置 provider", + project: "agentmemory", + maxQueries: 3, + })) as { success: boolean; expansion: { reformulations: string[] } }; + + expect(result.success).toBe(true); + expect(result.expansion.reformulations).toEqual(["config.yaml", "provider"]); + }); +}); diff --git a/test/readback.test.ts b/test/readback.test.ts new file mode 100644 index 00000000..73175aa7 --- /dev/null +++ b/test/readback.test.ts @@ -0,0 +1,147 @@ +import { beforeEach, describe, expect, it } from "vitest"; +import { registerReadbackFunction } from "../src/functions/readback.js"; +import { KV } from "../src/state/schema.js"; +import type { Memory, MemoryWriteCandidate, ReadbackResult } from "../src/types.js"; +import { mockKV, mockSdk } from "./helpers/mocks.js"; + +describe("Readback verification", () => { + let sdk: ReturnType; + let kv: ReturnType; + + beforeEach(() => { + sdk = mockSdk(); + kv = mockKV(); + registerReadbackFunction(sdk as never, kv as never); + }); + + it("passes when a memory id appears in search results", async () => { + const memory: Memory = { + id: "mem_react", + createdAt: "2026-05-27T00:00:00Z", + updatedAt: "2026-05-27T00:00:00Z", + type: "architecture", + title: "React frontend", + content: "The frontend uses React", + concepts: ["react", "frontend"], + files: ["src/App.tsx"], + sessionIds: [], + strength: 7, + version: 1, + isLatest: true, + }; + await kv.set(KV.memories, memory.id, memory); + sdk.registerFunction("mem::search", async () => ({ + results: [{ observation: { id: "mem_react" } }], + })); + + const result = (await sdk.trigger("mem::readback-verify", { + memoryId: "mem_react", + mode: "search", + limit: 5, + })) as { success: boolean; readback: ReadbackResult }; + + expect(result.success).toBe(true); + expect(result.readback.passed).toBe(true); + expect(result.readback.memoryId).toBe("mem_react"); + expect(result.readback.queries.length).toBeGreaterThanOrEqual(2); + expect(result.readback.queries.some((q) => q.matched)).toBe(true); + + const stored = await kv.list(KV.readbackResults); + expect(stored).toHaveLength(1); + expect(stored[0].passed).toBe(true); + }); + + it("fails and persists a readback result when target memory is not found", async () => { + await kv.set(KV.memories, "mem_missing", { + id: "mem_missing", + title: "Missing memory", + content: "This should not be found", + concepts: [], + files: [], + sessionIds: [], + type: "fact", + strength: 1, + version: 1, + isLatest: true, + createdAt: "2026-05-27T00:00:00Z", + updatedAt: "2026-05-27T00:00:00Z", + } satisfies Memory); + sdk.registerFunction("mem::search", async () => ({ + results: [{ observation: { id: "other" } }], + })); + + const result = (await sdk.trigger("mem::readback-verify", { + memoryId: "mem_missing", + queries: ["missing memory"], + })) as { success: boolean; readback: ReadbackResult }; + + expect(result.success).toBe(true); + expect(result.readback.passed).toBe(false); + expect(result.readback.failureReason).toBe("target not found in top results"); + expect((await kv.list(KV.readbackResults))[0].passed).toBe(false); + }); + + it("candidate-only readback records query previews without claiming pass", async () => { + const candidate: MemoryWriteCandidate = { + id: "cand_1", + createdAt: "2026-05-27T00:00:00Z", + scope: "project", + sourceText: "以后先查修复记录", + evidenceQuote: "以后先查修复记录", + subject: "agent_memory_workflow", + predicate: "procedural_rule", + value: "以后先查修复记录", + memoryType: "procedural_rule", + confidence: 0.9, + importance: 0.9, + target: "review", + requiresReview: true, + reason: "workflow", + readbackQueries: ["agent_memory_workflow procedural_rule", "修复记录"], + status: "shadow", + }; + await kv.set(KV.writeCandidates, candidate.id, candidate); + + const result = (await sdk.trigger("mem::readback-verify", { + candidateId: "cand_1", + })) as { success: boolean; readback: ReadbackResult }; + + expect(result.success).toBe(true); + expect(result.readback.candidateId).toBe("cand_1"); + expect(result.readback.passed).toBe(false); + expect(result.readback.failureReason).toBe( + "candidate has no durable memoryId yet", + ); + expect(result.readback.queries.map((q) => q.query)).toEqual(candidate.readbackQueries); + }); + + it("extracts ids from smart-search compact results", async () => { + await kv.set(KV.memories, "mem_smart", { + id: "mem_smart", + title: "Smart memory", + content: "Hybrid retrieval finds this", + concepts: [], + files: [], + sessionIds: [], + type: "fact", + strength: 1, + version: 1, + isLatest: true, + createdAt: "2026-05-27T00:00:00Z", + updatedAt: "2026-05-27T00:00:00Z", + } satisfies Memory); + sdk.registerFunction("mem::smart-search", async () => ({ + mode: "compact", + results: [{ obsId: "mem_smart" }], + })); + + const result = (await sdk.trigger("mem::readback-verify", { + memoryId: "mem_smart", + mode: "smart-search", + queries: ["smart memory"], + })) as { readback: ReadbackResult }; + + expect(result.readback.passed).toBe(true); + expect(result.readback.queries[0].topIds).toEqual(["mem_smart"]); + }); +}); diff --git a/test/write-candidates.test.ts b/test/write-candidates.test.ts new file mode 100644 index 00000000..448ac5ca --- /dev/null +++ b/test/write-candidates.test.ts @@ -0,0 +1,111 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import { registerWriteCandidatesFunction } from "../src/functions/write-candidates.js"; +import { KV } from "../src/state/schema.js"; +import type { MemoryWriteCandidate } from "../src/types.js"; +import { mockKV, mockSdk } from "./helpers/mocks.js"; + +vi.mock("../src/logger.js", () => ({ + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})); + +describe("Memory write candidates", () => { + let sdk: ReturnType; + let kv: ReturnType; + + beforeEach(() => { + sdk = mockSdk(); + kv = mockKV(); + registerWriteCandidatesFunction(sdk as never, kv as never); + }); + + it("extracts explicit user preferences into shadow candidates", async () => { + const result = (await sdk.trigger("mem::write-candidates-generate", { + sourceText: "我更喜欢简洁直接的回答", + project: "agentmemory", + agentId: "codex", + })) as { success: boolean; candidates: MemoryWriteCandidate[] }; + + expect(result.success).toBe(true); + expect(result.candidates).toHaveLength(1); + expect(result.candidates[0]).toMatchObject({ + memoryType: "preference", + target: "memory", + status: "shadow", + project: "agentmemory", + agentId: "codex", + scope: "agent", + requiresReview: false, + }); + expect(result.candidates[0].confidence).toBeGreaterThanOrEqual(0.75); + expect(result.candidates[0].readbackQueries.length).toBeGreaterThan(0); + + const stored = await kv.list(KV.writeCandidates); + expect(stored.map((c) => c.id)).toEqual([result.candidates[0].id]); + }); + + it("extracts procedural rules from future workflow instructions", async () => { + const result = (await sdk.trigger("mem::write-candidates-generate", { + sourceText: "以后遇到这种报错，先查之前的修复记录再动手", + project: "agentmemory", + })) as { success: boolean; candidates: MemoryWriteCandidate[] }; + + expect(result.candidates).toHaveLength(1); + expect(result.candidates[0]).toMatchObject({ + memoryType: "procedural_rule", + target: "review", + requiresReview: true, + scope: "project", + status: "shadow", + }); + expect(result.candidates[0].importance).toBeGreaterThanOrEqual(0.85); + }); + + it("ignores low-signal acknowledgements", async () => { + const result = (await sdk.trigger("mem::write-candidates-generate", { + sourceText: "哈哈可以", + })) as { success: boolean; candidates: MemoryWriteCandidate[] }; + + expect(result.success).toBe(true); + expect(result.candidates).toEqual([]); + expect(await kv.list(KV.writeCandidates)).toEqual([]); + }); + + it("redacts secret-like values before persisting candidates", async () => { + const result = (await sdk.trigger("mem::write-candidates-generate", { + sourceText: "我的 API key 是 sk-test-secret-value，以后先查凭据路径", + agentId: "codex", + })) as { success: boolean; candidates: MemoryWriteCandidate[] }; + + expect(result.candidates).toHaveLength(1); + const candidate = result.candidates[0]; + expect(candidate.memoryType).toBe("credential_route"); + expect(candidate.requiresReview).toBe(true); + expect(candidate.sourceText).not.toContain("sk-test-secret-value"); + expect(candidate.evidenceQuote).not.toContain("sk-test-secret-value"); + expect(candidate.value).not.toContain("sk-test-secret-value"); + expect(candidate.value).toContain("[REDACTED_SECRET]"); + }); + + it("reviews candidates without writing durable memories", async () => { + const generated = (await sdk.trigger("mem::write-candidates-generate", { + sourceText: "我更喜欢短句", + })) as { candidates: MemoryWriteCandidate[] }; + const candidateId = generated.candidates[0].id; + + const approved = (await sdk.trigger("mem::write-candidates-review", { + candidateId, + decision: "approve", + reason: "explicit preference", + })) as { success: boolean; candidate: MemoryWriteCandidate }; + + expect(approved.success).toBe(true); + expect(approved.candidate.status).toBe("approved"); + expect(await kv.list(KV.memories)).toEqual([]); + + const rejected = (await sdk.trigger("mem::write-candidates-review", { + candidateId, + decision: "reject", + })) as { success: boolean; candidate: MemoryWriteCandidate }; + expect(rejected.candidate.status).toBe("rejected"); + }); +});