Skip to content

Add coding-agent memory dataset seed#22

Open
koriyoshi2041 wants to merge 1 commit into
vectorize-io:mainfrom
koriyoshi2041:rios/coding-agent-memory-dataset
Open

Add coding-agent memory dataset seed#22
koriyoshi2041 wants to merge 1 commit into
vectorize-io:mainfrom
koriyoshi2041:rios/coding-agent-memory-dataset

Conversation

@koriyoshi2041

Copy link
Copy Markdown

What

  • add a small codingagent synthetic dataset with three multi-session engineering-memory cases
  • cover CI recurrence, review preference reuse, and stale docs/release boundary behavior
  • register the dataset with the existing CLI registry

Why

This gives the coding-agent memory evaluation discussion in vectorize-io/hindsight#2347 a small runnable seed inside the benchmark harness, without changing scoring or runner behavior.

Checks

  • uv run omb providers
  • uv run omb splits --dataset codingagent
  • uv run omb dataset-stats --dataset codingagent
  • lightweight loader/BM25 smoke check via uv run python
  • uv run python -m compileall src/memory_bench/dataset/codingagent.py src/memory_bench/dataset/__init__.py
  • git diff --check

@vercel

vercel Bot commented Jun 24, 2026

Copy link
Copy Markdown

@koriyoshi2041 is attempting to deploy a commit to the Vectorize Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant