Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: CI

on:
push:
branches: [main]
pull_request:

# Read-only, no secrets. Everything the suite needs is local: the log.sh tests
# stand up an http.server on loopback, the scaffolding tests write to tmp_path,
# and the GitHub integration tests mock subprocess. Keep it that way — CI has to
# stay free and fast so it can gate every PR, unlike Genesis Evolver which burns
# an ANTHROPIC_API_KEY per run.
permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7

# Exact release, not `@v9`. setup-uv stopped publishing floating major
# tags after v7 — `@v9` does not resolve and fails the job at setup.
- uses: astral-sh/setup-uv@v9.0.0
with:
# Test the floor declared in pyproject.toml (requires-python >=3.12),
# not whatever newer interpreter the runner happens to ship.
python-version: "3.12"
enable-cache: true

# --locked, not --frozen. Both install from the committed lock, but they
# disagree about a *stale* one: --frozen installs it anyway, so a PR that
# adds a dependency to pyproject.toml without re-locking goes green here
# and breaks for everyone else. Measured on this repo — dependency added,
# no re-lock: `uv sync --frozen` exits 0 having installed the old set,
# `uv sync --locked` exits 1 with "the lockfile needs to be updated".
- run: uv sync --locked

# --no-sync so the test step itself cannot touch the environment or the
# lock. It is also the only invocation that stays off the package index
# entirely: --frozen pins the dependency *graph*, but building this
# project still resolves `build-system.requires` (hatchling), which no
# lockfile covers, against whatever index is configured. So --frozen is
# not the shield against a proxy registry that it was assumed to be — the
# [[tool.uv.index]] pin in pyproject.toml is, and this ordering keeps the
# index out of the picture after the one sync above.
- run: uv run --no-sync pytest tests/ -q

# Belt and braces: if a change ever drops the flags above, this fails
# loudly instead of shipping a rewritten lock.
- name: uv.lock is unchanged by the test run
run: git diff --exit-code uv.lock
26 changes: 26 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,32 @@ template is unclassified or declares no `--max-turns` at all. Two separate dev-s
workflows died at 20 turns three weeks apart before this floor existed — when a run
dies at max-turns, raise the whole class and record why.

## CI

`.github/workflows/ci.yml` runs the suite on every push to `main` and every PR. It is
what turns the guards above from convention into enforcement — before it existed, the
suite ran only when a human remembered to, and several changes merged on the strength
of a hand-run result.

Two properties keep it usable, both asserted by `tests/e2e/test_workflows.py`:

- **No secrets.** CI must stay free and fast so it can gate every PR. The paid
Claude-invoking workflows are separate; never gate a PR on them.
- **`uv sync --locked` then `uv run --no-sync`, never `--frozen`.** `--frozen` reads
as the strict option and is not: it installs a stale lock without complaint, so a
dependency added to `pyproject.toml` without a re-lock passes CI and breaks
everyone else. `--locked` fails the run instead. `--frozen` also does not keep uv
off the configured package index, because building this project resolves
`build-system.requires` (hatchling), which no lockfile covers — the thing that
actually stops a proxy registry from rewriting source URLs is the
`[[tool.uv.index]]` pin in `pyproject.toml`, not a flag on the run command. A
`git diff --exit-code uv.lock` step catches a regression in any of this.

The suite must stay hermetic: no network, no ambient config. `tests/conftest.py` sets
`GIT_AUTHOR_*`/`GIT_COMMITTER_*` because the scaffolding tests end in a real `git
commit`, which aborts on a runner with no global identity. If a new test needs
credentials or a live service, it does not belong in this suite.

## Self-Improvement

This project opts in to self-improvement. Update this CLAUDE.md and project workflows as the design evolves. Keep `docs/` as the living design documents.
27 changes: 27 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
"""Root fixtures — keep the suite hermetic.

Everything here exists so `pytest tests/` passes on a machine that has nothing
configured: no ~/.gitconfig, no ~/.config/genesis, no network.
"""

import pytest


@pytest.fixture(autouse=True)
def git_identity(monkeypatch: pytest.MonkeyPatch) -> None:
"""Give git an author/committer so scaffold commits don't need global config.

`scaffold_new_repo` ends in `git commit`, which aborts with exit 128 and
"Author identity unknown" when neither ~/.gitconfig nor the env supplies
one. A fresh GitHub Actions runner has no global identity, so 19 tests
failed there while passing on every developer machine — the suite silently
depended on ambient state. These env vars override config, so they hold
regardless of what the host has set.
"""
for var, value in (
("GIT_AUTHOR_NAME", "genesis-tests"),
("GIT_AUTHOR_EMAIL", "tests@genesis.invalid"),
("GIT_COMMITTER_NAME", "genesis-tests"),
("GIT_COMMITTER_EMAIL", "tests@genesis.invalid"),
):
monkeypatch.setenv(var, value)
38 changes: 38 additions & 0 deletions tests/e2e/test_workflows.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,44 @@ def test_genesis_own_claude_workflows_meet_orchestrator_floor() -> None:
)


def test_ci_workflow_runs_the_suite_on_every_pr_without_secrets() -> None:
"""The guards in this file are only guards if something runs them.

Before ci.yml existed, the whole suite ran only when a human remembered to,
which made every assertion here advisory. Two properties keep it that way:
it must fire on `pull_request`, and it must need no secrets — a CI job that
costs money or an API key is one someone eventually disables.
"""
content = (REPO_ROOT / ".github" / "workflows" / "ci.yml").read_text()
assert "pull_request:" in content
# Match the commands, not the file. The comments in ci.yml discuss --frozen
# at length to explain why it is *not* used, and scanning raw text flags
# that prose — same reason the secrets check below matches on interpolation.
commands = "\n".join(
line
for line in content.splitlines()
if line.strip().lstrip("- ").startswith("run:")
)
# --locked, not --frozen: --frozen installs a stale lock without complaint,
# so a dependency added to pyproject.toml without a re-lock would pass CI.
assert "uv sync --locked" in commands, (
"CI must install with --locked so a stale uv.lock fails the run"
)
assert "uv run --no-sync pytest" in commands, (
"CI must run tests with --no-sync so the test step cannot touch the lock"
)
assert "--frozen" not in commands, (
"--frozen is not the guard it looks like: it accepts a stale lock, and "
"it does not keep uv off the configured index (build-system.requires is "
"resolved outside the lock). Use --locked + --no-sync."
)
# A workflow can only consume a secret through this interpolation, so match
# on it rather than the bare word — prose in a comment is not a secret.
assert "${{ secrets." not in content, (
"CI must not consume secrets — it has to run on every PR for free"
)


def test_scaffolded_workflows_match_templates(tmp_dir: Path) -> None:
repo = tmp_dir / "test-project"
scaffold_new_repo(repo, "test goal", "test-project")
Expand Down