Skip to content

No CI workflow: the test suite (including the new turn-budget guard) never runs automatically #37

Description

@the-gigi

Problem

Genesis has no CI workflow. gh workflow list --all shows exactly two: Genesis Evolver and Dependabot Updates. Nothing runs pytest on a push or a pull request.

That means the 124-test suite only executes when a human remembers to run it locally, which makes every guard in it advisory rather than enforced. #36 just landed a turn-budget floor guard - including test_genesis_own_claude_workflows_meet_orchestrator_floor, which asserts genesis's own workflows meet the floor. The test exists. The enforcement does not. A future change can lower a budget, drop a --max-turns, or add an unclassified Claude-invoking template, and nothing will notice.

Same gap covers everything else the suite protects: the scaffolding e2e tests, the workflow permission assertions, the Loki secret-forwarding guard added in ffe4492, and the log.sh behavior tests added in bda9638.

Concretely, both #36 and the two commits before it were merged on the strength of a human running uv run --frozen pytest by hand and reporting the result. That does not scale, and it is exactly the kind of thing this project fixes by turning a convention into a check.

Proposed fix

Add .github/workflows/ci.yml:

  • Triggers: push to main and pull_request.
  • Steps: checkout, set up Python (the project requires >= 3.12 per uv.lock), install uv, then uv run --frozen pytest tests/ -q.
  • Use --frozen. Without it, uv run re-resolves and rewrites uv.lock. On a contributor machine behind a proxy registry that silently repoints every source URL away from pypi.org - it happened here in f328364 and had to be reverted in 06ad4c4.
  • No secrets required. The suite is hermetic: the log.sh tests spin up a local http.server rather than talking to Grafana Cloud, and the scaffolding tests write to tmp_path.
  • Consider also asserting the lockfile is unchanged after the run (git diff --exit-code uv.lock), which would have caught the proxy-rewrite before it was pushed.

Nice property while implementing: a ci.yml added inside a PR runs on that same PR for a same-repo branch, so it proves itself before merge.

Notes for whoever picks this up

  • Genesis has no auto-merge workflow, so the resulting PR needs a human to merge it. Say so in the PR rather than waiting silently.
  • Don't gate on Genesis Evolver - it needs ANTHROPIC_API_KEY and costs money per run. CI should be free and fast.
  • Worth deciding in the same change whether genesis-merge.yml-style auto-merge belongs in genesis itself. Right now genesis is the only repo in the family that can't self-advance past a PR, which is why every fix above needed a human hand on the button.

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs:evolverFramework-level improvement for genesis's own evolver to evaluate

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions