Feat/129 ci migration validation - #1034
Open
kelp-Shake wants to merge 14 commits into
Open
Conversation
Ran 'alembic upgrade head' (clean) then 'alembic check', which fails reporting a removed unique constraint 'uq_users_email' on users - the migrations build a constraint the User model no longer declares.
Migration 001 creates a named unique constraint uq_users_email on users.email, but the model only set unique=True, index=True, which produces a unique index instead. The constraint was therefore present in every migrated database but absent from the model metadata, so `alembic check` reported drift and autogenerated a remove_constraint. Declare the constraint in __table_args__ so the model matches the schema the migrations build. This changes no database state: the constraint already exists wherever the migrations have run. Refs ascherj#129
Nothing in CI confirmed that the Alembic migrations apply to an empty database or that the schema they build still matches the SQLAlchemy models, so drift could reach main unnoticed. It already had: the uq_users_email constraint drifted from the User model. Add scripts/validate_migrations.sh, which runs `alembic upgrade head` followed by `alembic check` under `set -euo pipefail`, and a validate-migrations job that runs it against a fresh Postgres service container. The job fails the PR when a migration is broken or has drifted from the models. The script requires DATABASE_URL to use the asyncpg driver, since alembic/env.py builds an async engine and a sync URL fails with an unhelpful error. Refs ascherj#129
Cover the parts of migration health that don't need a database: unique revision ids, exactly one head, exactly one base, every down_revision resolving to a real migration, and every migration defining both upgrade() and downgrade(). Two further tests guard the uq_users_email fix from regressing. Revision metadata is read with ast rather than by importing the migration modules, because the repo's own alembic/ package directory shadows the installed alembic once the repo root is on sys.path, so `from alembic import op` fails at pytest collection time. Refs ascherj#129
Record the Week 9 check-ins, the decision to fix the uq_users_email drift on the model rather than with a drop-constraint migration, and the local verification of the script. Check-in 2 is marked draft: the CI job has not run on GitHub Actions yet. Refs ascherj#129
Week 8 feedback noted that PLAN.md left the uq_users_email fix as an open question rather than a decision. Rewrite that section as a decision record: the two options side by side, the choice and why, what would make me revisit it, and the trade-off being accepted. Also update JOURNAL.md with the results of the dry-run CI: the validate-migrations job passes on GitHub Actions, and each of the five failing jobs is checked against the files this branch touches. Refs ascherj#129
Verified the validate-migrations job fails, not just passes: pushed a model column with no migration behind it, confirmed the job went red with an add_column drift error and exit 255, then removed that commit from the branch. Refs ascherj#129
Both targets run now; record their output on this branch alongside the existing before/after comparison against main. Refs ascherj#129
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
There was no automated check that the Alembic migrations still apply cleanly or that the schema they build matches the SQLAlchemy models, so drift could reach
mainunnoticed — and it already had. This adds avalidate-migrationsCI job that applies every migration to a fresh, empty Postgres and then compares the result against the models, failing the PR when a migration is broken or has drifted. It also fixes the one piece of drift that was already present, so the new check passes onmain.Issue
Closes #129
Changes
core/models/user.py— declareUniqueConstraint("email", name="uq_users_email")in__table_args__. Migration 001 has always created this constraint, but the model only setunique=True, index=True, which produces a unique index instead. The constraint existed in every migrated database while being absent from the model metadata, soalembic checkreported drift. This changes no database state — the constraint already exists wherever the migrations have run. I chose this over a drop-constraint migration deliberately: it keeps a CI-scoped PR from becoming a schema change against live data.scripts/validate_migrations.sh(new) — runsalembic upgrade headthenalembic checkunderset -euo pipefail. It requiresDATABASE_URLto use the asyncpg driver, becausealembic/env.pybuilds an async engine and a sync URL otherwise fails with an unhelpful traceback..github/workflows/ci.yml— newvalidate-migrationsjob with a Postgres 16 service container (health check and setup copied from the existingtest-integrationjob), pointed at its own emptypathreview_migrationsdatabase.tests/unit/test_migrations.py(new, 8 tests) — the parts of migration health that don't need a database.Testing
make test-unit) — 53 failures, all pre-existing; see belowmake test-integration) — not runmake lint) — 182 errors, all pre-existing; clean on both files I touchedmake typecheck) — 99 errors, all pre-existing; none in files I touchedVerified locally against a
docker composePostgres, using a scratch database:I also confirmed the check goes red, not just green:
remove_constraint uq_users_email— the original drift, so the fix is what makes it pass.Profilemodel and no migration), it fails withadd_column ...and exits non-zero, which is what fails the job. Reverted afterwards.DATABASE_URLand a syncpostgresql://URL both exit 1 with an explanatory message.Pre-existing failures
This branch does not make
make checkormake test-unitgreen, because they aren't green onmain. I recorded a baseline before my changes and compared after:main(unmodified)ruff check .black --check .pytest tests/unit -m unitThe 53 failures are the same test ids before and after, and the 8 additional passes are the new tests.
ruffandblackare clean on both files I touched. These changes introduce no new failures.These numbers line up with CI: the
lintjob reports the same 182 ruff errors, andtest-unitreports the same 53 failed / 383 passed. None of thelintortypecheckoutput namescore/models/user.pyortests/unit/test_migrations.py. The pinned pre-commit hooks (ruff, black, mypy) also pass on every commit in this branch.CI status
I ran the workflow before opening this PR, via a dry-run PR into my own fork, and
validate-migrationspasses — service container healthy,pip install -e ".[dev]"fine on Python 3.11.15, the script executable in the checkout, both migrations applied andalembic checkclean, 1m 1s total.I also confirmed it fails on CI, not just locally. I pushed a commit adding a column to the
Profilemodel with no migration behind it, and the job went red:Two things that run confirms: the migrations still applied cleanly first, so the failure came from
alembic checkrather than a broken migration — the job distinguishes the two. Andlint,typecheckandtest-unitreturned identical numbers with that column present, as did the pre-commit hooks. A well-typed column with no migration behind it is invisible to ruff, black and mypy; only this check sees it. That commit is not part of this branch.The other five jobs fail, all of them pre-existing:
validate-migrationslintmaintypechecktest-unitmain, plus my 8 new passestest-integrationcollected 0 items— no tests present to runfrontend@testing-library/user-event, plus aReviewSectionassertionHappy to open separate issues for any of these if that's useful — I left them alone to keep this PR scoped to #129.
Screenshots / Demo
N/A
Notes for Reviewers
uq_users_emailfix is the judgement call here. Updating the model matches what deployed databases already contain; the alternative was a migration dropping the constraint. I went with the model change to avoid a schema change on live data in a CI-focused PR, but I'd happily switch if you'd rather the constraint go away — the column'sunique=Truealready provides a unique index.alembic checkdoesn't catch everything. It reliably catches added/removed tables, columns, indexes and constraints, but can miss some server defaults and check constraints. It's a large improvement over no check at all, not a guarantee of perfect parity.astinstead of importing the modules. The repo ships its ownalembic/package directory, which shadows the installedalembicdistribution once the repo root is onsys.path— as it is under pytest — sofrom alembic import opraises at collection time. Importing the migrations would have been the natural approach; this is the workaround. Happy to revisit if you'd prefer that shadowing addressed separately.