perf(controlplane): denormalize organization_id onto workflow_runs#3127
perf(controlplane): denormalize organization_id onto workflow_runs#3127migmartri wants to merge 5 commits into
Conversation
Makes org-scoped workflow run queries sargable via a new (organization_id, created_at DESC) index, fixing the timeouts on the workflow runs list reported in PFM-5839. The previous query joined to workflows via a correlated EXISTS, which let the planner pick a backward index scan on created_at and scan most of the table on orgs with sparse recent activity. A BEFORE INSERT trigger fills organization_id from the parent workflow during the rolling-deploy window so old replicas that don't set the column explicitly can keep writing. A follow-up release will drop the trigger and function once every replica runs code that sets the column directly. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: f00cda5d-4707-40cf-800e-54da06ada048
AI Session Analysis
|
| Status | Attribution | File | Lines |
|---|---|---|---|
| modified | ai | app/controlplane/pkg/data/ent/migrate/migrations/20260516210119.sql |
+87 / -1 |
| modified | ai | app/controlplane/pkg/biz/workflowrun_integration_test.go |
+83 / -0 |
| modified | ai | app/controlplane/pkg/data/workflowrun.go |
+32 / -36 |
| modified | human | app/controlplane/pkg/data/ent/migrate/schema.go |
+28 / -10 |
| modified | ai | app/controlplane/pkg/data/ent/schema/workflowrun.go |
+13 / -0 |
| modified | human | app/controlplane/pkg/data/ent/migrate/migrations/atlas.sum |
+2 / -1 |
| modified | ai | app/controlplane/pkg/data/ent/schema/organization.go |
+1 / -0 |
Policies (4)
| Status | Policy | Material | Messages |
|---|---|---|---|
| ✅ Passed | ai-config-ai-agents-allowed |
ai-coding-session-f00cda |
- |
| ✅ Passed | ai-config-no-dangerous-commands |
ai-coding-session-f00cda |
- |
| ✅ Passed | ai-config-no-secrets |
ai-coding-session-f00cda |
- |
| ✅ Passed | ai-config-mcp-servers-allowed |
ai-coding-session-f00cda |
- |
Powered by Chainloop and Chainloop Trace
There was a problem hiding this comment.
1 issue found across 20 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="app/controlplane/pkg/data/workflowrun.go">
<violation number="1" location="app/controlplane/pkg/data/workflowrun.go:375">
P2: List no longer consistently excludes runs from soft-deleted workflows. Apply `workflow.DeletedAtIsNil()` for all list queries, not only when `ProjectIDs` is provided.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Fix all with cubic | Re-trigger cubic
The previous commit moved org scoping off the workflows edge but only re-attached workflow.DeletedAtIsNil() when ProjectIDs was set, so runs from soft-deleted workflows leaked into the org-level list when no project filter was applied. Apply DeletedAtIsNil() unconditionally via the workflows edge — it's a per-row PK lookup after the org-scoped index narrows the candidate set, so it doesn't reintroduce the planner ambiguity from PFM-5839. Adds an integration test that creates a workflow + run, soft-deletes the workflow, and asserts the run is excluded from List. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: f00cda5d-4707-40cf-800e-54da06ada048
…orkflow run List The refactor in PFM-5839 changed how org scoping and project filtering compose in WorkflowRunRepo.List. The previous table-driven cases didn't assert either behavior — they only listed runs in a single org and never filtered by ProjectIDs — so the regression cubic caught had no coverage beyond the soft-delete case added in the prior commit. Adds two integration tests: - TestListIsolatedByOrg: listing in org1 returns only org1 runs and vice versa. - TestListFilterByProjectIDs: creates a workflow in a second project, asserts filtering by the original project's ID excludes it. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: f00cda5d-4707-40cf-800e-54da06ada048
Removes the PFM-5839 references in the migration header and the WorkflowRunRepo.List comment. The PFM tracker is private; the public codebase shouldn't link to it. The technical context remains in place. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: f00cda5d-4707-40cf-800e-54da06ada048
The prior commit edited the migration SQL but left atlas.sum stale, which the ent CI check caught. Rerunning atlas migrate hash. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
Fixes #3128.
Summary
organization_idtoworkflow_runsand a(organization_id, created_at DESC)index so org-scoped list/aggregate queries are sargable without joiningworkflows.EXISTS, which let the planner pick a backward index scan oncreated_atand walk most of the table for orgs with sparse recent activity. See workflow run list times out on some organizations #3128 for the full root-cause analysis.Migration
The migration is designed to run safely under a rolling deploy:
NOT VALID.BEFORE INSERTtrigger that fillsorganization_idfrom the parent workflow when the caller doesn't set it. This protects against the rolling-update window where old replicas are still inserting without the column set.DOloop with oneCOMMITper 5000-row batch (no giant transaction, no long row locks).VALIDATE CONSTRAINT+SET NOT NULL(verify-only scan in PG 12+).CREATE INDEX CONCURRENTLY.A follow-up PR will drop the trigger and function in a separate release once every replica runs code that sets the column explicitly.
Assisted-by: Claude Code