Skip to content

docs: update production-to-rc data cloning for eks and pipeline ordering - #3124

Draft
umar8hassan wants to merge 1 commit into
masterfrom
umar/clone-prod-to-rc
Draft

docs: update production-to-rc data cloning for eks and pipeline ordering#3124
umar8hassan wants to merge 1 commit into
masterfrom
umar/clone-prod-to-rc

Conversation

@umar8hassan

Copy link
Copy Markdown
Contributor

What are the relevant tickets?

N/A

Description (What does it do?)

Updates docs/production-to-rc-data-cloning.md, which still described a Heroku-era Concourse pipeline for database replication. Production and RC databases now live in EKS behind Vault dynamic secrets, not reachable directly over the network, so the old pipeline could never connect regardless of credentials.

  • Step 1 (Database Replication): replaced the Concourse pipeline / fly set-pipeline / heroku config:get approach with ephemeral kubectl pods in each cluster, chunked transfer to work around the EKS auth token's fixed ~15 minute session limit, and a verify-before-combine step so a dropped chunk can't silently corrupt the restore.
  • Step 3: moved backpopulate_pipelines ahead of mass_publish (was in Step 4). Cloned sites inherit a publish_date from production, which publish_website treats as "pipeline already exists," so publishing before creating RC's own per-site pipelines fails every previously-published site with Could not find live build <id>. Also fixed the mass_publish example command, which was missing its required version argument.

How can this be tested?

  1. Follow Step 1 against the real applications-production/applications-qa clusters and confirm the dump/restore completes.
  2. Run Step 3 in the old order (mass_publish before backpopulate_pipelines) and confirm it reproduces Could not find live build in the celery worker logs, then confirm the new order does not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant