ci: consolidate docker service startup into start-services action (#16277)

# Overview

CI jobs intermittently fail because `pnpm docker:start` uses `--profile
all`, pulling every docker image regardless of which the job actually
needs. When Docker Hub has transient auth timeouts on unneeded images
(e.g. `mongodb/mongodb-community-search` in a postgres-only job), the
entire `docker compose up` fails.

**Example failed run:**
[int-postgres-custom-schema](https://github.com/payloadcms/payload/actions/runs/24411041892/job/71308895281?pr=16275#step:5:1144)
— `mongot` image pull timed out with `context deadline exceeded`,
failing a job that doesn't need MongoDB at all.

## Key Changes

- **Renamed `start-database` action to `start-services`**
- The action now starts storage services (LocalStack, Azurite, fake-gcs,
vercel-blob) unconditionally via `--profile storage`, then starts the
requested database conditionally (existing behavior unchanged).

- **Removed separate "Start LocalStack" steps from CI workflows**
- `tests-int` no longer calls `pnpm docker:start` before the action —
storage is handled by `start-services`.
- `tests-e2e` no longer has a conditional "Start LocalStack" step —
storage comes with `start-services` for all E2E jobs.

- **Added retry wrapper for docker compose pulls**
- Each `docker compose` step now separates `pull` (with 3 attempts, 10s
backoff) from `up`. Transient Docker Hub timeouts on needed images are
retried instead of failing the job immediately.

## Design Decisions

Each CI job previously had two service startup phases: a blanket `pnpm
docker:start` (`--profile all`) and then the `start-database` action
with a targeted profile. The `start-database` action already used the
correct profiles for databases, so the `--profile all` step was
redundant and pulled unnecessary images. Merging storage into the action
removes the redundancy and reduces the pull surface to only what each
job needs.

`docker compose pull` has no built-in retry. The daemon's
`max-download-attempts` only retries individual layer downloads, not
auth token requests (which is what timed out here). A shell retry loop
around `pull` is the standard workaround.

The `pnpm docker:start` script in `package.json` still uses `--profile
all` for local development convenience.

## Overall Flow

```mermaid
sequenceDiagram
    participant CI as CI Job
    participant SS as start-services action
    participant DC as docker compose

    CI->>SS: uses: start-services (database: postgres)
    SS->>DC: --profile storage pull (up to 3 attempts)
    SS->>DC: --profile storage up -d --wait
    Note over DC: LocalStack, Azurite, fake-gcs, vercel-blob
    SS->>DC: --profile postgres pull (up to 3 attempts)
    SS->>DC: --profile postgres up -d --wait
    Note over DC: PostgreSQL + replica
    SS-->>CI: POSTGRES_URL output
```

Previously, CI called `pnpm docker:start` (`--profile all`) before
`start-database`, pulling all images including mongodb/mongot even for
postgres jobs. Now only the needed profiles are pulled, with retry for
transient failures.

Author: denolfe

.github/actions/start-database/action.yml .github/actions/start-services/action.yml.github/actions/start-database/action.yml renamed to .github/actions/start-services/action.yml

@@ -1,5 +1,5 @@
-name: Start Database
-description: Starts the required database for tests
+name: Start Services
+description: Starts storage and database services for tests
 
 inputs:
   database:
@@ -20,11 +20,26 @@ outputs:
 runs:
   using: composite
   steps:
+    - name: Start storage services
+      shell: bash
+      run: |
+        for i in 1 2 3; do
+          docker compose -f test/docker-compose.yml --profile storage pull && break
+          echo "Pull attempt $i/3 for 'storage' profile failed, retrying in 10s..."
+          sleep 10
+        done
+        docker compose -f test/docker-compose.yml --profile storage up -d --wait
+
     - name: Start MongoDB
       id: mongodb
       if: contains(fromJSON('["mongodb", "cosmosdb", "documentdb", "firestore"]'), inputs.database)
       shell: bash
       run: |
+        for i in 1 2 3; do
+          docker compose -f test/docker-compose.yml --profile mongodb pull && break
+          echo "Pull attempt $i/3 for 'mongodb' profile failed, retrying in 10s..."
+          sleep 10
+        done
         docker compose -f test/docker-compose.yml --profile mongodb up -d --wait
         echo "url=mongodb://payload:payload@localhost:27018/payload?authSource=admin&directConnection=true&replicaSet=rs0" >> $GITHUB_OUTPUT
 
@@ -33,6 +48,11 @@ runs:
       if: inputs.database == 'mongodb-atlas'
       shell: bash
       run: |
+        for i in 1 2 3; do
+          docker compose -f test/docker-compose.yml --profile mongodb-atlas pull && break
+          echo "Pull attempt $i/3 for 'mongodb-atlas' profile failed, retrying in 10s..."
+          sleep 10
+        done
         docker compose -f test/docker-compose.yml --profile mongodb-atlas up -d --wait
         echo "url=mongodb://localhost:27019/payload?directConnection=true&replicaSet=mongodb-atlas-local" >> $GITHUB_OUTPUT
 
@@ -41,6 +61,11 @@ runs:
       if: startsWith(inputs.database, 'postgres')
       shell: bash
       run: |
+        for i in 1 2 3; do
+          docker compose -f test/docker-compose.yml --profile postgres pull && break
+          echo "Pull attempt $i/3 for 'postgres' profile failed, retrying in 10s..."
+          sleep 10
+        done
         docker compose -f test/docker-compose.yml --profile postgres up -d --wait
         echo "url=postgresql://payload:payload@localhost:5433/payload" >> $GITHUB_OUTPUT
 

.github/workflows/main.yml

@@ -182,16 +182,13 @@ jobs:
         with:
           restore-build: true
 
-      - name: Start LocalStack
-        run: pnpm docker:start
-
       - name: Configure Redis
         run: |
           echo "REDIS_URL=redis://127.0.0.1:6379" >> $GITHUB_ENV
 
-      - name: Start database
+      - name: Start services
         id: db
-        uses: ./.github/actions/start-database
+        uses: ./.github/actions/start-services
         with:
           database: ${{ matrix.database }}
 
@@ -284,13 +281,9 @@ jobs:
           key: e2e-prep-${{ github.sha }}
           fail-on-cache-miss: true
 
-      - name: Start LocalStack
-        run: pnpm docker:start
-        if: contains(fromJson('["plugin-cloud-storage", "plugin-import-export"]'), matrix.suite) || contains(matrix.suite, 'storage-s3__client-uploads') || contains(matrix.suite, 'storage-vercel-blob__client-uploads')
-
-      - name: Start database
+      - name: Start services
         id: db
-        uses: ./.github/actions/start-database
+        uses: ./.github/actions/start-services
         with:
           database: mongodb
 
@@ -404,10 +397,10 @@ jobs:
         with:
           restore-build: true
 
-      - name: Start database
+      - name: Start services
         id: db
         if: matrix.database
-        uses: ./.github/actions/start-database
+        uses: ./.github/actions/start-services
         with:
           database: ${{ matrix.database }}
 

.github/workflows/post-release-templates.yml

@@ -57,13 +57,13 @@ jobs:
 
       - name: Start PostgreSQL
         id: postgres
-        uses: ./.github/actions/start-database
+        uses: ./.github/actions/start-services
         with:
           database: postgres
 
       - name: Start MongoDB
         id: mongodb
-        uses: ./.github/actions/start-database
+        uses: ./.github/actions/start-services
         with:
           database: mongodb