Product/dataset creation error when using `gwa` apply

[rustyjux]

API Services Portal Bug Issue

User Story

As a API provider,

I want to seamlessly publish Gateway resources

so that I don't get confused or frustrated!.

When publishing a new GatewayService and a Product (and Dataset) using that service at the same time (e.g. using the quick-start or client-credentials-shared-idp template), the Product will fail. Based on analysis (see Error below), it looks to be because the Gateway Service doesn't exist yet in the db when it is being looked up for Product creation. Sometimes Dataset fails to publish as well.

Note on priority: This isn't a high priority issue because the error rarely occurs in deployed environments and there is an easy workaround (re-run gwa apply, always works). However it does impact new user getting started flow.

Test Case

ENV

• e2e
• DEV
• TEST?
• PROD?

Pretty sure I've seen this happen in dev and prod, but it's not common. Currently happens every run in e2e tests.
In e2e, it was causing 16-gwa-cli/02-cli-generate-config-quick-start.ts - Check gwa command to apply generated config to fail, but I added retry logic to run apply again.

TESTCASE

gwa gateway create -g
gwa generate-config \
  --template quick-start \
  --service myservice \
  --upstream https://httpbin.org \
  --out gw-config.yaml
gwa apply -i gw-config.yaml

EXPECTED

• Success, all resources published

ACTUAL

↑ Publishing Gateway Services
√ Gateway Services published
creating service 12345afsd1-dev
creating route 12345afsd1-dev
Summary:
  Created: 2
  Updated: 0
  Deleted: 0

x [DraftDataset] 12345afsd1-dataset failed
x [Product] 12345afsd1 API failed

1/3 Published, 0 Skipped

x Errors encountered
Resource [DraftDataset] 12345afsd1-dataset: something broke
Resource [Product] 12345afsd1 API: something broke

ERROR

Portal log: product_error.log

Analysis: Missing GatewayService (Lines 168-190)

What happened:

1. Product creation starts (line 157): Creating product "afsd1 API" with an environment that references service "afsd1-dev"
2. Dataset connection succeeds (line 165): Dataset "afsd1-dataset" is found and connected
3. Environment creation fails (lines 168-184): When creating the environment, it tries to connect to GatewayService "afsd1-dev" but it doesn't exist yet

Error details:

• Line 173-182: Lookup for allGatewayServices with namespace="gw-70dc0" and name="afsd1-dev" returns empty: {"data":{"allGatewayServices":[]}}
• Line 183: Lookup failed for allGatewayServices name!
• Line 184: Error: Failed to find afsd1-dev in allGatewayServices
• Line 190: Batch sync fails for /ds/api/v3/gateways/gw-70dc0/products

Timing issue:

• Lines 191-214: After the product creation fails, the GatewayService "afsd1-dev" is created (line 214: createGatewayService with id="4")
• The service hasn't been synced to the database yet when the product tries to reference it

Root cause:

• The Product/Environment references a GatewayService that hasn't been synced yet. The service sync happens after the product sync attempt, causing a dependency ordering issue.

Solution:

• Ensure GatewayServices are synced before creating Products that reference them, or make the service connection optional/retryable during product creation.