Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions commons/swagger/openapi-entreprise.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9487,6 +9487,30 @@ paths:
- data
- links
- meta
'404':
description: Non trouvé
content:
application/json:
examples:
entite_non_trouvee_43003:
value:
errors:
- code: '43003'
title: Entité non trouvée
detail: Le ou les paramètre(s) d'entrée n'existent pas, ne sont
pas connus, ou ne comportent aucune information pour cet appel.
Veuillez vérifier que votre recherche est couverte par le
périmètre de l'API.
source:
meta:
provider: DGFIP - TVA
summary: Entité non trouvée
description: Le ou les paramètre(s) d'entrée n'existent pas, ne
sont pas connus, ou ne comportent aucune information pour cet
appel. Veuillez vérifier que votre recherche est couverte par
le périmètre de l'API.
schema:
"$ref": "#/components/schemas/Error"
'422':
description: Paramètre(s) invalide(s)
content:
Expand Down
120 changes: 120 additions & 0 deletions commons/swagger/openapi-particulier.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11657,6 +11657,26 @@ paths:
- data
- links
- meta
'404':
description: Étudiant non identifié
content:
application/json:
examples:
entite_non_trouvee_26003:
value:
errors:
- code: '26003'
title: Entité non trouvée
detail: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis.
source:
meta:
provider: CNOUS
summary: Entité non trouvée
description: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis.
schema:
"$ref": "#/components/schemas/Error"
'401':
description: Non autorisé
content:
Expand Down Expand Up @@ -11979,6 +11999,24 @@ paths:
- data
- links
- meta
'404':
description: Étudiant non identifié
content:
application/json:
examples:
entite_non_trouvee_26003:
value:
errors:
- code: '26003'
title: Entité non trouvée
detail: Aucun étudiant boursier n'a pu être trouvé.
source:
meta:
provider: CNOUS
summary: Entité non trouvée
description: Aucun étudiant boursier n'a pu être trouvé.
schema:
"$ref": "#/components/schemas/Error"
'401':
description: Non autorisé
content:
Expand Down Expand Up @@ -12367,6 +12405,28 @@ paths:
- data
- links
- meta
'404':
description: Étudiant non identifié
content:
application/json:
examples:
entite_non_trouvee_26003:
value:
errors:
- code: '26003'
title: Entité non trouvée
detail: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis. Veuillez vérifier que l'identifiant
correspond au périmètre couvert par l'API.
source:
meta:
provider: CNOUS
summary: Entité non trouvée
description: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis. Veuillez vérifier que l'identifiant
correspond au périmètre couvert par l'API.
schema:
"$ref": "#/components/schemas/Error"
'401':
description: Non autorisé
content:
Expand Down Expand Up @@ -12863,6 +12923,26 @@ paths:
- data
- links
- meta
'404':
description: Étudiant non identifié
content:
application/json:
examples:
entite_non_trouvee_26003:
value:
errors:
- code: '26003'
title: Entité non trouvée
detail: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis.
source:
meta:
provider: CNOUS
summary: Entité non trouvée
description: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis.
schema:
"$ref": "#/components/schemas/Error"
'401':
description: Non autorisé
content:
Expand Down Expand Up @@ -13192,6 +13272,24 @@ paths:
- data
- links
- meta
'404':
description: Étudiant non identifié
content:
application/json:
examples:
entite_non_trouvee_26003:
value:
errors:
- code: '26003'
title: Entité non trouvée
detail: Aucun étudiant boursier n'a pu être trouvé.
source:
meta:
provider: CNOUS
summary: Entité non trouvée
description: Aucun étudiant boursier n'a pu être trouvé.
schema:
"$ref": "#/components/schemas/Error"
'401':
description: Non autorisé
content:
Expand Down Expand Up @@ -13587,6 +13685,28 @@ paths:
- data
- links
- meta
'404':
description: Étudiant non identifié
content:
application/json:
examples:
entite_non_trouvee_26003:
value:
errors:
- code: '26003'
title: Entité non trouvée
detail: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis. Veuillez vérifier que l'identifiant
correspond au périmètre couvert par l'API.
source:
meta:
provider: CNOUS
summary: Entité non trouvée
description: Aucun étudiant boursier n'a pu être trouvé avec les
critères de recherche fournis. Veuillez vérifier que l'identifiant
correspond au périmètre couvert par l'API.
schema:
"$ref": "#/components/schemas/Error"
'401':
description: Non autorisé
content:
Expand Down
62 changes: 50 additions & 12 deletions mocks/.claude/skills/siade-sync-openapi-payloads/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,54 @@
---
name: siade-sync-openapi-payloads
description: Use when the user asks to sync, import, or check SIADE staging data payload folders against OpenAPI specs - detects missing endpoint folders, creates test payloads matching OpenAPI schemas, and validates with rspec
description: Use when the user asks to sync, import, or check SIADE staging data payload folders against OpenAPI specs, or to check 404/not-found coverage in payloads - detects missing endpoint folders and folders missing a 404 case, creates test payloads matching OpenAPI schemas, and validates with rspec
---

# Sync OpenAPI Specs with Staging Data Payloads

## Overview

Ensures every endpoint defined in the SIADE OpenAPI specs has a corresponding payload folder in `payloads/` with valid test cases. Imports latest specs, finds gaps, creates missing folders with schema-compliant YAML files, and runs the test suite.
Ensures every endpoint defined in the SIADE OpenAPI specs has a corresponding payload folder in `payloads/` with valid test cases. Finds gaps (missing folders and folders missing a 404 case), creates schema-compliant YAML files, and runs the test suite.

## When to Use

- User asks to import/sync OpenAPI files and check for missing payloads
- User asks to add staging data for new endpoints
- User asks to check coverage of payload folders against OpenAPI specs
- User asks to check 404/not-found coverage in payloads
- After new endpoints are added to the apistration/siade OpenAPI files

## Workflow

```dot
digraph sync_flow {
"Import OpenAPI files" [shape=box];
"Extract all x-operationId values" [shape=box];
"Compare against payloads/ folders" [shape=box];
"Missing folders?" [shape=diamond];
"Check 404 coverage on existing folders" [shape=box];
"Any folder missing 404?" [shape=diamond];
"Done" [shape=doublecircle];
"For each missing endpoint" [shape=box];
"Read OpenAPI schema" [shape=box];
"Find similar existing folder" [shape=box];
"Create payload files" [shape=box];
"Ensure 404 case exists (3d)" [shape=box];
"Run rspec" [shape=box];
"Failures?" [shape=diamond];
"Fix schema mismatches" [shape=box];

"Import OpenAPI files" -> "Extract all x-operationId values";
"Extract all x-operationId values" -> "Compare against payloads/ folders";
"Compare against payloads/ folders" -> "Missing folders?";
"Missing folders?" -> "Done" [label="none"];
"Missing folders?" -> "Check 404 coverage on existing folders" [label="none"];
"Missing folders?" -> "For each missing endpoint" [label="yes"];
"For each missing endpoint" -> "Read OpenAPI schema";
"Read OpenAPI schema" -> "Find similar existing folder";
"Find similar existing folder" -> "Create payload files";
"Create payload files" -> "Run rspec";
"Create payload files" -> "Ensure 404 case exists (3d)";
"Ensure 404 case exists (3d)" -> "Check 404 coverage on existing folders";
"Check 404 coverage on existing folders" -> "Any folder missing 404?";
"Any folder missing 404?" -> "Done" [label="none"];
"Any folder missing 404?" -> "Ensure 404 case exists (3d)" [label="yes"];
"Ensure 404 case exists (3d)" -> "Run rspec";
"Run rspec" -> "Failures?";
"Failures?" -> "Done" [label="no"];
"Failures?" -> "Fix schema mismatches" [label="yes"];
Expand All @@ -51,10 +58,15 @@ digraph sync_flow {

## Step 1: Locate the OpenAPI Files

The OpenAPI YAML files are now generated by siade and live at the repository
The OpenAPI YAML files are generated by siade and live at the repository
root in `commons/swagger/` (visible from `mocks/` via the `mocks/commons`
symlink). Make sure they are up to date by running
`siade/bin/generate_swagger.sh` from the `siade/` directory.
`bin/generate_swagger.sh` from the `siade/` directory.

These are **generated files** (see root `CLAUDE.md`) — never hand-edit them.
If you need to change a documented response (including adding a missing
`404`, see Step 3d), edit the rswag spec in `siade/spec/requests/` and
regenerate — don't touch the YAML directly.

## Step 2: Find Missing Payload Folders

Expand All @@ -68,6 +80,18 @@ comm -23 /tmp/openapi_ids.txt /tmp/payload_dirs.txt

Ignore malformed operation IDs (e.g. `api_entreprise_vrivileges_`).

### Also Check 404 Coverage on Folders That Already Exist

A folder can exist and still be incomplete — check separately whether every folder has at least one not-found case (`payloads/france_connect/` is not an endpoint folder and is exempt — see 3d):

```bash
find payloads -maxdepth 2 -iname '*404*' -printf '%h\n' | sed 's|^payloads/||' | sort -u > /tmp/with_404.txt
ls payloads/ | grep -v '^france_connect$' | sort -u > /tmp/all_dirs.txt
comm -23 /tmp/all_dirs.txt /tmp/with_404.txt
```

Any folder listed here needs a `404.yaml` — go to Step 3d for each one, even though the folder itself isn't "missing."

## Step 3: Create Missing Payload Folders

For each missing endpoint:
Expand All @@ -94,7 +118,7 @@ Map the missing endpoint to an existing one:

Each folder needs **at minimum**:
- One `200.yaml` (or descriptive name) with a valid success response
- One error file (e.g. `404.yaml`) if the OpenAPI spec defines that error response
- One `404.yaml` — see 3d below for how to handle this when the OpenAPI spec doesn't document 404 yet

**Critical rules:**
- Every file needs unique `params` within its folder (CLAUDE.md requirement)
Expand Down Expand Up @@ -145,9 +169,21 @@ payload: |-
}
```

### 3d. Only Create Error Files for Defined Responses
### 3d. Every Folder Needs At Least One 404 Case — No Exceptions Without Checking the Code

A `NotFoundError` is a **universal** error: every SIADE interactor can emit one regardless of whether the OpenAPI spec documents it (see `siade/spec/support/validate_response_emission_guard.rb`'s `UNIVERSAL_ERRORS`, which includes `NotFoundError` alongside the baseline network/provider errors). So a missing `404.yaml` is a gap almost every time — treat "the spec doesn't document 404" as a reason to investigate, never as a reason to skip.

1. **OpenAPI already documents `404` for this operation** (the common case — most endpoints do): read the example under `responses.'404'.content.application/json.examples` and write the payload from it, same as any other status.

2. **OpenAPI does NOT document `404`**: don't skip it. Check the endpoint's `validate_response.rb` interactor in `siade/app/interactors/<provider>/.../validate_response.rb` for a call to `resource_not_found!` (directly, or via an `http_not_found?` / empty-response check). This is almost always present. If it is:
- Add a `response '404', ...` block to the matching rswag request spec in `siade/spec/requests/api_*/v3_and_more/**/*_spec.rb`, copying the pattern from a sibling endpoint (same provider, or the version this one was cloned from — e.g. a `v4` endpoint missing 404 should copy its `v3` counterpart's block almost verbatim). Add a WebMock "not found" stub in `siade/spec/support/provider_stubs/<provider>.rb` if one doesn't already exist for this provider.
- Run `bundle exec rspec <that spec file>` from `siade/` and confirm it's green.
- Regenerate the docs: run `bin/generate_swagger.sh` from `siade/` (runs the full `spec/requests/api_*/**/*_spec.rb` suite and rewrites `commons/swagger/*.yaml` — never hand-edit those files).
- Now create the mocks payload from the newly-documented example, same as case 1.

3. **The endpoint genuinely cannot 404** (rare): skip it, but say so explicitly — don't silently omit the file. `payloads/france_connect/` is the one folder-wide exemption in this repo: it mocks FranceConnect's own token-introspection response, isn't tied to any `x-operationId`, and its own acceptance spec (`spec/acceptances/france_connect_spec.rb`) asserts every fixture is `status: 200`.

Check which HTTP status codes are defined in the OpenAPI spec for each endpoint. If the spec only defines `200`, `401`, `403`, `422`, `429` (but not `404`), do NOT create a `404.yaml`.
**Worked example** (from a real gap found in this repo): `api_entreprise_v3_dgfip_tva` had no documented 404, but `DGFIP::TVA::ValidateResponse#handle_data_response` calls `resource_not_found!` when the provider returns an empty `data` array — nobody had written the `response '404'` rswag block. Same root cause hit all 6 `cnous_etudiant_boursier` v4/v5 folders: their v3 counterpart documents 404 (and more), but the v4/v5 specs were cloned with only the `200` case, dropping every error response including 404.

## Step 4: Check open_api_helpers.rb

Expand Down Expand Up @@ -198,8 +234,10 @@ When creating many folders, use parallel agents grouped by similarity:

## Common Mistakes

- Creating error files for HTTP codes not defined in the OpenAPI spec (causes nil errors in tests)
- Treating "OpenAPI doesn't document 404" as "this endpoint has no not-found case" — check the siade interactor's `validate_response.rb` first; if it calls `resource_not_found!`, the spec is just missing that response block. Fix the rswag spec and regenerate, don't skip the mock (see 3d).
- Creating a `404.yaml` for a status the spec truly can't produce, without adding the matching rswag response first — the payload validation test does `path_spec['responses']['404']['content']...`, which raises on `nil` if the response block doesn't exist yet.
- Guessing payload structure instead of reading the actual OpenAPI schema
- Forgetting `additionalProperties: false` means NO extra fields allowed
- Not updating `open_api_helpers.rb` when new version prefixes appear
- Forgetting to fix root-owned files after Docker test runs
- Hand-editing `commons/swagger/*.yaml` instead of editing the rswag spec and regenerating
21 changes: 21 additions & 0 deletions mocks/payloads/api_entreprise_v3_dgfip_tva/404.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: '404'
description: 'Payload DGFIP Numéro de TVA intracommunautaire - not found 404'
example: true
params:
siren: '000000000'
status: 404
payload: |-
{
"errors": [
{
"code": "43003",
"title": "Entité non trouvée",
"detail": "Le ou les paramètre(s) d'entrée n'existent pas, ne sont pas connus, ou ne comportent aucune information pour cet appel. Veuillez vérifier que votre recherche est couverte par le périmètre de l'API.",
"source": null,
"meta": {
"provider": "DGFIP - TVA"
}
}
]
}
Loading