-
Notifications
You must be signed in to change notification settings - Fork 8
test(bdd): Gherkin/BDD e2e pilot on the existing Vitest runner #1038
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
branarakic
wants to merge
2
commits into
main
Choose a base branch
from
test/bdd-gherkin-pilot
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| # BDD / Gherkin pilot | ||
|
|
||
| A small, self-contained pilot that runs **Gherkin `.feature` specs on the | ||
| project's existing Vitest runner** — no second test framework, no extra CI lane. | ||
|
|
||
| The goal is to evaluate whether describing system behaviour in human-readable | ||
| `Given / When / Then` is worth adopting more widely, **before** committing to it. | ||
|
|
||
| ``` | ||
| bdd/ | ||
| ├── features/ # the human-readable specs (.feature) | ||
| │ ├── entity-predicate.feature # pure smoke spec — always runnable | ||
| │ └── ka-lifecycle.feature # @devnet — the real KA publish lifecycle | ||
| ├── steps/ # step definitions that bind specs to code | ||
| │ ├── entity-predicate.steps.ts | ||
| │ └── ka-lifecycle.steps.ts | ||
| ├── support/ | ||
| │ └── world.ts # shared devnet connection + HTTP helpers | ||
| ├── vitest.config.ts | ||
| └── package.json | ||
| ``` | ||
|
|
||
| ## Why this approach | ||
|
|
||
| The binding is [`@amiceli/vitest-cucumber`](https://www.npmjs.com/package/@amiceli/vitest-cucumber). | ||
| It was chosen deliberately over standalone Cucumber: | ||
|
|
||
| - **One runner, one CI lane.** `describeFeature/Scenario/Given/When/Then` | ||
| compile down to Vitest's own `describe/test`. Its peer dependency is | ||
| `vitest ^4.0.4`, which the repo's `^4.0.18` satisfies — no runner duplication, | ||
| no Jest typings, nothing new for CI to learn. | ||
| - **Real `.feature` files.** `loadFeature('./x.feature')` parses actual Gherkin | ||
| (Scenario Outline, Examples, Background, tags, hooks) — the specs are not | ||
| inline strings, so non-engineers can read and edit them. | ||
| - **Maintained and Vitest-4 native** (unlike `jest-cucumber`, which is Jest-first, | ||
| or `@deepracticex/vitest-cucumber`, which doesn't declare Vitest 4 support). | ||
|
|
||
| It is **strict by design**: every step in a `.feature` must have a matching step | ||
| definition or the suite fails. That keeps specs and code honest. | ||
|
|
||
| ## The two specs | ||
|
|
||
| ### 1. `entity-predicate.feature` — the smoke spec (always runnable) | ||
|
|
||
| Pins the OT-RFC-43/44 entity-predicate rename invariant | ||
| (`dkg:rootEntity → dkg:entity`, `dkg:assertionRootEntity → dkg:assertionEntity`): | ||
| during the dual-write migration window, a mixed-fleet node **must recognise both | ||
| the new and the legacy predicate** or it silently drops entity members. | ||
|
|
||
| It exercises the real helpers in | ||
| [`packages/core/src/entity-predicate.ts`](../packages/core/src/entity-predicate.ts) | ||
| via a data-driven `Scenario Outline`. They are imported through the public | ||
| `@origintrail-official/dkg-core` export (declared as a workspace dependency), so | ||
| Turbo treats `bdd` as depending on `dkg-core` and **invalidates the smoke-spec | ||
| cache when that code changes** — the regression check stays honest. No devnet, | ||
| no network — it runs in milliseconds and proves the Gherkin→Vitest binding works | ||
| end to end. | ||
|
|
||
| This is the package's default `test` script, so `turbo test` (the standard CI | ||
| command) runs it automatically — no extra CI lane: | ||
|
|
||
| ```bash | ||
| pnpm --filter @origintrail-official/dkg-bdd test | ||
| ``` | ||
|
|
||
| ### 2. `ka-lifecycle.feature` — the real e2e (tagged `@devnet`) | ||
|
|
||
| The canonical V10 flow expressed as behaviour: a draft assertion goes | ||
| `create → write → finalize → promote → publish` and ends as a confirmed | ||
| on-chain Knowledge Asset. The step definitions call the same daemon routes as | ||
| `devnet/v10-core-flows`' `fullPublish()`, sharing the produced | ||
| assertion/`kaId` through the test World. | ||
|
|
||
| It is **tag-gated**: if no devnet is reachable it is skipped cleanly (rather than | ||
| failing), mirroring how the existing devnet suites guard. Availability is decided | ||
| by a **live `/api/status` probe**, not just on-disk provisioning — so a | ||
| stopped-but-provisioned cluster also skips instead of erroring. Like the other | ||
| `devnet/*` suites it is intentionally kept out of the default `turbo test` lane | ||
| and run explicitly: | ||
|
|
||
| ```bash | ||
| ./scripts/devnet.sh clean && ./scripts/devnet.sh start 6 | ||
| node devnet/_bootstrap/bootstrap.cjs | ||
| pnpm --filter @origintrail-official/dkg-bdd test:devnet | ||
| ``` | ||
|
|
||
| To run both specs at once (the devnet one self-skips without a cluster): | ||
| `pnpm --filter @origintrail-official/dkg-bdd test:all`. | ||
|
|
||
| ## How to add a new spec | ||
|
|
||
| 1. Write `features/my-thing.feature` in plain Gherkin. | ||
| 2. Create `steps/my-thing.steps.ts`: `loadFeature(...)` + `describeFeature(...)`, | ||
| implementing one step definition per line in the feature. | ||
| 3. Reuse `support/world.ts` for any devnet/HTTP plumbing. | ||
| 4. Tag devnet-dependent scenarios `@devnet` so they self-skip without a cluster. | ||
|
|
||
| ## Honest tradeoffs | ||
|
|
||
| - **Where it pays off:** multi-step, spec-driven flows with stakeholders who read | ||
| them (KA lifecycle, conviction staking tiers, provenance modes, the KA-routes | ||
| parity/must-not-regress invariants). The `.feature` doubles as living spec. | ||
| - **Where it does not:** simple unit assertions, and anything where the cost is | ||
| the *infrastructure* (booting a 6-node devnet) rather than the assertion syntax | ||
| — Gherkin does nothing for that hard part. Don't rewrite the 9 existing devnet | ||
| scenarios; add Gherkin where the readable-spec value is real. | ||
| - The strict matching means every feature line needs an implementation — good for | ||
| rigor, but it is real maintenance. | ||
|
|
||
| ## Pilot status (branch `test/bdd-gherkin-pilot`) | ||
|
|
||
| - **Smoke spec** — verified green locally: `9` Example rows / `27` step-tests | ||
| pass in ~8ms. It is the package's `test` script, so it runs under the standard | ||
| `turbo test` CI lane. | ||
| - **`@devnet` spec** — verified to skip cleanly (exit 0) when no cluster is | ||
| present. Every HTTP route, request payload and response field it uses has been | ||
| cross-checked against the live daemon handlers (`assertion.ts`, `memory.ts`, | ||
| `status.ts`) and matches `devnet/v10-core-flows`' `fullPublish()` — but it has | ||
| not been executed against a live cluster as part of this pilot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| Feature: Knowledge Asset entity-member predicate recognition | ||
| # Behaviour spec for the OT-RFC-43 §10.1 / OT-RFC-44 §4 predicate rename: | ||
| # dkg:rootEntity -> dkg:entity | ||
| # dkg:assertionRootEntity -> dkg:assertionEntity | ||
| # During the dual-write migration window a mixed-fleet node MUST recognise | ||
| # BOTH the new and the legacy predicate, or entity members get silently | ||
| # dropped. This spec pins that behaviour in human-readable terms. | ||
| # | ||
| # This is the "smoke" spec: pure, deterministic, no devnet required. | ||
|
|
||
| Background: | ||
| Given the entity-predicate migration helpers | ||
|
|
||
| Scenario Outline: A KA entity-member predicate is recognised across the rename | ||
| When I check whether "<iri>" is a KA entity predicate | ||
| Then the recognition result is "<expected>" | ||
|
|
||
| Examples: | ||
| | iri | expected | | ||
| | http://dkg.io/ontology/entity | true | | ||
| | http://dkg.io/ontology/rootEntity | true | | ||
| | http://dkg.io/ontology/assertionEntity | false | | ||
| | http://dkg.io/ontology/unrelated | false | | ||
| | http://schema.org/name | false | | ||
|
|
||
| Scenario Outline: An assertion-seal entity predicate is recognised across the rename | ||
| When I check whether "<iri>" is an assertion-seal entity predicate | ||
| Then the recognition result is "<expected>" | ||
|
|
||
| Examples: | ||
| | iri | expected | | ||
| | http://dkg.io/ontology/assertionEntity | true | | ||
| | http://dkg.io/ontology/assertionRootEntity | true | | ||
| | http://dkg.io/ontology/entity | false | | ||
| | http://dkg.io/ontology/rootEntity | false | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| @devnet | ||
| Feature: Knowledge Asset publish lifecycle (Working Memory -> Shared WM -> Verified) | ||
| # The canonical V10 flow a DKG agent performs: take a draft assertion through | ||
| # create -> write -> finalize -> promote -> publish, ending as a verifiable | ||
| # on-chain Knowledge Asset. Mirrors devnet/v10-core-flows fullPublish(), but | ||
| # expressed as behaviour anyone can read. | ||
| # | ||
| # Tagged @devnet: skipped automatically unless a local devnet is running | ||
| # ./scripts/devnet.sh clean && ./scripts/devnet.sh start 6 | ||
| # node devnet/_bootstrap/bootstrap.cjs | ||
|
|
||
| Background: | ||
| Given a reachable devnet node with an authenticated agent | ||
| And the "devnet-test" context graph | ||
|
|
||
| # Tag is repeated on the scenario (not only the feature): @amiceli/vitest-cucumber | ||
| # filters per-scenario, so the gate that skips this without a devnet lives here. | ||
| @devnet | ||
| Scenario: A signed assertion becomes a confirmed on-chain Knowledge Asset | ||
| Given a fresh draft assertion in the context graph | ||
| When I write 2 entity quads to the assertion | ||
| And I finalize the assertion | ||
| Then the assertion has a merkle root and an EIP-712 digest | ||
|
|
||
| When I promote the assertion to shared working memory | ||
| Then at least 1 share is created | ||
|
|
||
| When I publish the assertion to the chain | ||
| Then a knowledge asset id is returned | ||
| And the knowledge asset status is one of "confirmed,tentative,finalized" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| { | ||
| "name": "@origintrail-official/dkg-bdd", | ||
| "version": "0.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "description": "Gherkin/BDD test pilot — human-readable .feature specs running on the existing Vitest runner.", | ||
| "scripts": { | ||
| "test": "vitest run --config vitest.config.ts steps/entity-predicate.steps.ts", | ||
| "test:devnet": "vitest run --config vitest.config.ts steps/ka-lifecycle.steps.ts", | ||
| "test:all": "vitest run --config vitest.config.ts" | ||
| }, | ||
| "dependencies": { | ||
| "@origintrail-official/dkg-core": "workspace:*" | ||
| }, | ||
| "devDependencies": { | ||
| "@amiceli/vitest-cucumber": "^6.5.0", | ||
| "vitest": "^4.0.18" | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| /** | ||
| * Smoke spec — proves the Gherkin -> Vitest binding runs end to end against | ||
| * real product code, with no devnet and no network. | ||
| * | ||
| * Binds bdd/features/entity-predicate.feature to the pure helpers in | ||
| * packages/core/src/entity-predicate.ts (the OT-RFC-43/44 dual-read invariant). | ||
| */ | ||
| import { describeFeature, loadFeature } from '@amiceli/vitest-cucumber'; | ||
| import { expect } from 'vitest'; | ||
| import { dirname, resolve } from 'node:path'; | ||
| import { fileURLToPath } from 'node:url'; | ||
| import { isAssertionEntityPredicate, isEntityPredicate } from '@origintrail-official/dkg-core'; | ||
|
|
||
| const here = dirname(fileURLToPath(import.meta.url)); | ||
| const feature = await loadFeature(resolve(here, '../features/entity-predicate.feature')); | ||
|
|
||
| describeFeature(feature, ({ Background, ScenarioOutline }) => { | ||
| Background(({ Given }) => { | ||
| Given('the entity-predicate migration helpers', () => { | ||
| expect(typeof isEntityPredicate).toBe('function'); | ||
| expect(typeof isAssertionEntityPredicate).toBe('function'); | ||
| }); | ||
| }); | ||
|
|
||
| ScenarioOutline( | ||
| 'A KA entity-member predicate is recognised across the rename', | ||
| ({ When, Then }, variables) => { | ||
| let result = false; | ||
| When('I check whether "<iri>" is a KA entity predicate', () => { | ||
| result = isEntityPredicate(String(variables.iri)); | ||
| }); | ||
| Then('the recognition result is "<expected>"', () => { | ||
| expect(String(result)).toBe(String(variables.expected)); | ||
| }); | ||
| }, | ||
| ); | ||
|
|
||
| ScenarioOutline( | ||
| 'An assertion-seal entity predicate is recognised across the rename', | ||
| ({ When, Then }, variables) => { | ||
| let result = false; | ||
| When('I check whether "<iri>" is an assertion-seal entity predicate', () => { | ||
| result = isAssertionEntityPredicate(String(variables.iri)); | ||
| }); | ||
| Then('the recognition result is "<expected>"', () => { | ||
| expect(String(result)).toBe(String(variables.expected)); | ||
| }); | ||
| }, | ||
| ); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Bug: This PR adds a new external dependency but does not update
pnpm-lock.yaml. All of the repo's CI jobs install withpnpm install --frozen-lockfile, so they will fail as soon as this package lands. Regenerate and commit the lockfile alongside this manifest change.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like a false positive:
pnpm-lock.yamlwas committed in the initial commit (3@amiceli/vitest-cucumberentries) andpnpm install --frozen-lockfilepasses (exit 0) on this branch.9448c877eadds the new@origintrail-official/dkg-coreworkspace dep with the lockfile updated alongside; frozen install still passes.