Skip to content

feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2)#2196

Draft
cjlg-soloio wants to merge 16 commits into
kagent-dev:mainfrom
cjlg-soloio:feat(ui)--Playwright-tests-foundation-stage-2
Draft

feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2)#2196
cjlg-soloio wants to merge 16 commits into
kagent-dev:mainfrom
cjlg-soloio:feat(ui)--Playwright-tests-foundation-stage-2

Conversation

@cjlg-soloio

@cjlg-soloio cjlg-soloio commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown

Description

Stage 2 of the Playwright UI E2E suite — the Stage 1 driver foundation plus the full set of multi-step feature flows built on top of it. Rebased directly on Stage 1 (#2180), so this branch stacks cleanly: once #2180 merges, this PR narrows to just the 9 commits below.

Foundation (helpers, stage 1):

  • Scenario-override engine in the stub backend — POST /__mock/scenario sets a per-endpoint { status, body } override; POST /__mock/reset clears them. Default stays the happy path.
  • Typed control seam (mocks/control.ts) surfaced as a mock fixture: mock.noAgents(), mock.agentsError(), etc. The fixture resets the stub before every test, so scenarios never leak between tests.
  • Driver helpers — page.ts (loadPage, expectNoErrors, expectToast), nav.ts (dropdown-aware gotoView / gotoCreate), select.ts (combobox driver), and a2a.ts (chat/stream fixtures).

Feature flows (stage 2): one spec per core UI journey, each asserting both UI state and the payload POSTed/PUT to the backend, plus empty/error/validation edge states.

Spec Flow
tests/agents/create-agent.spec.ts Declarative create-agent wizard
tests/chat/chat-session.spec.ts Chat send / streamed reply / tool call / feedback / session load
tests/models/models.spec.ts Model config create, edit-save, delete, gateway-key path
tests/mcp/mcp-server.spec.ts Remote + stdio MCP server create, delete, filter
tests/agents/create-harness.spec.ts Harness agent create
tests/agents/agent-lifecycle.spec.ts Agent delete flow
tests/prompts/prompt-libraries.spec.ts Prompt library create / edit / delete
tests/onboarding/onboarding.spec.ts Onboarding wizard completion + skip

Product code changes are limited to a handful of additive data-testid hooks (chat input/send, tool-call card, combobox/dialog anchors) where role/text selectors were ambiguous — no behavior changed.

Testing

cd ui
npm install        # picks up @mcp-ui/client from the rebased base
npm run test:pw

Playwright boots the stub backend and dev server automatically and drives each flow through happy, empty, error, and validation states. Runs serially (workers: 1) against the shared stub. Full suite: 43 passed locally.

Change type

test — extends the Playwright E2E suite. Product changes limited to additive data-testid hooks; no runtime behavior changed.

Changelog

NONE

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 9, 2026
cjlg-soloio and others added 16 commits July 13, 2026 15:12
@cjlg-soloio
config for playwright
fa4de65 
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio
Run with 1 worker
631d1c3 
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: cjlg-soloio <carlos.logrono@solo.io>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
fix: resolve Copilot review comments on Playwright config
09d28ed 
- Force single Playwright worker unconditionally (shared stub + Next server)
- Default req.method/req.url before parsing in the stub server
- Fail fast on stub reset failure in CI; stay non-fatal locally
- Drop unused pull-requests:write permission from the workflow

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
docs: explain defensive req.method/url defaults in stub server
083bcbf 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): disable fullyParallel until Stage 1 isolation
151d0c1 
Pairs fullyParallel with workers: 1 so both stay off until per-test data
isolation lands, preventing tests from silently racing against the shared
stub backend if the worker count is raised.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
fix(ui-test): address review comments on Playwright config and mocks
a0a9d05 
- Pin STUB_PORT on the stub webServer so a shell-set STUB_PORT can't misbind it
- Stop reusing the dev server so BACKEND_INTERNAL_URL always applies (no silent stub bypass)
- Type ToolServerListEntry as the real RemoteMCPServerResponse for compile-time drift detection
- Drop no-op **/*.mjs from the playwright tsconfig include

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): address review — remove Cypress, use .nvmrc, enable Playwri…
9b89053 
…ght video

Resolve Charlesthebird's review on PR kagent-dev#2180:
- Playwright workflow now reads node-version-file: ui/.nvmrc instead of a
  hardcoded major version.
- Remove Cypress entirely: delete cypress.config.ts and cypress/e2e/smoke.cy.ts,
  drop the cypress and now-unused start-server-and-test devDeps plus the
  test:e2e/test:e2e:cypress scripts, and remove the dead CYPRESS_INSTALL_BINARY
  env from the ui-playwright, ci, and ui-chromatic workflows and the Dockerfile.
- Enable Playwright video recording (video: "on"); artifacts already upload via
  the existing test-results step.
- De-Cypress the Playwright README (Jest, Vitest, Storybook untouched).

data.ts is intentionally kept — it is consumed by the stacked stage-2 PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright helpers + per-test scenario overrides (stage 1)
d5b2987 
Builds on the Stage 0 foundation to unlock non-happy-path testing and give
specs a small, proven driver surface:

- Scenario-override engine in the stub backend: POST /__mock/scenario sets a
  per-endpoint { status, body } override; /__mock/reset clears them. Default
  stays the happy path.
- Typed control seam (mocks/control.ts) surfaced as a `mock` fixture
  (mock.noAgents(), mock.agentsError(), ...); the fixture resets the stub
  before every test so scenarios never leak between tests.
- page.ts (loadPage, expectNoErrors, expectToast) and nav.ts (dropdown-aware
  gotoView/gotoCreate) driver helpers.
- home spec (happy / empty / error) proving the engine, and a nav spec proving
  dropdown routing.

Form/select/dialog helpers are deferred to Stage 2 (built demand-driven with the
create-agent flow).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright create-agent declarative flow (stage 2.1)
c18521c 
First Stage 2 feature spec — the declarative create-agent flow:
- happy path: fill name/description, pick the model, submit -> asserts the
  redirect to /agents and the captured POST /api/agents payload
- validation: empty required fields block submit (no request sent)
- failure: forced create error surfaces an error toast, no redirect

Groundwork introduced here (demand-driven, reused by later sub-stages):
- stub captures mutation (POST/PUT/DELETE) bodies and exposes them via
  GET /__mock/requests; supports per-mutation error overrides; reset clears both
- control.ts: setMutation, agentsCreateError, capturedRequests, lastRequest
- helpers/select.ts: Radix <Select> driver

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright chat/session flow (stage 2.2)
0bc9b9d 
Second Stage 2 feature (commit 2/8) — the chat streaming flow:
- happy path: send a message, mock the A2A SSE reply -> asserts the user
  message, the streamed agent text, and a rendered tool-call block
- empty state: a new chat shows "Start a conversation"
- failure: an aborted stream surfaces an error toast, no agent reply

Groundwork introduced here:
- helpers/a2a.ts: mockAgentReply / mockAgentStreamError — page.route mock of the
  browser-side A2A SSE call (text/event-stream JSON-RPC frames)
- stub: dynamic GET routes (agent detail, sessions-for-agent) + POST /api/sessions
  returning a session with an id (needed for the streamed contextId)
- data-testid hooks: chat-input, chat-send (ChatInterface), tool-call (ToolDisplay)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright models create + edit-save (stage 2.3)
69388a8 
Third Stage 2 feature (commit 3/8) — model config lifecycle:
- create: pick provider (OpenAI) + model + API key -> asserts redirect to
  /models, success toast, and the captured POST /api/modelconfigs payload
- edit-save: open the seeded config via the edit-model hook -> Save Changes ->
  asserts redirect + captured PUT /api/modelconfigs/<ref>

Groundwork:
- stub: /api/models keyed "OpenAI" (was lowercase); stock/configured provider
  endpoints (/api/modelproviderconfigs/*); GET /api/modelconfigs/<ns>/<name> detail
- data-testid hooks: model-provider-select, model-select, model-api-key-input

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright MCP server create (stage 2.4)
4a887e3 
Fourth Stage 2 feature (commit 4/8) — create a RemoteMCPServer tool server:
- happy: fill server name + URL, submit -> asserts redirect to /mcp and the
  captured POST /api/toolservers payload (type, url, name)
- failure: forced 500 shows the inline "Couldn't create server" alert, no redirect

Groundwork: stub GET /api/toolservertypes (blocking gate on /mcp/new).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright harness agent create (stage 2.5)
c06452a 
Fifth Stage 2 feature (commit 5/8) — the harness create variant at
/agents/new-harness:
- happy: name + model, submit -> asserts redirect to /agents and the captured
  POST /api/agentharnesses (kind AgentHarness, backend openclaw)
- validation: empty required fields block submit (no request)

Groundwork: stub POST /api/agentharnesses returns data.agent (required by
createAgentHarnessFromForm).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright agent delete flow (stage 2.6)
60b34ba 
Sixth Stage 2 feature (commit 6/8) — delete an agent from the list:
- happy: Agent options menu -> Delete -> confirm -> asserts the dialog closes
  and DELETE /api/agents/<ns>/<name> was captured
- cancel: dismissing the dialog issues no DELETE

No stub change (the generic mutation capture covers DELETE).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright prompt libraries (stage 2.7)
8804263 
Seventh Stage 2 feature (commit 7/8) — prompt libraries:
- empty state: /prompts (namespace kagent) with no libraries
- create: name + a fragment key/content, submit -> asserts redirect to the
  detail page, success toast, and the captured POST /api/prompttemplates payload

Groundwork: stub GET /api/prompttemplates (list) + GET /api/prompttemplates/<ns>/<name> (detail).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio @claude
test(ui): Playwright onboarding completion (stage 2.8)
3154d21 
Eighth Stage 2 feature (commit 8/8) — complete the first-run onboarding wizard:
walks Welcome -> Model (select existing) -> Agent Setup -> Tools -> Review ->
Finish, asserting the POST /api/agents at finalize, that the wizard dismisses to
the Agents list, and that localStorage['kagent-onboarding'] flips to "true".

No new stub routes (all endpoints already seeded).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>
@cjlg-soloio cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-2 branch from 1da788e to 3154d21 Compare July 13, 2026 22:13
@cjlg-soloio cjlg-soloio changed the title feat(ui-test): Playwright ui tests helpers + per-test scenarios stage 1 feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2) Jul 13, 2026
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@peterj peterj Awaiting requested review from peterj peterj will be requested when the pull request is marked ready for review peterj is a code owner
Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cjlg-soloio