Skip to content

testing.md

Latest commit

 

History

History
967 lines (826 loc) · 57.8 KB

testing.md

File metadata and controls

967 lines (826 loc) · 57.8 KB
Raw

Testing

Diátaxis type: Reference Audience: 🔧 Developers, contributors Prerequisites: Go testing basics, understanding of httptest

Comprehensive test documentation for gitlab-mcp-server.

Maintenance Rule: Whenever tests are added, modified, or removed, run go run ./cmd/gen_testing_docs/ to refresh the generated counts and coverage values.

Overview

This section is generated by go run ./cmd/gen_testing_docs/. It runs unit coverage for ./internal/... and ./cmd/...; E2E tests are counted statically from test/e2e/suite because they require a real GitLab fixture to execute.

Metric Value
Total test functions 12,012
Unit test functions 11,637
E2E test functions 375
cmd test functions 955
Test files (internal/) 459
Test files (cmd/) 68
Test files (test/e2e/suite/) 168
Tool sub-packages tested 175
Core packages tested 17
Overall coverage (go test ./internal/... ./cmd/...) 91.1%
Overall coverage (go test ./internal/...) 94.8%
Average package coverage 95.3%

Naming Convention Stats

Pattern Count %
TestFunc_Scenario (2-part) 10,538 87.7%
TestFunc (no underscore) 963 8.0%
TestFunc_Scenario_Expected (3+ part) 511 4.3%

Test Distribution

By Layer

Layer Test Functions Test Files Description
Core packages 2,054 101 shared runtime packages such as config, GitLab client, OAuth, resources, prompts, and utilities
Tools orchestration 285 13 registration, meta-tool dispatch, safe mode, validation, markdown, and routing tests
Tool sub-packages (175) 8,343 345 domain-specific GitLab tool handlers
E2E integration 375 168 build-tagged real GitLab integration suite
cmd packages 955 68 server entry point and developer command utilities
Total 12,012 695

Core Packages

Package Tests Coverage Description
auditclient 2 100.0% Package auditclient creates GitLab clients for command-line audit tools.
autoupdate 122 94.7% Package autoupdate provides self-update capability for the gitlab-mcp-server MCP server.
cachehints 7 100.0% Package cachehints applies SEP-2549 cache hints (ttlMs/cacheScope) to MCP results.
cmdutil 5 100.0% Package cmdutil provides shared helpers for repository command utilities.
completions 96 100.0% Package completions provides a CompletionHandler for GitLab-aware autocomplete of prompt arguments and resource URI template parameters.
config 74 99.7% Package config loads, normalizes, and validates runtime configuration for the GitLab MCP server.
edition 5 87.0% Package edition defines the GitLab licensing tier model used to gate tool availability across the MCP server.
elicitation 100 85.0% Package elicitation provides a Client for requesting structured user input via the MCP elicitation protocol.
gitlab 44 100.0% Package gitlab provides a wrapper around the GitLab REST API v4 client.
oauth 35 98.6% Package oauth provides GitLab-specific OAuth 2.0 support for HTTP mode.
progress 17 100.0% Package progress provides a Tracker for sending MCP progress notifications to the client during long-running tool operations.
prompts 261 100.0% Package prompts registers MCP prompt templates that generate AI-optimized summaries, reviews, reports, and assessments from GitLab project, group, and cross-project data.
resources 182 100.0% Package resources registers read-only MCP resources for GitLab and server metadata.
serverpool 47 99.6% Package serverpool manages a pool of MCP servers keyed by GitLab token and URL.
testutil 29 85.3% Package testutil provides test helpers for gitlab-mcp-server.
toolutil 701 98.3% Package toolutil provides shared utilities for MCP tool handler sub-packages.
wizard 327 100.0% Package wizard implements the setup wizard that configures GitLab MCP Server credentials, binary installation, and IDE client configuration when the binary runs interactively instead of as an MCP stdio server.
Subtotal 2,054

Tool Sub-Packages (Top Domains by Test Count)

Sub-package Tests Coverage Tools
projects 390 100.0% 57
groups 244 99.7% 37
mergerequests 241 99.5% 30
issues 223 100.0% 21
users 210 100.0% 38
dynamic 159 99.7% 2
jobs 147 99.5% 17
packages 123 99.2% 9
search 118 100.0% 10
commits 114 99.8% 13
resourceevents 114 100.0% 17
awardemoji 113 100.0% 24
pipelines 111 100.0% 12
runners 109 100.0% 19
containerregistry 101 100.0% 16
accesstokens 96 99.8% 18
branches 95 100.0% 10
pipelineschedules 94 99.7% 11
groupmilestones 90 100.0% 8
snippets 89 99.5% 15
workitems 89 97.2% 6
mrapprovals 85 100.0% 7
tags 83 100.0% 9
groupmembers 82 100.0% 10
integrations 82 98.8% 12

Complete Tool Sub-Package Test Counts

All 175 tested sub-packages (click to expand)
Sub-package Tests Test Files Coverage Tools
accessrequests 41 2 100.0% 8
accesstokens 96 2 99.8% 18
actioncatalog 33 4 99.2% 0
actioncompat 44 2 100.0% 1
adminspecs 6 1 100.0% 92
alertmanagement 29 2 100.0% 4
appearance 10 1 100.0% 2
applications 20 1 100.0% 4
appstatistics 9 1 96.8% 1
attestations 18 2 100.0% 2
auditevents 45 2 100.0% 6
avatar 9 1 100.0% 1
awardemoji 113 1 100.0% 24
badges 56 1 100.0% 12
boards 73 2 99.3% 10
branches 95 1 100.0% 10
branchrules 16 1 100.0% 1
broadcastmessages 30 2 100.0% 5
bulkimports 35 2 100.0% 7
cicatalog 22 1 100.0% 2
cilint 26 1 100.0% 2
civariables 46 2 100.0% 5
ciyamltemplates 23 1 100.0% 2
clusteragents 42 1 100.0% 8
commitdiscussions 39 3 100.0% 6
commits 114 1 99.8% 13
compliancepolicy 6 1 100.0% 2
containerregistry 101 4 100.0% 16
customattributes 32 1 100.0% 4
customemoji 26 2 100.0% 3
dbmigrations 7 1 100.0% 1
dependencies 15 2 100.0% 4
dependencyproxy 5 1 100.0% 1
deploykeys 71 2 100.0% 9
deploymentmergerequests 24 1 95.8% 1
deployments 64 2 100.0% 6
deploytokens 68 2 100.0% 9
dockerfiletemplates 15 1 100.0% 2
dorametrics 11 2 100.0% 2
dynamic 159 8 99.7% 2
elicitationtools 61 2 99.5% 4
enterpriseusers 35 3 100.0% 4
environments 56 2 100.0% 6
epicdiscussions 16 2 100.0% 6
epicissues 18 2 100.0% 4
epicnotes 11 2 100.0% 5
epics 52 2 100.0% 6
epicworkitems 3 1 100.0% 0
errortracking 25 2 100.0% 5
events 52 2 100.0% 2
externalstatuschecks 51 3 100.0% 8
featureflags 43 2 99.5% 5
features 23 2 97.7% 4
ffuserlists 33 2 100.0% 5
files 80 2 100.0% 8
freezeperiods 36 2 100.0% 5
geo 54 2 100.0% 8
gitignoretemplates 15 1 100.0% 2
groupanalytics 8 2 100.0% 3
groupboards 65 2 98.5% 10
groupcredentials 43 3 100.0% 4
groupepicboards 15 3 98.3% 2
groupimportexport 29 1 100.0% 3
groupiterations 21 1 100.0% 1
grouplabels 54 2 100.0% 7
groupldap 14 2 100.0% 5
groupmarkdownuploads 37 2 100.0% 3
groupmembers 82 3 100.0% 10
groupmilestones 90 2 100.0% 8
groupprotectedbranches 16 2 100.0% 5
groupprotectedenvs 19 2 100.0% 5
grouprelationsexport 26 2 100.0% 2
groupreleases 18 3 100.0% 1
groups 244 8 99.7% 37
groupsaml 33 3 100.0% 5
groupscim 31 3 100.0% 4
groupserviceaccounts 20 2 100.0% 8
groupsshcerts 25 3 100.0% 3
groupstoragemoves 36 2 100.0% 6
groupvariables 54 2 100.0% 5
groupwikis 33 3 100.0% 5
health 17 1 100.0% 2
impersonationtokens 40 2 100.0% 5
importservice 31 1 100.0% 5
instancevariables 41 2 100.0% 5
integrations 82 4 98.8% 12
invites 40 1 100.0% 4
issuediscussions 44 2 100.0% 6
issuelinks 68 3 100.0% 4
issuenotes 48 2 100.0% 5
issues 223 2 100.0% 21
issuestatistics 43 1 100.0% 3
iterationdata 8 1 100.0% 0
jobs 147 4 99.5% 17
jobtokenscope 52 2 100.0% 8
keys 21 1 100.0% 2
labeldata 6 1 100.0% 0
labels 60 1 100.0% 8
license 18 2 100.0% 3
licensetemplates 21 1 100.0% 2
markdown 8 1 100.0% 1
memberroles 47 3 100.0% 6
members 60 2 100.0% 6
mergerequests 241 2 99.5% 30
mergetrains 16 2 100.0% 4
metadata 8 1 100.0% 1
milestones 73 1 100.0% 7
modelregistry 5 3 100.0% 1
mrapprovals 85 3 100.0% 7
mrapprovalsettings 9 2 100.0% 4
mrchanges 37 1 100.0% 4
mrcontextcommits 22 1 100.0% 3
mrdiscussions 58 1 100.0% 7
mrdraftnotes 71 2 100.0% 7
mrnotes 49 2 100.0% 5
namespaces 37 1 99.1% 4
notifications 29 1 100.0% 6
orbit 57 4 100.0% 6
packages 123 5 99.2% 9
pages 54 2 100.0% 9
pipelines 111 3 100.0% 12
pipelineschedules 94 2 99.7% 11
pipelinetriggers 56 2 100.0% 6
planlimits 13 2 100.0% 2
projectaliases 26 2 100.0% 4
projectdiscovery 19 1 100.0% 1
projectimportexport 40 1 100.0% 5
projectiterations 18 1 100.0% 1
projectmirrors 63 2 100.0% 7
projects 390 6 100.0% 57
projectserviceaccounts 11 2 100.0% 8
projectstatistics 8 2 100.0% 1
projectstoragemoves 19 2 100.0% 6
projecttemplates 19 1 100.0% 2
protectedenvs 42 2 100.0% 5
protectedpackages 33 2 100.0% 4
releaselinks 60 2 100.0% 6
releases 67 1 100.0% 6
repository 69 1 99.7% 9
repositorysubmodules 53 3 100.0% 3
resourceevents 114 3 100.0% 17
resourcegroups 16 1 100.0% 4
runnercontrollers 34 2 100.0% 5
runnercontrollerscopes 34 2 100.0% 5
runnercontrollertokens 41 2 100.0% 5
runners 109 2 100.0% 19
search 118 1 100.0% 10
securefiles 28 2 100.0% 4
securityattributes 24 1 100.0% 5
securitycategories 16 1 100.0% 3
securityfindings 20 1 100.0% 1
securityscanprofiles 17 1 100.0% 3
securitysettings 32 3 100.0% 3
serverupdate 25 1 98.8% 2
settings 17 1 93.8% 2
sidekiq 18 2 100.0% 4
snippetdiscussions 36 2 100.0% 6
snippetnotes 48 2 100.0% 5
snippets 89 3 99.5% 15
snippetstoragemoves 41 2 100.0% 6
surfaces 10 1 100.0% 0
systemhooks 36 2 100.0% 8
tags 83 2 100.0% 9
terraformstates 18 1 100.0% 6
todos 33 1 100.0% 3
topics 29 2 100.0% 5
uploads 47 2 100.0% 4
usagedata 28 1 100.0% 6
useremails 24 2 100.0% 6
usergpgkeys 44 2 100.0% 8
users 210 7 100.0% 38
vulnerabilities 61 3 100.0% 8
waitpoll 13 1 100.0% 0
wikis 60 2 100.0% 6
workitems 89 2 97.2% 6
Total 8,343 345 1,169

Coverage Report

cmd Package Snapshot

Package Coverage
cmd/audit_1to1 39.0%
cmd/audit_1to1/internal/actions 87.7%
cmd/audit_1to1/internal/merge 85.4%
cmd/audit_1to1/internal/metadata 69.1%
cmd/audit_1to1/internal/structs 95.0%
cmd/audit_catalog_first 80.5%
cmd/audit_discovery_completeness 77.9%
cmd/audit_doc_coverage 81.6%
cmd/audit_dynamic_aliases 48.4%
cmd/audit_e2e_gaps 87.8%
cmd/audit_edition_tier 36.0%
cmd/audit_metrics 52.4%
cmd/audit_string_dupes 90.7%
cmd/audit_surface_quality 71.8%
cmd/audit_test_names 48.1%
cmd/audit_tokens 59.2%
cmd/eval_mcp_surfaces/internal/evalrun 88.9%
cmd/eval_mcp_surfaces/internal/evaluator 69.2%
cmd/eval_mcp_surfaces/internal/evaluator/cases 99.6%
cmd/eval_mcp_surfaces/internal/termio 45.7%
cmd/format_md_tables 92.7%
cmd/gen_action_catalog_manifest 41.0%
cmd/gen_docker_tools 86.6%
cmd/gen_lhm_manifest 78.5%
cmd/gen_llms 24.7%
cmd/gen_stats 49.8%
cmd/gen_testing_docs 27.4%
cmd/godoc_tool 58.9%
cmd/internal/apidocs 86.7%
cmd/internal/docgen 99.5%
cmd/internal/mcpsurface 81.9%
cmd/server 78.7%

Core Packages

Package Coverage
auditclient 100.0%
autoupdate 94.7%
cachehints 100.0%
cmdutil 100.0%
completions 100.0%
config 99.7%
edition 87.0%
elicitation 85.0%
gitlab 100.0%
oauth 98.6%
progress 100.0%
prompts 100.0%
resources 100.0%
serverpool 99.6%
testutil 85.3%
toolutil 98.3%
wizard 100.0%

Tool Sub-Packages

Package Coverage
tools (orch.) 96.3%
accessrequests 100.0%
accesstokens 99.8%
actioncatalog 99.2%
actioncompat 100.0%
adminspecs 100.0%
alertmanagement 100.0%
appearance 100.0%
applications 100.0%
appstatistics 96.8%
attestations 100.0%
auditevents 100.0%
avatar 100.0%
awardemoji 100.0%
badges 100.0%
boards 99.3%
branches 100.0%
branchrules 100.0%
broadcastmessages 100.0%
bulkimports 100.0%
cicatalog 100.0%
cilint 100.0%
civariables 100.0%
ciyamltemplates 100.0%
clusteragents 100.0%
commitdiscussions 100.0%
commits 99.8%
compliancepolicy 100.0%
containerregistry 100.0%
customattributes 100.0%
customemoji 100.0%
dbmigrations 100.0%
dependencies 100.0%
dependencyproxy 100.0%
deploykeys 100.0%
deploymentmergerequests 95.8%
deployments 100.0%
deploytokens 100.0%
dockerfiletemplates 100.0%
dorametrics 100.0%
dynamic 99.7%
elicitationtools 99.5%
enterpriseusers 100.0%
environments 100.0%
epicdiscussions 100.0%
epicissues 100.0%
epicnotes 100.0%
epics 100.0%
epicworkitems 100.0%
errortracking 100.0%
events 100.0%
externalstatuschecks 100.0%
featureflags 99.5%
features 97.7%
ffuserlists 100.0%
files 100.0%
freezeperiods 100.0%
geo 100.0%
gitignoretemplates 100.0%
groupanalytics 100.0%
groupboards 98.5%
groupcredentials 100.0%
groupepicboards 98.3%
groupimportexport 100.0%
groupiterations 100.0%
grouplabels 100.0%
groupldap 100.0%
groupmarkdownuploads 100.0%
groupmembers 100.0%
groupmilestones 100.0%
groupprotectedbranches 100.0%
groupprotectedenvs 100.0%
grouprelationsexport 100.0%
groupreleases 100.0%
groups 99.7%
groupsaml 100.0%
groupscim 100.0%
groupserviceaccounts 100.0%
groupsshcerts 100.0%
groupstoragemoves 100.0%
groupvariables 100.0%
groupwikis 100.0%
health 100.0%
impersonationtokens 100.0%
importservice 100.0%
instancevariables 100.0%
integrations 98.8%
invites 100.0%
issuediscussions 100.0%
issuelinks 100.0%
issuenotes 100.0%
issues 100.0%
issuestatistics 100.0%
iterationdata 100.0%
jobs 99.5%
jobtokenscope 100.0%
keys 100.0%
labeldata 100.0%
labels 100.0%
license 100.0%
licensetemplates 100.0%
markdown 100.0%
memberroles 100.0%
members 100.0%
mergerequests 99.5%
mergetrains 100.0%
metadata 100.0%
milestones 100.0%
modelregistry 100.0%
mrapprovals 100.0%
mrapprovalsettings 100.0%
mrchanges 100.0%
mrcontextcommits 100.0%
mrdiscussions 100.0%
mrdraftnotes 100.0%
mrnotes 100.0%
namespaces 99.1%
notifications 100.0%
orbit 100.0%
packages 99.2%
pages 100.0%
pipelines 100.0%
pipelineschedules 99.7%
pipelinetriggers 100.0%
planlimits 100.0%
projectaliases 100.0%
projectdiscovery 100.0%
projectimportexport 100.0%
projectiterations 100.0%
projectmirrors 100.0%
projects 100.0%
projectserviceaccounts 100.0%
projectstatistics 100.0%
projectstoragemoves 100.0%
projecttemplates 100.0%
protectedenvs 100.0%
protectedpackages 100.0%
releaselinks 100.0%
releases 100.0%
repository 99.7%
repositorysubmodules 100.0%
resourceevents 100.0%
resourcegroups 100.0%
runnercontrollers 100.0%
runnercontrollerscopes 100.0%
runnercontrollertokens 100.0%
runners 100.0%
search 100.0%
securefiles 100.0%
securityattributes 100.0%
securitycategories 100.0%
securityfindings 100.0%
securityscanprofiles 100.0%
securitysettings 100.0%
serverupdate 98.8%
settings 93.8%
sidekiq 100.0%
snippetdiscussions 100.0%
snippetnotes 100.0%
snippets 99.5%
snippetstoragemoves 100.0%
surfaces 100.0%
systemhooks 100.0%
tags 100.0%
terraformstates 100.0%
todos 100.0%
topics 100.0%
uploads 100.0%
usagedata 100.0%
useremails 100.0%
usergpgkeys 100.0%
users 100.0%
vulnerabilities 100.0%
waitpoll 100.0%
wikis 100.0%
workitems 97.2%

Coverage target: >90% per package. Packages below the target in the latest generated coverage snapshot:

  • cmd/gen_llms (24.7%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/gen_testing_docs (27.4%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_edition_tier (36.0%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_1to1 (39.0%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/gen_action_catalog_manifest (41.0%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/eval_mcp_surfaces/internal/termio (45.7%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_test_names (48.1%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_dynamic_aliases (48.4%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/gen_stats (49.8%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_metrics (52.4%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/godoc_tool (58.9%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_tokens (59.2%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_1to1/internal/metadata (69.1%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/eval_mcp_surfaces/internal/evaluator (69.2%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_surface_quality (71.8%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_discovery_completeness (77.9%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/gen_lhm_manifest (78.5%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/server (78.7%) - entry-point glue, signal handling, and transport startup are validated mostly through integration and E2E coverage.
  • cmd/audit_catalog_first (80.5%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_doc_coverage (81.6%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/internal/mcpsurface (81.9%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • elicitation (85.0%) - review this package for missing unit coverage or add an explicit exception if the remaining paths are integration-only.
  • testutil (85.3%) - some helpers are exercised by external packages or the build-tagged E2E suite rather than this package's own tests.
  • cmd/audit_1to1/internal/merge (85.4%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/gen_docker_tools (86.6%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/internal/apidocs (86.7%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • edition (87.0%) - review this package for missing unit coverage or add an explicit exception if the remaining paths are integration-only.
  • cmd/audit_1to1/internal/actions (87.7%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/audit_e2e_gaps (87.8%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.
  • cmd/eval_mcp_surfaces/internal/evalrun (88.9%) - developer command formatting and reporting branches are covered by focused unit tests plus manual/CI tooling runs.

Test Types

Unit Tests

All unit tests use httptest to mock GitLab API responses. No real GitLab API calls are made during unit testing.

Patterns used:

  • Table-driven tests with t.Run() subtests — standard across all packages
  • Mock server: testutil.NewTestClient() creates a GitLab client pointing to a local httptest.Server
  • JSON responses: testutil.RespondJSON() and testutil.RespondJSONWithPagination() helpers
  • Naming convention: TestToolName_Scenario_ExpectedResult

Example structure:

func TestGetBranch_Success(t *testing.T) {
    client, mux, cleanup := testutil.NewTestClient()
    defer cleanup()

    mux.HandleFunc("/api/v4/projects/1/repository/branches/main", func(w http.ResponseWriter, r *http.Request) {
        testutil.RespondJSON(w, gitlab.Branch{Name: "main"})
    })

    // ... invoke tool handler, assert result
}

End-to-End Tests

E2E tests run against a real GitLab instance using in-memory MCP transport (build tag e2e). Two modes are supported:

Self-Hosted Mode

Requires a running GitLab instance with credentials in .env:

# .env
GITLAB_URL=https://gitlab.example.com
GITLAB_TOKEN=glpat-...
go test -v -tags e2e -timeout 300s ./test/e2e/suite/
make test-e2e

Docker Mode

Uses an ephemeral GitLab CE container provisioned by Docker Compose. Requires Docker and ~4 GB RAM. Enterprise mode uses the same topology with a GitLab EE image plus a locally supplied Ultimate license.

All E2E Docker infrastructure is version-controlled under test/e2e/:

  • test/e2e/docker-compose.yml — GitLab CE/EE + Runner + fixture service compose definition
  • test/e2e/scripts/setup-gitlab.sh — Creates test user, PAT, installs ENTERPRISE_LICENSE when requested, writes .env.docker
  • test/e2e/scripts/register-runner.sh — Registers CI runner in GitLab
  • test/e2e/scripts/wait-for-gitlab.sh — Polls GitLab readiness endpoint
# Start GitLab container and provision test environment
docker compose -f test/e2e/docker-compose.yml up -d
./test/e2e/scripts/wait-for-gitlab.sh
./test/e2e/scripts/setup-gitlab.sh    # Creates .env.docker
./test/e2e/scripts/register-runner.sh # Registers CI runner

# Run tests
set -a && source test/e2e/.env.docker && set +a
go test -v -tags e2e -timeout 600s ./test/e2e/suite/

# Cleanup
docker compose -f test/e2e/docker-compose.yml down -v

Or use the Makefile target that automates the full lifecycle:

make test-e2e-docker

For Enterprise/Premium E2E coverage, set ENTERPRISE_LICENSE in .env or the shell and use:

make test-e2e-docker-enterprise

The Enterprise target runs with the e2e enterprise build tags, so common harness files plus test/e2e/suite/*_ee_test.go Enterprise/Premium tests are compiled and executed. CE-only tests live in test/e2e/suite/*_ce_test.go and remain in make test-e2e-docker, while Enterprise-specific fixture behavior can be tuned independently.

The E2E harness also re-validates the GitLab tier at runtime by calling the License API (GET /api/v4/license). When an enterprise tier is requested (via GITLAB_TIER=premium/ultimate, or the legacy GITLAB_ENTERPRISE=true harness toggle) but the fixture reports a Free license, the session downgrades to CE and *_ee_test.go tests skip cleanly with a logged reason instead of failing outright. This keeps the suite safe against accidental CE/EE mismatches.

Docker mode enables pipeline and job tests that require a CI runner. It also starts an internal e2e-fixture HTTP service and configures GitLab to allow local outbound requests, so project webhook, push mirror, and custom emoji tests use deterministic in-network endpoints instead of public Internet access.

Test Reports

make test-e2e, make test-e2e-docker, and make test-e2e-docker-enterprise use gotestsum to produce structured test reports in dist/e2e-reports/:

File Format Purpose
e2e-junit.xml JUnit XML CI/CD integration (GitHub Actions, SonarQube)
e2e-log.json JSON Programmatic analysis, filtering
e2e-output.txt Plain Human-readable console output (testdox)

Docker mode files use the e2e-docker- prefix, and Enterprise Docker files use the e2e-docker-enterprise- prefix. Reports are written to dist/e2e-reports/ (gitignored via dist/).

The Makefile targets run gotestsum through tee with pipefail so test failures propagate to the target exit code. Docker targets still tear down containers and volumes before returning a non-zero status on failure.

Install gotestsum via make install-tools or go install gotest.tools/gotestsum@latest.

Test Architecture

The suite uses five MCP server/client pairs via mcp.NewInMemoryTransports():

Session Purpose
individual Individual GitLab tools
meta Domain meta-tools and action dispatch
elicitation Elicitation tools with mock user handler
safeMode Mutating tools wrapped as safe-mode previews

Workflows:

Area Description
Individual tools Exercises domain tools directly against real GitLab
Meta-tools Exercises domain action dispatch through meta-tools
MCP capabilities Verifies progress, completions, elicitation, and safe mode
Docker-only runner Exercises CI pipeline and job behavior with a registered runner

Docker validation snapshots are written under dist/e2e-reports/ after make test-e2e-docker or make test-e2e-docker-enterprise. The generated metrics above count E2E Test* entry points statically; they do not replace the runtime report produced by gotestsum.

Lifecycle covered: user → project CRUD → commits → branches → tags → releases → issues → labels → milestones → members → upload → MR lifecycle → notes → discussions → search → groups → pipelines → packages → wikis → CI variables → environments → issue links → deploy keys → snippets → pipeline schedules → badges → access tokens → award emoji → elicitation → cleanup

Domains added in Docker mode (require CI runner):

  • Pipeline create/get/cancel/retry/delete
  • Job get/log/retry/cancel

MCP capability tests (mock handlers):

  • Sampling tools (11 tests): summarize issue, analyze MR changes, generate release notes, etc.
  • Elicitation tools (1 test): confirm destructive action

Fixture Cleanup

Test fixtures (fixture_ce_test.go / fixture_ee_test.go) register t.Cleanup handlers that permanently delete projects created during tests. GitLab's Delayed Deletion feature requires a two-step process:

  1. Mark the project for deletion (DELETE /projects/:id)
  2. Permanently remove it (DELETE /projects/:id?permanently_remove=true&full_path=...)

The cleanupOrphanedProjects function in setup_test.go runs at suite start to remove leftover projects from interrupted runs, including those already in pending-delete state (IncludePendingDelete option).

Meta-Tool Tests

Meta-tool tests verify the action-dispatch layer that consolidates individual tools into base and Enterprise/Premium domain meta-tools. These tests live in internal/tools/ (the orchestration package).

What meta-tool tests cover:

  • Action routing: Each meta-tool correctly dispatches to the underlying sub-package handler based on the action parameter
  • Invalid action: Requests with unknown actions return an error listing valid actions
  • Metadata audit: TestMetadataAudit_* tests enforce naming conventions, annotations, and tool count invariants across the registered tool catalog
  • Destructive metadata consistency: TestDestructiveMetadataConsistency cross-checks ActionRoute.Destructive metadata against toolutil.DeleteAnnotations on individual tools — ensures meta-tool routes and individual tools agree on which actions are destructive
  • Markdown formatting: markdownForResult delegates to the type-based registry (toolutil.MarkdownForResult) which invokes the formatter registered by the sub-package init() function
  • next_steps enrichment: enrichWithHints() correctly extracts hints from Markdown and injects them into JSON structuredContent

Running meta-tool tests:

# All orchestration tests (register, metatool, markdown, errors)
go test ./internal/tools/ -count=1 -v

# Metadata audit only
go test ./internal/tools/ -run TestMetadataAudit -count=1 -v

# Specific domain meta-tool tests
go test ./internal/tools/ -run TestProject -count=1 -v
go test ./internal/tools/ -run TestBranch -count=1 -v

E2E meta-tool tests:

The E2E suite now uses domain-focused TestMeta_* entry points rather than one large workflow test. These tests exercise project lifecycle operations and extended domains through meta-tool action dispatch, validating routing, parameter passthrough, and response formatting in a real GitLab environment.

# Run all meta-tool E2E tests
go test -v -tags e2e -timeout 300s -run '^TestMeta_' ./test/e2e/suite/

Validation Tests

Validation tests in internal/tools/register_validation_test.go ensure structural integrity across all sub-packages:

Test Purpose
TestRegisterAllDoesNotUseDomainRegisterTools Verifies root individual registration stays catalog-backed and cannot regress to per-domain RegisterTools loops
TestActionSpecCoverage_AllCatalogRoutesClassified Builds the GitLab.com Enterprise dynamic catalog and verifies every catalog action is spec-backed
TestAllMarkdownFormattersRegistered Verifies all ~266 output types across 76 sub-packages have registered markdown formatters via toolutil.RegisterMarkdown[T]
TestAllHintReferencesValid Validates all action 'xxx' and backtick-quoted `gitlab_xxx` references in WriteHints across all markdown.go files match registered tools/actions 
# Run validation tests
go test ./internal/tools/ -run "TestRegisterAllDoesNotUseDomainRegisterTools|TestActionSpecCoverage|TestAllMarkdown|TestAllHint" -count=1 -v

Running Tests

Unit Tests

# All unit tests
go test ./internal/... -count=1

# Specific package (verbose)
go test ./internal/tools/branches/ -count=1 -v

# Specific test by name
go test ./internal/tools/ -run TestBranch -count=1

# With coverage
go test ./internal/tools/branches/ -coverprofile=cover.out -count=1
go tool cover -func=cover.out

# With race detector
go test ./internal/... -race -count=1

E2E Tests

# Full suite (self-hosted GitLab)
go test -v -tags e2e -timeout 300s ./test/e2e/suite/
make test-e2e

# Docker mode (ephemeral GitLab CE container)
docker compose -f test/e2e/docker-compose.yml up -d
./test/e2e/scripts/wait-for-gitlab.sh && ./test/e2e/scripts/setup-gitlab.sh && ./test/e2e/scripts/register-runner.sh
set -a && source test/e2e/.env.docker && set +a
go test -v -tags e2e -timeout 600s ./test/e2e/suite/
docker compose -f test/e2e/docker-compose.yml down -v

# Individual and meta-tool domains
go test -v -tags e2e -timeout 300s -run '^TestIndividual_' ./test/e2e/suite/
go test -v -tags e2e -timeout 300s -run '^TestMeta_' ./test/e2e/suite/

# Compile-only (verify builds without GitLab)
go test -tags e2e -c -o NUL ./test/e2e/suite/       # Windows
go test -tags e2e -c -o /dev/null ./test/e2e/suite/  # Linux

Coverage Report

# Full coverage for all internal packages
go test ./internal/... -coverprofile=coverage.out -count=1
go tool cover -func=coverage.out

# HTML coverage report
go tool cover -html=coverage.out -o coverage.html

# Per-package summary
go test ./internal/... -cover -count=1

Makefile Targets

make test          # Run all unit tests
make test-race     # Run with race detector
make test-e2e      # Run E2E tests (self-hosted GitLab) — generates JUnit + JSON reports
make test-e2e-docker # Run E2E tests with ephemeral GitLab CE — generates JUnit + JSON reports
make test-e2e-docker-enterprise # Run E2E tests with ephemeral GitLab EE + license
make test-e2e-gitlab-com # Run Orbit live tests against GitLab.com (provisions fixtures, waits for indexer, then runs the orbitlive-tagged tests)
make coverage      # Generate coverage report
make lint          # Run consolidated golangci-lint checks
make inspector     # Compile + launch MCP Inspector UI via stdio
make inspector-stop # Stop Inspector and clean up temp binary

Orbit Live Tests

The six gitlab_orbit_* tools have a separate orbitlive-gated live test suite at test/e2e/orbit/live_test.go that exercises the real https://gitlab.com/api/v4/orbit/* endpoints against a fixture-provisioned namespace. Unlike the e2e-tagged suite, these tests are not run by make test or any CI gate — they require a GitLab.com Personal Access Token and explicit opt-in.

The suite is organized as four entry points:

Entry point Subtests What it exercises
TestOrbitLiveGitLabCom 14 All six handlers against the live API: status, schema, tools, DSL (default/llm/raw), query (traversal/aggregation/neighbors/path_finding/llm-format), and graph_status (full_path/namespace_id)
TestOrbitLiveGitLabCom_ShapeDiscovery 6 Regression coverage of the canonical Query DSL shapes for each query_type variant — aggregation_with_filter, aggregation_with_node_ids, neighbors_id_reference, path_finding_shortest, and the default schema format
TestOrbitLiveGitLabCom_Fixtures 7 Filter-based queries against the live kg-fixtures and security-fixtures projects, scoped by ORBIT_FIXTURES_NAMESPACE so the test is portable across developer namespaces
TestOrbitLiveGitLabCom_FeatureCoverage 14 Comprehensive DSL surface: filter operators (in, contains, gt), multi-node traversal with IN_PROJECT, aggregations with group_by (node/property), sum/max/avg, order_by, virtual columns (diff, content), cursor pagination, id_range scope, and options.dynamic_columns

Total: 4 suites, 41 subtests behind the orbitlive build tag.

The Orbit indexer is eventually consistent. Subtests that match content the indexer has not yet picked up will report row_count=0 and pass — they are informational, not strict equality. Re-run the live test a few minutes after make test-e2e-gitlab-com to allow the indexer to catch up.

To run the full flow:

# Add a Personal Access Token (api scope) to .env first
echo 'GITLAB_COM_TOKEN=glpat-...' >> .env

# Default namespace is plens1; override with ORBIT_FIXTURES_NAMESPACE
make test-e2e-gitlab-com ORBIT_FIXTURES_NAMESPACE=acme-research

To run only the live tests (when fixtures are already provisioned):

GITLAB_COM_TOKEN=glpat-... \
  go test -tags orbitlive -count=1 -v -timeout 300s ./test/e2e/orbit/

# Just one suite
GITLAB_COM_TOKEN=glpat-... \
  go test -tags orbitlive -count=1 -v -run '^TestOrbitLiveGitLabCom_Fixtures$' ./test/e2e/orbit/

See Orbit Live Test Fixtures for fixture contents, the scripts/setup-orbit-fixtures.sh script, and the indexer caveat.

Test Infrastructure

Shared Helpers (internal/testutil/)

Helper Purpose
NewTestClient() Creates mock GitLab client + httptest server
RespondJSON() Writes JSON response body
RespondJSONWithPagination() Writes JSON + pagination headers

Test File Organization

Each tool sub-package follows this structure:

internal/tools/{domain}/
├── {domain}.go          # Tool handlers
├── {domain}_test.go     # Unit tests
├── action_specs.go      # Canonical ActionSpec route metadata
├── markdown.go          # Markdown formatters (if any)
└── markdown_test.go     # Formatter tests (if any)

E2E Test Structure

test/e2e/
├── docker-compose.yml        # Ephemeral GitLab CE + Runner + fixture service
├── .env.docker               # Docker mode environment variables
├── README.md                 # E2E documentation
├── scripts/                  # Provisioning scripts
│   ├── register-runner.sh
│   ├── setup-gitlab.sh
│   └── wait-for-gitlab.sh
└── suite/                    # Go test package (137 test files)
    ├── setup_test.go         # MCP server setup, helpers, shared state
    ├── fixture_ce_test.go    # Self-contained GitLab CE resource builders
    ├── fixture_ee_test.go    # Self-contained GitLab EE resource builders
    └── *_test.go             # Domain-specific test files

Wizard Test Helpers

The internal/wizard/ package tests interactive UI code (Web UI, Bubble Tea TUI, CLI) that would normally open browsers, OS dialogs, and write to real user config files. Test isolation is achieved via package-level function variables overridden in tests with t.Cleanup to restore originals.

Function variables (defined in source files, overridden in tests):

Variable Source file Real function Purpose
allClientsFn clients.go AllClients() Returns MCP client configs
openBrowserFn browser.go openBrowser() Launches default browser
pickDirectoryFn dirpicker.go pickDirectory() Opens OS directory picker

Test helpers (testhelpers_test.go):

Helper Purpose
useFakeClients(t) Overrides allClientsFn with clients using temp dir paths — prevents writing to real mcp.json files
stubPickDirectory(t, path, err) Overrides pickDirectoryFn — prevents OS directory dialog
stubOpenBrowser(t) Overrides openBrowserFn — prevents browser launch

File organization (12 test files, 159 test functions):

internal/wizard/
├── clients_test.go        # 25 tests — MCP client detection and config paths
├── cli_test.go            # 18 tests — CLI-mode wizard flow
├── envfile_test.go        #  3 tests — .env file operations
├── install_test.go        #  6 tests — Binary installation logic
├── jsonmerge_test.go      #  9 tests — JSON config merge operations
├── paths_test.go          #  4 tests — Platform-specific path resolution
├── prompt_test.go         # 20 tests — User prompt/input handling
├── run_test.go            #  1 test  — Top-level Run() entry point
├── testhelpers_test.go    #  0 tests — Shared test helpers only
├── tui_test.go            # 43 tests — Bubble Tea TUI model and view
├── webui_test.go          # 15 tests — Web UI HTTP handlers
└── wizard_test.go         # 15 tests — Core wizard orchestration