Skip to content

docs(skills): refresh SKILL.md frontmatter for routing clarity (sensei audit)#51

Open
spboyer wants to merge 1 commit intogoogle:mainfrom
spboyer:sensei/frontmatter-improvements
Open

docs(skills): refresh SKILL.md frontmatter for routing clarity (sensei audit)#51
spboyer wants to merge 1 commit intogoogle:mainfrom
spboyer:sensei/frontmatter-improvements

Conversation

@spboyer
Copy link
Copy Markdown

@spboyer spboyer commented Apr 29, 2026

Summary

Refreshes the YAML frontmatter on all 13 skills under skills/cloud/ to improve agent skill-routing accuracy. Body content is unchanged across all files — your ## Quick Start, ## Prerequisites, ## Reference Directory, and reference graph (references/cli-usage.md, references/iam-security.md, etc.) are preserved as-is.

This was generated with sensei, a SKILL.md frontmatter compliance auditor that follows research-backed routing patterns from Anthropic's skill design guidance and recent cross-model trigger studies.

Why this matters

When an agent host loads many skills (Copilot CLI, Claude, Cursor, etc.), it relies on the YAML description: field to decide which skill to invoke for a given user prompt. Sparse or generic descriptions cause:

  • Wrong-skill activationbigquery-basics triggering for a Cloud SQL prompt
  • Skill collisions — multiple skills firing for the same intent
  • Missed activations — the right skill not loading because keywords don't match

The fix is positive routing: an action verb, a unique domain, and quoted trigger phrases the LLM can pattern-match.

What changed (per skill)

Each description: field now follows this template:

description: "**[CATEGORY] SKILL** — [Action verb + domain]. WHEN: \"phrase1\", \"phrase2\", .... INVOKES: [tools/MCP servers]."

Concretely:

  1. Single-line description: "..." instead of YAML block scalars (>- or |).
    Why: some skill loaders use minimal YAML parsers and miscount multiline values. Single-line strings are universally parsed.
  2. **WORKFLOW SKILL** / **UTILITY SKILL** / **ANALYSIS SKILL** prefix.
    Why: gives the host a quick classification signal — orchestration vs. helper vs. read-only assessment.
  3. WHEN: "phrase1", "phrase2", ... with quoted, distinctive triggers.
    Why: quoted phrases anchor pattern matching. Bare keywords blend into prose; quoted phrases stand out as routing signals.
  4. INVOKES: ... listing MCP servers and CLI tools each skill calls (e.g., BigQuery MCP server, bq CLI, Gemini CLI extension).
    Why: helps the host disambiguate when both a workflow skill and a same-named MCP tool are available.
  5. Description length kept under 1024 chars (range: 369–481 chars across the 13).

Bug fix included

skills/cloud/google-cloud-networking-observability/SKILL.md had:

name: google-cloud-recipe-networking-observability   # ← did not match folder

The folder is google-cloud-networking-observability (no recipe-). The other recipe skills (google-cloud-recipe-auth, google-cloud-recipe-onboarding) match their folders correctly. Updated name: to match the folder.

Sensei score (baseline → after refresh)

Sensei's content-quality scorer (0.0–1.0, higher is better) on each skill:

Skill Before After Δ
alloydb-basics 0.00 0.33 +0.33
bigquery-basics 0.17 0.50 +0.33
cloud-run-basics 0.17 0.50 +0.33
cloud-sql-basics 0.00 0.33 +0.33
firebase-basics 0.31 0.50 +0.19
gemini-api 0.17 0.33 +0.16
gke-basics 0.50 0.50 0.00 (already had WHEN:)
google-cloud-networking-observability 0.17 0.50 +0.33
google-cloud-recipe-auth 0.33 0.50 +0.17
google-cloud-recipe-onboarding 0.50 0.67 +0.17
google-cloud-waf-cost-optimization 0.33 0.50 +0.17
google-cloud-waf-reliability 0.33 0.50 +0.17
google-cloud-waf-security 0.33 0.50 +0.17

The remaining ceiling (~0.50) is body-structural — sensei's scorer also rewards ## Rules and ## Steps sections in the body, but those would require rewriting your existing ## Quick Start / ## Reference Directory structure. This PR deliberately stops at frontmatter to keep the change scope focused and non-destructive.

What this PR does NOT change

  • ❌ No body content modifications — every ## Quick Start, code sample, and references/ link is preserved
  • ❌ No file additions or deletions
  • ❌ No license, metadata, or compatibility: field changes (other than to fix the one name mismatch)
  • ❌ No removal of the Use when ... natural-language guidance — WHEN: augments routing, doesn't replace existing prose

Reproducing locally

git clone https://github.com/spboyer/sensei
cd sensei/scripts/src/gepa
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt litellm

# Score baseline
python auto_evaluator.py score-all \
  --skills-dir /path/to/skills/cloud \
  --tests-dir /path/to/tests

References


@google-cla
Copy link
Copy Markdown

google-cla Bot commented Apr 29, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@spboyer spboyer force-pushed the sensei/frontmatter-improvements branch 3 times, most recently from 1473a43 to 1a21c6c Compare May 4, 2026 18:14
@spboyer spboyer force-pushed the sensei/frontmatter-improvements branch from 1a21c6c to 036e1c2 Compare May 4, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant