[prompt-analysis] Copilot PR Prompt Analysis - 2026-08-15

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-08-15

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 769 (76.9%) | Closed: 227 (22.7%) | Open: 4 (0.4%)

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Bug Fix	535	411	122	77.1%
Feature Addition	288	222	65	77.4%
Other/Uncategorized	103	82	20	80.4%
Testing	33	26	7	78.8%
Refactoring	24	17	7	70.8%
Documentation	17	11	6	64.7%

(Success rate = merged / (merged + closed), open PRs excluded from denominator.)

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 173.8 words — notably shorter than closed PRs
• Prompts are overwhelmingly auto-generated task descriptions (from daily-* scanning/maintenance workflows) that embed a specific, scoped problem statement — e.g. a single CVE, a single vulnerable dependency, or a single failing behavior — plus a concrete fix action.
• High density of concrete technical references: package/image names, version numbers, file paths, and workflow names.

Example successful prompts:

1. PR Remove vulnerable cli-proxy 0.27.44 container pin #52934: "Remove vulnerable cli-proxy 0.27.44 container pin. The pinned image contains multiple high-severity Node.js and npm dependency vulnerabilities..." → Merged
2. PR Expand Grant license exceptions for gh-aw-mcpg container scan findings #52933: "Expand Grant license exceptions for gh-aw-mcpg container scan findings. The daily container image scan reported ghcr.io/github/gh-aw-mcpg:v0.4.9 with high CVEs..." → Merged
3. PR Enable cloud-hypervisor agent runtime on eligible agentic workflows #52932: "Enable cloud-hypervisor agent runtime on eligible agentic workflows. Opts a set of repository workflows into the Cloud Hypervisor preview microVM agent runtime..." → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 219.8 words — ~26% longer than merged prompts
• Heavier reliance on raw vulnerability-scanner dump text (CVE IDs, securitytracker/Debian gpl/license-scan boilerplate) copy-pasted into the prompt rather than a distilled ask, making the actual required change harder to infer.
• Several closed PRs describe investigation/confirmation tasks rather than a concrete change ("Confirm safe patch dependency updates are already present" — the fix already existed, so there was nothing to merge) or broad, multi-service routing changes with higher blast radius (e.g. re-pointing an MCP server image to :latest with digest pinning across workflows).

Example unsuccessful prompts:

1. PR Route Serena MCP workflows through :latest with digest pinning for security refreshes #52935: "Route Serena MCP workflows through :latest with digest pinning for security refreshes. ...highest-risk image in daily scan output, with large critical/high CVE volume..." → Closed
2. PR Confirm safe patch dependency updates are already present #52919: "Confirm safe patch dependency updates are already present. The issue requested patch updates for golang.org/x/crypto and charm.land/lipgloss/v2. Current branch state already includes both..." → Closed
3. PR Fail fast when required MCP servers are omitted #52854: "Fail fast when required MCP servers are omitted. Missing MCP secrets could cause Sentry/Grafana servers to disappear from the gateway output..." → Closed

Key Insights

• Length matters inversely here: merged PRs average ~46 fewer words than closed ones (173.8 vs 219.8). Longer prompts tend to correlate with copy-pasted scanner/CVE dumps rather than distilled, actionable instructions — suggesting conciseness and pre-digestion of the problem improves merge odds.
• Documentation and refactoring prompts underperform (64.7% and 70.8% success respectively) relative to bug fixes and features (~77%), suggesting these categories may need clearer acceptance criteria or smaller, more atomic scope to avoid reviewer friction or scope creep.
• "Confirm/verify" style prompts have low value-add: tasks framed as "confirm X is already true" rather than "change X to Y" are more likely to close without a merge since there's often no code change needed — these should be filtered out at the issue-creation stage rather than dispatched to Copilot as coding tasks.
• Single, scoped technical asks outperform broad routing/infra changes: the closed examples show multi-workflow or multi-service migrations (e.g., repointing several workflows to :latest with new pinning) carry more review risk than single-package/single-file fixes.

Recommendations

Based on today's analysis:

1. DO: Keep prompts concise (~150-180 words) and centered on one concrete technical change (one CVE, one file, one behavior) rather than pasting raw scanner output.
2. DO: For security/dependency-bump prompts, distill the vulnerability report into a single actionable sentence (package, version, target) instead of including the full scanner dump — this correlates with materially higher merge rates in this sample.
3. AVOID: Dispatching "confirm/verify already-fixed" issues as Copilot coding tasks — pre-check current state before creating the issue to avoid a guaranteed no-op PR.
4. AVOID: Bundling broad, cross-workflow infrastructure changes (e.g., simultaneous image repointing + digest pinning across several MCP servers) into a single ask; split into smaller, single-service PRs to reduce review overhead.

Historical Trends

Date	PRs	Merge Rate	Notes
2026-08-15 (today)	1000	77.2%	bug_fix remains top category by volume
2026-07-07	1000	80.0%	bug_fix top category
2026-07-01	—	81.2%	earliest tracked snapshot

Trend: Merge rate has declined from ~81% (early July) to ~77% (today), a ~4 point drop over the tracked period. Category mix has remained bug-fix dominant throughout, but the growing share of longer, scanner-dump-style prompts (closed-PR avg word count rose relative to merged) may be contributing to the softer success rate.

---

Generated by Copilot PR Prompt Analysis (Run: 31907448435)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · auto · 25.2 AIC · ⌖ 12.7 AIC · ⊞ 10.9K · ◷

• expires on Aug 16, 2026, 12:50 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #53203.