AI diagnosis: fetch full source files and validate suggested patches by PhenX · Pull Request #202 · PiwiTests/platform

PhenX · 2026-07-03T21:39:26Z

Phase 0 (hygiene):

Add testSource to TestCasePayload/StreamEventPayload (it was captured, stored and used but missing from the canonical wire types).
Namespace SCM provider caches by a token hash so a public (token-less) fetch can't serve its result to an authenticated caller or vice-versa.
Compute GitLab additions/deletions from the diff text (were hardcoded 0).
Implement Bitbucket probeError (was a stub returning null).
Sync the docs context-limits table with the actual defaults.

Phase 1 (verified patches on today's context):

Add fetchFileAtRef + fetchTree to every SCM provider (file content at a ref, recursive tree), with per-SHA caches and a 200 KB size cap.
New "Source Files" diagnosis context section: full content of the top-suspect changed files plus the failing test's local import closure, at the commit under test, line-numbered so the model can compute correct hunk headers. Capped by maxSourceFiles / sourceFileChars context limits (SectionId, DIAGNOSIS_SECTIONS, coverage and the limit registry updated together).
shared/patch.ts: a dependency-free unified-diff parser and dry-run applier; every suggestedFix.patch is validated against the exact source files the model was shown and the result stored in details.patchValidation.
DiagnosisResult.vue renders a validation badge (applies cleanly / with offset / does not apply / invalid / unverified) plus a Download .patch action.
System prompt now tells the model to ground hunks in the shown source files and that patches are validated.

Unit tests for the patch validator and the path-resolution helpers; docs updated.

Claude-Session: https://claude.ai/code/session_01GoDzedfU4hJXqFg9fja5XP

Phase 0 (hygiene): - Add testSource to TestCasePayload/StreamEventPayload (it was captured, stored and used but missing from the canonical wire types). - Namespace SCM provider caches by a token hash so a public (token-less) fetch can't serve its result to an authenticated caller or vice-versa. - Compute GitLab additions/deletions from the diff text (were hardcoded 0). - Implement Bitbucket probeError (was a stub returning null). - Sync the docs context-limits table with the actual defaults. Phase 1 (verified patches on today's context): - Add fetchFileAtRef + fetchTree to every SCM provider (file content at a ref, recursive tree), with per-SHA caches and a 200 KB size cap. - New "Source Files" diagnosis context section: full content of the top-suspect changed files plus the failing test's local import closure, at the commit under test, line-numbered so the model can compute correct hunk headers. Capped by maxSourceFiles / sourceFileChars context limits (SectionId, DIAGNOSIS_SECTIONS, coverage and the limit registry updated together). - shared/patch.ts: a dependency-free unified-diff parser and dry-run applier; every suggestedFix.patch is validated against the exact source files the model was shown and the result stored in details.patchValidation. - DiagnosisResult.vue renders a validation badge (applies cleanly / with offset / does not apply / invalid / unverified) plus a Download .patch action. - System prompt now tells the model to ground hunks in the shown source files and that patches are validated. Unit tests for the patch validator and the path-resolution helpers; docs updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoDzedfU4hJXqFg9fja5XP

Copilot

Pull request overview

This PR strengthens AI diagnosis patch reliability by fetching full source files from SCM to ground patch hunks, then validating suggested unified-diff patches server-side against the provided source, and surfacing the validation status + patch download in the UI. It also includes SCM hygiene improvements (cache token namespacing, GitLab diff stats, Bitbucket probing) and updates context-limit documentation/types.

Changes:

Add a new “Source Files” diagnosis context section (SCM-fetched full files + failing test import closure) and new context limits/env vars to cap it.
Introduce a dependency-free unified-diff parser + dry-run patch validator, store validation results on diagnoses, and render badges + “Download .patch” in the UI.
Improve SCM provider behavior (token-scoped caches, GitLab additions/deletions parsing, Bitbucket probeError, file-at-ref fetching).

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
docs/ai-diagnosis.md	Documents source-file grounding, patch validation statuses, and updates context-limit defaults.
application/types/api.ts	Extends context coverage typing to include fetched source file coverage info.
application/tests/unit/patch.test.ts	Adds unit coverage for unified diff parsing + patch validation behaviors.
application/tests/unit/ai-context-sections.test.ts	Adds unit tests for import path resolution and candidate file path expansion.
application/shared/types.ts	Adds `testSource` to canonical wire payload types (test case + stream events).
application/shared/piwi-env-vars.ts	Registers env vars for the new source-file context limits.
application/shared/patch.ts	New unified-diff parser + dry-run validator for suggested patches.
application/shared/diagnosis-sections.ts	Adds `sourceFiles` to the diagnosis sections registry.
application/shared/ai-context-limits.ts	Adds defaults + env-backed config for source file count/size caps.
application/server/utils/scm/ScmProvider.ts	Extends SCM provider contract (file content + tree) and token-hash cache prefixing.
application/server/utils/scm/GitLabProvider.ts	Adds token-namespaced caches, line-add/del counting, file-at-ref, and tree listing.
application/server/utils/scm/GitHubProvider.ts	Adds token-namespaced caches, file-at-ref, and tree listing (used for path normalization).
application/server/utils/scm/BitbucketProvider.ts	Adds token-namespaced caches, file-at-ref, and implements `probeError` messaging.
application/server/utils/ai-system-prompt.ts	Updates patch instructions to require grounding in Source Files / Test Source.
application/server/utils/ai-diagnosis.ts	Validates `suggestedFix.patch` and stores `details.patchValidation`.
application/server/utils/ai-context.types.ts	Adds `sourceFiles` section id and stores fetched file content in built context.
application/server/utils/ai-context.ts	Builds the Source Files context section and collects fetched files for validation.
application/app/components/diagnosis/DiagnosisResult.vue	Renders patch validation badge + provides “Download .patch”.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+  async fetchTree(ref: string): Promise<string[] | null> {
+    const key = `${this.keyPrefix}:tree:${this.repoPath}:${ref}`;
+    const hit = fetchTreeCache.get(key);
+    if (hit !== undefined) return hit;
+
+    const res = await fetch(`https://api.github.com/repos/${this.repoPath}/git/trees/${ref}?recursive=1`, {
+      headers: this.makeHeaders(),
+      signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
+    });
+    if (!res.ok) return null;
+    const data = (await res.json()) as { tree?: Array<{ path: string; type: string }> };
+    const result = (data.tree ?? []).filter((e) => e.type === 'blob').map((e) => e.path);
+    fetchTreeCache.set(key, result);
+    return result;
+  }


+    const files: Array<{ path: string; content: string }> = [];
+    let truncatedAny = false;
+    for (const candidates of targets) {
+      if (files.length >= limits.maxSourceFiles) break;
+      for (const path of candidates) {
+        const fetched = await provider.fetchFileAtRef(path, ref).catch(() => null);
+        if (fetched) {
+          files.push({ path: fetched.path, content: fetched.content });
+          if (fetched.truncated) truncatedAny = true;
+          break; // first resolving candidate for this target
+        }
+      }
+    }
+
+    if (files.length === 0) return empty;
+
+    const blocks = files.map((f) => {
+      const clipped =
+        f.content.length > limits.sourceFileChars
+          ? f.content.slice(0, limits.sourceFileChars) + '\n[truncated]'
+          : f.content;
+      return `### ${f.path}\n\`\`\`\n${numberSourceLines(clipped)}\n\`\`\``;
+    });


+export type PatchValidationStatus =
+  | 'applies'
+  | 'applies-with-offset'
+  | 'stale-file'
+  | 'bad-path'


+function downloadPatch() {
+  const patch = details.value?.suggestedFix?.patch as string | undefined;
+  if (!patch) return;
+  const body = patch.endsWith('\n') ? patch : patch + '\n';
+  const blob = new Blob([body], { type: 'text/x-patch' });
+  const url = URL.createObjectURL(blob);
+  const a = document.createElement('a');
+  a.href = url;
+  a.download = `piwi-diagnosis-${props.diagnosis?.id ?? 'fix'}.patch`;
+  a.click();
+  URL.revokeObjectURL(url);
+}


…robust tree/download - buildSourceFiles: clip source content to sourceFileChars at fetch time and store the clipped text, so patch validation runs against exactly what the model saw; a clip now counts toward coverage.sourceFiles.truncated. - patch.ts / DiagnosisResult.vue: remove the unreachable 'bad-path' status (validatePatch never emitted it) from the union, badge switch and template. - GitHubProvider.fetchTree: fall back to resolving a commit to its tree SHA when the trees endpoint doesn't accept the ref directly. - DiagnosisResult.downloadPatch: append the anchor to the DOM and revoke the object URL on a timeout to avoid cutting the download short in some browsers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GoDzedfU4hJXqFg9fja5XP

PhenX-Pro · 2026-07-04T06:38:06Z

Addressed all four review comments in 64b3b30:

validatePatch runs against content the model didn't see (ai-context.ts) — real contract bug, fixed. buildSourceFiles now clips to sourceFileChars at fetch time and stores the clipped text, so validation runs against exactly what the model saw. A clip now also counts toward coverage.sourceFiles.truncated.
Unreachable 'bad-path' status (patch.ts) — removed from the union, the badge switch, and the template. validatePatch never emitted it (reliable path-existence detection would need a tree lookup, which is out of scope for this PR).
downloadPatch revoke race (DiagnosisResult.vue) — the anchor is now appended to the DOM and the object URL is revoked on a 1s timeout.
fetchTree commit-vs-tree SHA (GitHubProvider.ts) — GitHub's trees endpoint does resolve commit SHAs/refs in practice, but to remove the doubt fetchTree now falls back to resolving the commit to its tree SHA (/commits/{ref} → commit.tree.sha) and retrying if a direct lookup fails. For context: this path is best-effort and only used to normalize absolute test paths for the import closure — suspect changed-file fetching uses repo-relative filenames directly and is unaffected either way, so a tree miss degrades gracefully rather than breaking source fetching.

Typecheck and the 183-test unit suite are green.

Generated by Claude Code

PhenX requested a review from Copilot July 4, 2026 05:58

Copilot started reviewing on behalf of PhenX July 4, 2026 05:58 View session

Copilot AI reviewed Jul 4, 2026

View reviewed changes

PhenX merged commit 279bd39 into main Jul 4, 2026
2 checks passed

PhenX deleted the claude/diagnosis-code-retrieval-plan-iydkd0 branch July 4, 2026 06:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

AI diagnosis: fetch full source files and validate suggested patches#202

AI diagnosis: fetch full source files and validate suggested patches#202
PhenX merged 2 commits into
mainfrom
claude/diagnosis-code-retrieval-plan-iydkd0

PhenX commented Jul 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

PhenX-Pro commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Uh oh!

Conversation

PhenX commented Jul 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

PhenX-Pro commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants