Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,37 @@ Automatically applies safe fixes:

Only fixes marked `auto: true` are applied. Permission changes require human review.

### External Rule Packs (`--rule-pack`)

Run community or private detection rules alongside the built-ins, without recompiling:

```bash
agentshield scan --rule-pack ./my-pack.json
agentshield scan --rule-pack ./pack-a.json --rule-pack ./pack-b.json # repeatable
```

A pack is a JSON file validated with the same fail-closed approach as `--policy` (bad JSON, schema violations, duplicate ids, or an uncompilable regex abort the scan):

```json
{
"version": 1,
"name": "my-pack",
"rules": [
{
"id": "tool-poisoning-001",
"name": "Tool description poisoning",
"description": "Hidden instruction in a tool description",
"severity": "high",
"category": "injection",
"patterns": ["ignore (?:all )?previous instructions"],
"fileTypes": ["agent-md", "claude-md"]
}
]
}
```

Each pattern is a JS regex run against file content; `fileTypes` is optional and scopes a rule to specific config types. External findings count toward the overall grade. Anyone with a pack in this shape can plug in.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Document that external regexes are normalized to global matching.

The docs describe pattern execution but omit that the loader enforces global matching, which affects match cardinality and expected findings.

Suggested wording tweak
-Each pattern is a JS regex run against file content; `fileTypes` is optional and scopes a rule to specific config types. External findings count toward the overall grade. Anyone with a pack in this shape can plug in.
+Each pattern is a JS regex run against file content, and AgentShield normalizes patterns to global matching so every occurrence can be reported; `fileTypes` is optional and scopes a rule to specific config types. External findings count toward the overall grade. Anyone with a pack in this shape can plug in.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Each pattern is a JS regex run against file content; `fileTypes` is optional and scopes a rule to specific config types. External findings count toward the overall grade. Anyone with a pack in this shape can plug in.
Each pattern is a JS regex run against file content, and AgentShield normalizes patterns to global matching so every occurrence can be reported; `fileTypes` is optional and scopes a rule to specific config types. External findings count toward the overall grade. Anyone with a pack in this shape can plug in.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` at line 391, The README.md documentation at line 391 describes how
patterns execute as JS regexes but omits that the loader enforces global
matching on external regexes. Add documentation clarifying that external regex
patterns are normalized to global matching by the loader, and explain how this
normalization affects match cardinality and the number of expected findings,
ensuring users understand the behavior of their patterns when using external
regex configurations.


### Secure Init (`agentshield init`)

Generates a hardened `.claude/` directory with scoped permissions, safety hooks, and security best practices. Existing files are never overwritten.
Expand Down Expand Up @@ -616,6 +647,7 @@ agentshield scan [options] Scan configuration directory
--gate Fail on new critical/high findings or score drop
--supply-chain Verify MCP package provenance and risk
--supply-chain-online Include npm registry metadata
--rule-pack <path> Load an external JSON rule pack (repeatable)
--policy <path> Validate against an organization policy
--evidence-pack <dir> Write portable evidence bundle
--remediation-plan <path> Write stable-fingerprint JSON remediation plan
Expand Down
29 changes: 28 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { resolve } from "node:path";
import { dirname, join } from "node:path";
import { existsSync, writeFileSync, appendFileSync, mkdirSync } from "node:fs";
import { scan } from "./scanner/index.js";
import { loadRulePacks } from "./rules/external.js";
import { calculateScore } from "./reporter/score.js";
import { renderTerminalReport } from "./reporter/terminal.js";
import { renderJsonReport, renderMarkdownReport } from "./reporter/json.js";
Expand Down Expand Up @@ -288,6 +289,12 @@ program
.option("--gate", "Fail if new critical/high findings or score drops (use with --baseline)", false)
.option("--supply-chain", "Verify MCP npm packages against known-bad list and typosquatting", false)
.option("--supply-chain-online", "Also query npm registry for metadata (requires network)", false)
.option(
"--rule-pack <path>",
"Load an external JSON rule pack and run it alongside built-in rules (repeatable)",
(value: string, previous: string[]) => [...previous, value],
[] as string[]
)
.option("--policy <path>", "Validate against an organization policy file")
.option("--evidence-pack <dir>", "Write a portable evidence bundle for audits and security reviews")
.option("--remediation-plan <path>", "Write a stable-fingerprint JSON remediation plan")
Expand All @@ -312,9 +319,29 @@ program
const enableTaint = options.deep || options.taint;
const enableOpus = options.deep || options.opus;

// ── External rule packs (--rule-pack) ────────────────────
const rulePackPaths: string[] = options.rulePack ?? [];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Use immutable array typing for collected rule-pack paths.

The new local collection is typed as mutable string[] in src/; prefer ReadonlyArray<string> for policy compliance.

Suggested fix
-    const rulePackPaths: string[] = options.rulePack ?? [];
+    const rulePackPaths: ReadonlyArray<string> = options.rulePack ?? [];

As per coding guidelines, src/**/*.ts: “All arrays must be typed as ReadonlyArray … No mutation, no any types, and no console.log in src/ files”.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const rulePackPaths: string[] = options.rulePack ?? [];
const rulePackPaths: ReadonlyArray<string> = options.rulePack ?? [];
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/index.ts` at line 323, The variable rulePackPaths is currently typed as a
mutable string[] but should be typed as ReadonlyArray<string> to comply with
coding guidelines requiring immutable array typing in src/ files. Change the
type annotation of rulePackPaths from string[] to ReadonlyArray<string> while
keeping the initialization logic the same.

Source: Coding guidelines

let extraRules = undefined;
if (rulePackPaths.length > 0) {
const loaded = loadRulePacks(rulePackPaths);
if (!loaded.success) {
console.error(`Error: ${loaded.error}`);
process.exit(1);
}
extraRules = loaded.rules;
for (const pack of loaded.packs) {
process.stderr.write(` Loaded ${pack.ruleCount} external rules from ${pack.name}\n`);
logger.log({
level: "info",
phase: "init",
message: `Loaded ${pack.ruleCount} external rules from ${pack.name}`,
});
}
}

// ── Phase 1: Static rule-based scan ──────────────────────
logger.log({ level: "info", phase: "static", message: "Running static analysis" });
const result = scan(targetPath);
const result = scan(targetPath, { extraRules });

// Filter by severity
const filteredResult = {
Expand Down
184 changes: 184 additions & 0 deletions src/rules/external.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
import { existsSync, readFileSync } from "node:fs";
import { z } from "zod";
import type { ConfigFile, Finding, FindingCategory, Rule, Severity } from "../types.js";

/**
* External rule-pack loader (`--rule-pack`).
*
* AgentShield's built-in detectors are declarative (id/severity/category/pattern
* rows). This loads the same shape from an external JSON pack so scans can run
* community detection rules (e.g. the MIT Agent Threat Rules pack) without
* recompiling the binary. Generic: anyone with a pack in this shape can plug in.
*
* Security note: patterns are compiled to RegExp and run against file content.
* Packs are loaded only when the operator explicitly passes `--rule-pack`, so
* they are trusted input, but we still fail closed on invalid JSON, bad schema,
* or any pattern that does not compile, and we cap findings per rule per file to
* bound pathological packs. ReDoS-hardening of arbitrary external patterns is a
* known v1 limitation (documented for the loader, same as any regex linter).
*/

const MAX_FINDINGS_PER_RULE_PER_FILE = 200;

const SeveritySchema = z.enum(["critical", "high", "medium", "low", "info"]);

const CategorySchema = z.enum([
"secrets",
"permissions",
"hooks",
"mcp",
"skills",
"agents",
"injection",
"exposure",
"exfiltration",
"misconfiguration",
]);

const RulePackEntrySchema = z.object({
id: z.string().min(1),
name: z.string().min(1),
description: z.string().optional(),
severity: SeveritySchema,
category: CategorySchema,
patterns: z.array(z.string().min(1)).min(1),
flags: z.string().optional(),
fileTypes: z.array(z.string()).optional(),
});

export const RulePackSchema = z.object({
version: z.literal(1),
name: z.string().optional(),
description: z.string().optional(),
rules: z.array(RulePackEntrySchema).min(1),
});

export type RulePackEntry = z.infer<typeof RulePackEntrySchema>;
export type RulePack = z.infer<typeof RulePackSchema>;

export interface LoadRulePackResult {
readonly success: boolean;
readonly rules?: ReadonlyArray<Rule>;
readonly meta?: { readonly name: string; readonly ruleCount: number };
readonly error?: string;
}

function findLineNumber(content: string, matchIndex: number): number {
return content.substring(0, matchIndex).split("\n").length;
}

function findAllMatches(content: string, pattern: RegExp): Array<RegExpMatchArray> {
return [...content.matchAll(pattern)];
}
Comment on lines +70 to +72

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift

Refactor new collection logic to immutable ReadonlyArray style.

This module introduces mutable array typings and mutation (Array<...>, Finding[], Rule[], .push(...)) in src/**/*.ts, which conflicts with repo immutability rules.

As per coding guidelines, src/**/*.ts: “All arrays must be typed as ReadonlyArray … No mutation, no any types, and no console.log in src/ files”.

Also applies to: 97-99, 139-140, 173-174

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rules/external.ts` around lines 70 - 72, The findAllMatches function
returns a mutable Array type instead of an immutable ReadonlyArray type. Change
the return type of findAllMatches from Array<RegExpMatchArray> to
ReadonlyArray<RegExpMatchArray>. Additionally, review the code at lines 97-99,
139-140, and 173-174 and apply the same immutability pattern to any other
mutable array type annotations (such as Finding[] and Rule[]) and remove any
array mutations like .push() calls, replacing them with immutable alternatives
like spread operators or concat.

Source: Coding guidelines


/** Ensure the global flag is present (matchAll requires it) and dedupe flags. */
function normalizeFlags(flags: string | undefined): string {
const set = new Set((flags ?? "").split(""));
set.add("g");
return [...set].join("");
}

function compileEntryPatterns(entry: RulePackEntry): ReadonlyArray<RegExp> {
const flags = normalizeFlags(entry.flags);
return entry.patterns.map((source) => new RegExp(source, flags));
}

function entryToRule(entry: RulePackEntry, compiled: ReadonlyArray<RegExp>): Rule {
const fileTypes = entry.fileTypes ? new Set(entry.fileTypes) : null;
return {
id: `external-${entry.id}`,
name: entry.name,
description: entry.description ?? entry.name,
severity: entry.severity as Severity,
category: entry.category as FindingCategory,
check(file: ConfigFile): ReadonlyArray<Finding> {
if (fileTypes && !fileTypes.has(file.type)) return [];

const findings: Finding[] = [];
let seq = 0;
for (const pattern of compiled) {
for (const match of findAllMatches(file.content, pattern)) {
findings.push({
id: `external-${entry.id}-${seq}`,
severity: entry.severity as Severity,
category: entry.category as FindingCategory,
title: entry.name,
description: `${entry.description ?? entry.name} (external rule ${entry.id}).`,
file: file.path,
line: findLineNumber(file.content, match.index ?? 0),
evidence: match[0].substring(0, 100),
});
seq += 1;
if (findings.length >= MAX_FINDINGS_PER_RULE_PER_FILE) return findings;
Comment on lines +97 to +112

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Non-unique finding IDs across files

seq resets to 0 at the start of every check() call, so every file that matches the same rule produces findings with the same IDs (external-foo-0, external-foo-1, …). When a rule fires on two different files, both scans emit external-foo-0 — a collision that breaks baseline comparison and remediation-plan fingerprinting, since those features rely on stable, unique IDs to correlate findings across runs. The fix is to incorporate the file path into the ID (e.g. external-${entry.id}-${file.path}-${seq} or a hash thereof) so the ID is globally unique within a single scan.

}
}
return findings;
},
};
}

/**
* Load and validate an external rule pack from a JSON file path, returning
* ready-to-run Rule objects. Fails closed on missing file, invalid JSON, schema
* mismatch, duplicate ids, or any pattern that does not compile.
*/
export function loadRulePack(rulePackPath: string): LoadRulePackResult {
if (!existsSync(rulePackPath)) {
return { success: false, error: `Rule pack not found: ${rulePackPath}` };
}

let pack: RulePack;
try {
pack = RulePackSchema.parse(JSON.parse(readFileSync(rulePackPath, "utf-8")));
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
return { success: false, error: `Invalid rule pack ${rulePackPath}: ${message}` };
}

const seenIds = new Set<string>();
const rules: Rule[] = [];
for (const entry of pack.rules) {
if (seenIds.has(entry.id)) {
return { success: false, error: `Duplicate rule id in pack: ${entry.id}` };
}
seenIds.add(entry.id);

let compiled: ReadonlyArray<RegExp>;
try {
compiled = compileEntryPatterns(entry);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
return { success: false, error: `Rule "${entry.id}" has an invalid pattern: ${message}` };
}
rules.push(entryToRule(entry, compiled));
}

return {
success: true,
rules,
meta: { name: pack.name ?? rulePackPath, ruleCount: rules.length },
};
}

/**
* Load several rule packs and concatenate their rules. Returns the first error
* encountered, or all rules plus per-pack metadata on success.
*/
export function loadRulePacks(paths: ReadonlyArray<string>): {
readonly success: boolean;
readonly rules: ReadonlyArray<Rule>;
readonly packs: ReadonlyArray<{ name: string; ruleCount: number }>;
readonly error?: string;
} {
const rules: Rule[] = [];
const packs: Array<{ name: string; ruleCount: number }> = [];
for (const path of paths) {
const result = loadRulePack(path);
if (!result.success || !result.rules || !result.meta) {
return { success: false, rules: [], packs: [], error: result.error };
}
rules.push(...result.rules);
packs.push(result.meta);
}
Comment on lines +167 to +182

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Enforce rule-ID uniqueness across all loaded packs.

loadRulePacks only preserves per-pack uniqueness. Two packs can both define entry.id = "x", producing identical runtime rule.id (external-x) and overlapping finding ID namespaces.

Suggested fix
 export function loadRulePacks(paths: ReadonlyArray<string>): {
   readonly success: boolean;
   readonly rules: ReadonlyArray<Rule>;
   readonly packs: ReadonlyArray<{ name: string; ruleCount: number }>;
   readonly error?: string;
 } {
   const rules: Rule[] = [];
   const packs: Array<{ name: string; ruleCount: number }> = [];
+  const seenRuleIds = new Set<string>();

   for (const path of paths) {
     const result = loadRulePack(path);
     if (!result.success || !result.rules || !result.meta) {
       return { success: false, rules: [], packs: [], error: result.error };
     }
+    for (const rule of result.rules) {
+      if (seenRuleIds.has(rule.id)) {
+        return { success: false, rules: [], packs: [], error: `Duplicate rule id across packs: ${rule.id}` };
+      }
+      seenRuleIds.add(rule.id);
+    }
     rules.push(...result.rules);
     packs.push(result.meta);
   }
   return { success: true, rules, packs };
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rules/external.ts` around lines 167 - 182, The loadRulePacks function
currently only ensures rule ID uniqueness within individual packs but not across
all loaded packs. Two different packs can define rules with identical IDs,
resulting in identical runtime rule.id values and overlapping finding ID
namespaces. After the loop that collects all rules from all packs, add a
validation step to check for duplicate rule IDs across the entire rules array.
If duplicates are found, return an error response indicating which rule IDs are
duplicated rather than returning success with the overlapping rules.

return { success: true, rules, packs };
Comment on lines +167 to +183

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 No cross-pack duplicate rule ID check

loadRulePack correctly rejects duplicate rule IDs within a single pack, but loadRulePacks never checks whether the same rule ID appears in multiple packs. If pack-a.json and pack-b.json both declare id: "tool-001", the loader silently produces two rules with Rule.id = "external-tool-001" and finding IDs that are entirely indistinguishable (external-tool-001-0, etc.). The PR description says the loader "fails closed on … duplicate id", but that guarantee only holds within a pack. A global seenRuleIds set checked after merging each pack's rules would close this gap.

}
10 changes: 8 additions & 2 deletions src/scanner/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,18 @@ export interface ScanResult {
readonly harnessAdapters?: HarnessAdapterSummary;
}

export interface ScanOptions {
/** Extra rules loaded from external rule packs (`--rule-pack`). */
readonly extraRules?: ReadonlyArray<Rule>;
}

/**
* Main scanner: discovers config files and runs all rules against them.
* External rule packs (if any) run alongside the built-in rules.
*/
export function scan(targetPath: string): ScanResult {
export function scan(targetPath: string, options: ScanOptions = {}): ScanResult {
const target = discoverConfigFiles(targetPath);
const rules = getBuiltinRules();
const rules = [...getBuiltinRules(), ...(options.extraRules ?? [])];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Type the merged rules list as ReadonlyArray<Rule>.

Line 35 currently infers a mutable array type. Keep the scanner’s effective rule set immutable at the type level.

Suggested fix
-  const rules = [...getBuiltinRules(), ...(options.extraRules ?? [])];
+  const rules: ReadonlyArray<Rule> = [...getBuiltinRules(), ...(options.extraRules ?? [])];

As per coding guidelines, src/**/*.ts: “All arrays must be typed as ReadonlyArray … No mutation, no any types, and no console.log in src/ files”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/scanner/index.ts` at line 35, The rules variable on line 35 that merges
the builtin rules with extra rules is being inferred as a mutable array type
instead of a ReadonlyArray. Add an explicit type annotation to the rules
variable declaration to type it as ReadonlyArray<Rule>. This ensures the merged
rules list is immutable at the type level, aligning with the coding guideline
that all arrays in src/ files must be typed as ReadonlyArray.

Source: Coding guidelines

const findings = runRules(target.files, rules, target.path);
const skillHealth = analyzeSkillHealth(target.files);
const harnessAdapters = detectHarnessAdapters(targetPath);
Expand Down
Loading
Loading