Auditing agent skills and instruction files

Audit an agent skill by first resolving the instruction files that apply to a concrete target, then validating the package graph and reviewing risky capability combinations separately from writing quality. Keep scanning read-only, preserve the exact evidence behind each finding, and make repair a distinct consented action. Sources: Skills Doctor rule catalog, Agent Skills specification.

AI agent systems

A repeatable way to inspect SKILL.md, AGENTS.md, referenced scripts, and local overrides for structural drift, unsafe capability combinations, and instructions an agent cannot actually execute.

12 min readPublished August 7, 2026Last reviewed August 11, 2026

Tested with: Skills Doctor 0.6.2
Validation scope: Non-interactive flags, JSON finding fields, package traversal, and consent boundaries were checked against the linked package version; no automated repair was executed.

This guide answers

How do you audit an Agent Skill or SKILL.md file?
What security problems can hide in coding-agent instruction files?
How do you detect conflicting AGENTS.md and global skill instructions?

When to use this / when not to use this

Use this when

You need a repeatable pre-publish or CI review of SKILL.md, AGENTS.md, scripts, and referenced assets.
You need to distinguish deterministic package defects from capability risks and editorial quality.

Do not use this when

You only need to review ordinary application code; use the repository's code and security gates instead.
You expect a score alone to prove a skill is safe or effective without reviewing its evidence and behavior.

Resolve the instructions the agent will actually see

Repositories can contain root and nested AGENTS.md files, project skills, user-level skills, ecosystem-specific copies, and generated or cached packages. An audit that scans one familiar directory can miss the file that wins at runtime. Begin by discovering candidate roots and computing the effective instruction chain for a concrete target path. Sources: Codex AGENTS.md documentation, Agent Skills specification.

Preserve provenance for every resolved instruction: absolute or workspace-relative path, scope, content digest, source ecosystem, and precedence. When two skills share a name or one local file shadows a global copy, report the relationship. Ambiguity should be visible rather than resolved with an undocumented guess.

Resolution checks

Root and nested instruction files are found without crossing the requested workspace boundary.
Project-local, user-level, cached, and disabled skill roots are classified separately.
Duplicate names include content digests and the precedence rule that selects one copy.
Symlinks are resolved and checked for escapes before content is read.
The audit records which concrete file or route the effective chain applies to.

Validate structure and references deterministically

Parse frontmatter instead of searching it with regular expressions. Validate required fields, trigger descriptions, supported metadata, and package naming. Then walk references from SKILL.md into scripts, templates, assets, and additional guidance, checking that every target exists and stays inside the package unless an external dependency is explicitly allowed. Sources: Agent Skills package and frontmatter rules.

A good rule emits a stable identifier, severity, exact location, evidence, explanation, and recovery. Keep repeated occurrences available, but do not let twenty copies of the same weak phrase distort the package score twenty times. Scoring and finding frequency answer different questions.

High-value deterministic rules

Missing, malformed, or unsupported frontmatter fields.
Broken relative references and case-sensitive path mismatches.
Instruction steps that name a tool or script the package does not provide.
Executable files with no invocation path or referenced scripts without execute intent.
Contradictory trigger, prerequisite, output, or verification requirements.

System diagramInstruction audit pipeline

Discovery establishes effective scope before structural rules and security correlations feed a separate repair handoff.

Source: Skills Doctor security specification

01DiscoverRoots, nesting, duplicates, shadowing
02ResolveEffective instruction chain
03ValidateFrontmatter, files, references
04CorrelateCapabilities and security incidents
05RepairConsent-gated handoff + re-scan

Model capability combinations before judging intent

Instruction text is executable influence. A skill that can read secrets, call the network, execute downloaded content, persist changes, or suppress approval has a larger security surface even if each sentence looks ordinary. Detect these capabilities separately, then group combinations into review incidents with the exact evidence that triggered them. Sources: MITRE CWE-73: External Control of File Name or Path, MITRE CWE-59: Improper Link Resolution.

Static analysis cannot prove malicious intent. Phrase findings as observable capability and missing control: reads credential files and sends HTTP requests without a domain allowlist, for example. That gives a reviewer something testable and avoids presenting a heuristic as a vulnerability verdict.

Capabilities worth correlating

Secret or credential access with outbound network transfer.
Remote download followed by shell or interpreter execution.
Writes to global instruction roots or persistence locations.
Approval bypass language combined with destructive or external-state actions.
Prompt override language that tells the agent to ignore higher-priority instructions.

Separate audit, triage, and repair

The scan should be non-destructive and deterministic enough for CI. Triage can group signals, add local usage evidence, or choose a bounded subset for repair. Repair may invoke another coding agent, but only after the user sees the target files, findings, prompt, and command that will run. Sources: JSON Schema structural validation specification.

After repair, rerun the same scanner and compare finding identifiers. Do not accept a higher aggregate score as proof that the important issue disappeared. A repair can improve formatting while leaving the dangerous capability combination intact, or delete enough context that the skill no longer works.

CI contract

Machine-readable output has a schema version and deterministic ordering.
Threshold flags distinguish quality severity from security priority.
Ambiguous roots fail with candidate paths instead of choosing silently.
The scanner performs no network calls or agent launches unless explicitly requested.
A post-repair scan proves the selected finding identifiers are resolved.

A failure taxonomy for skill audits

The categories below come from implementing Skills Doctor’s deterministic scanner. They keep authoring defects, capability risks, packaging errors, and unsafe repair behavior from collapsing into one unhelpful score.

Failure taxonomyDerived from Skills Doctor rule catalog

A failure taxonomy for skill audits
Failure class	Trigger	Boundary	Disposition
Contract	Invalid frontmatter, vague triggers, or no executable workflow	SKILL.md	Block distribution until the package contract is valid.
Reference integrity	Missing file, path escape, hidden executable, or broken progressive disclosure	Package tree	Resolve paths inside the package and inventory every referenced asset.
Capability risk	Secret access, network egress, remote execution, persistence, or approval bypass	Instruction + scripts	Keep raw findings; group related signals only in the human incident view.
Evaluation gap	No realistic prompts, assertions, expected outputs, or baseline	evals	Warn or block according to skill complexity and declared behavior.
Repair regression	The score improves while referenced workflows or scripts break	consent-gated handoff	Re-scan the same roots and run the package’s native verification gate.

Implementation examples

Concrete commands and data shapes you can adapt.

Run a non-interactive audit suitable for CIshell

npx skills-doctor@latest \
  --yes \
  --json \
  --fail-on warning \
  --fail-on-security P1 \
  --min-score 95 > skills-audit.json

The shape of an actionable findingjson

{
  "rule_id": "references/missing-file",
  "severity": "error",
  "file": "skills/deploy/SKILL.md",
  "line": 38,
  "evidence": "scripts/release.sh",
  "message": "Referenced script does not exist",
  "recovery": "Add the script or remove the invocation step"
}

Decision log

The choices that shape the design—and what each choice costs.

Keep rule logic in the CLI
The packaged Agent Skill remains a thin discovery layer and does not duplicate scanner behavior in prose.
Tradeoff: The CLI becomes the versioned dependency that every integration must install or invoke.
Group security signals without hiding raw findings
Reviewers see the meaningful capability chain and can still inspect each deterministic trigger.
Tradeoff: Incident correlation needs careful identifiers so grouping changes do not break CI baselines.
Require confirmation before agent repair
Scanning files does not imply permission to edit them or launch another process with their contents.
Tradeoff: Fully automated repair pipelines must add their own explicit approval policy.

Failure cases

What breaks, how it presents, and the recovery boundary.

The scanner audits the wrong copy: Signal
A local skill shadows a global skill with the same name, or multiple roots match in CI.; Response
Report every candidate, its digest and precedence; require an explicit root when resolution remains ambiguous.
A referenced script escapes the package: Signal
The normalized path or resolved symlink lands outside the approved skill root.; Response
Reject the reference unless the policy explicitly permits that external dependency.
Repair improves the score but breaks the workflow: Signal
Selected findings disappear while required steps, tools, or output guarantees are also removed.; Response
Re-run structural tests and task-level evals; compare behavior, not only the aggregate score.

Repositories and primary references

Read the implementation, specifications, and tool documentation behind the article.

Inspect the systems that ground this guide in implementation work.

Last updated: August 11, 2026