arman1o1 blog
AI/ML Skills Engineering

From Prompts to Playbooks: Distilling Anthropic’s Guide to Agent Skills

• ☕ 7 min read

Yesterday, I crawled through Anthropic's new guide on building Agent Skills. It is a brilliant blueprint, but let's be honest - at dozens of pages of PDF, it is packed with enterprise fluff. Most of us just want to know how to build skills that actually execute without choking.

Having built several skills for Claude and local IDE agents like Codex, I wanted to distill the absolute essentials. This is the practical handbook: the architectural principles, the routing mechanics, and the verification patterns you actually need to build portable, production-ready AI playbooks.

The Core Mental Model: Progressive Disclosure

The standard way people build AI workflows is by writing massive system prompts. They pack rules, error codes, guidelines, and formatting templates into a single block of text. This is a terrible design. It bloats the context window, kills latency, and makes the model erratic.

Anthropic's skill architecture solves this using progressive disclosure. Instead of dumping everything into context on day one, you separate the information into three distinct, layered boundaries:

Layer Components Included When It Loads Primary Purpose
1. Metadata (Router) YAML frontmatter (name, description) Always active Decides if the skill is needed
2. Playbook (Instructions) SKILL.md core body Loaded on demand Guides the model's step-by-step logic
3. Depth (References & Scripts) references/, scripts/, assets/ Accessed only if needed Provides granular context or deterministic checks

By splitting your skill this way, the agent stays lightweight. It only carries the heavy context (like brand style guides or complex API tables) when it is actively working on a task that requires them.

The Router: Crafting Bulletproof Frontmatter

Most "my skill won't trigger" bugs are actually routing failures. Frontmatter acts as your skill's entry point. If it is too generic, the agent will ignore it. If it is too broad, it will hijack unrelated queries.

A production-ready skill router needs to clearly define its decision boundaries using three elements:

  1. Specific Action Verb: What concrete action does the skill perform? (e.g., "Generates a sprint plan" rather than "Helps with sprints").
  2. Positive Triggers: Explicit user phrases, terminology, or file formats that should activate the skill.
  3. Negative Triggers: Concrete scenarios where the skill must not load. This shapes the decision boundary and prevents overtriggering.
---
name: notion-sprint-planner
description: Plans active developer sprints using backlog tickets and capacity files. Use when the user says "plan a sprint", "kickoff next sprint", or uploads a sprint-backlog.csv. Do NOT use for general chat about Notion or basic task updates.
---

Security Tip: Because frontmatter is loaded directly into the primary system prompt, never include angle brackets (< or >) in your YAML block. They cause parser confusion and open the door to prompt injection vulnerabilities. Keep your frontmatter metadata clean and strictly alphanumeric.

Playbooks vs. Scripts: The Validation Boundary

One of the hardest lessons in AI engineering is finding the boundary between natural language instructions and code. If you try to enforce strict compliance patterns using only English instructions (e.g., "Ensure the output JSON never contains missing fields"), the model will eventually slip.

We solve this by separating responsibilities. Use language for reasoning and adaptability, but use code for validation and deterministic processing.

flowchart LR L["Language Playbook (SKILL.md)"] -->|"Reasoning & Logic"| M["Model Loop"] M -->|"Brittle Verification"| S["Deterministic Scripts (validate.py)"] S -->|"Error Feedback"| M

Instead of burying a 200-line formatting ruleset in your playbook, write a validation script. The playbook simply directs the model to execute the script and self-correct based on the output:

Instructions:
1. Generate the initial report payload.
2. Execute the validation script: scripts/validate_report.py --input report.json
3. If the script outputs any error details:
   - Read the error trace.
   - Refactor the payload to address the specific failure.
   - Re-run steps 2 and 3 until the validation passes.

This "generate-check-fix" loop is incredibly robust. It turns brittle natural language guidelines into stable, deterministic software constraints.

The 3-Part Eval Suite

You cannot build operational skills based on vibes. If you edit a skill's description to fix one overtriggering issue, you will likely break three other workflows. You need a simple, repeatable testing loop.

A minimal, highly effective eval suite consists of three parts:

  • Triggering Tests: A bank of 10-20 user queries. Include obvious hits ("kickoff sprint"), paraphrases ("let's plan"), and decoy triggers ("how do I use Notion?") to ensure the router works correctly.
  • Functional Scenario Tests: Mocked inputs designed to run the skill end-to-end. Verify that the agent calls the correct sequence of tools and successfully hits the script validation gates.
  • Performance Comparison: Compare your runs against a baseline system without the skill. Specifically track three metrics: message turns, failed tool calls, and token consumption (fewer turns and zero failures is the target).

Keep these test cases stored alongside your skill. Every time you tighten your playbook instructions, run the test set to ensure you did not introduce regression.

Going Local: IDEs and Desktop Agent Apps

While Anthropic’s guide is built around Claude’s cloud ecosystem, the exact same architectural principles apply on your local machine. Today, almost every major AI lab and IDE environment supports local agentic apps (like Claude Desktop, Cursor, Copilot, or Codex) with their own ways of packaging custom instructions.

Depending on your tooling, the folder structures differ slightly, but the routing concept remains identical:

  • IDE Configurations: Environments like Cursor use files like .cursorrules, while Copilot relies on .github/copilot-instructions.md to act as the primary workspace playbook.
  • Local Agent Folders: Local developer environments like Codex look for structured skill directories in your user profile (e.g., ~/.codex/skills/git-helper/SKILL.md).
  • Desktop Run Loops: Local runtimes (like Claude Desktop) parse configured MCP servers to import tools and descriptions dynamically.

The transition is straightforward: create your workspace-local playbook, define a clean description to act as the router, and write clear, actionable steps for local execution (like linking to local scripts for brittle validations). Whether you are deployed in the cloud or executing inside your local editor, treating your prompts as structured, portable playbooks is the only way to build agentic systems that scale.

References