Building LLM Agents (the practical way)

Agents aren't chatbots. They're systems that run workflows end-to-end: they decide what to do next, call tools, recover from hiccups, and stop when the job is done.

Based on the PDF A Practical Guide to Building Agents.

TL;DR

• An agent is a workflow runner: an LLM in a loop, with tools and instructions, operating inside guardrails.
• Build agents when the work needs judgment, your rules are getting brittle, or the data is messy (unstructured).
• Start with a single agent and expand its toolset; go multi-agent only when complexity forces you there.
• "Orchestration" is just the control logic: runs, exits, handoffs, and how you manage state.
• Tools are product surface area. Treat them like real APIs: stable contracts, tests, docs, and explicit side effects.
• Good instructions look like SOPs: numbered steps, explicit actions, and branches for edge cases.
• Guardrails are defense-in-depth: relevance + safety + PII + moderation + tool risk checks + deterministic limits.
• Human intervention isn't a failure mode; it's part of the design for early reliability and high-risk actions.

Table of contents

• What is an agent (and what isn't)
• When should you build an agent
• Agent design foundations: model + tools + instructions
  • Models: pick capability, then optimize
  • Tools: extend the agent into your systems
  • Instructions: reduce ambiguity on purpose
• Orchestration: how the whole thing runs
  • Single-agent systems (the default)
  • Prompt templates (complexity without prompt sprawl)
  • When to split into multiple agents
  • Multi-agent patterns: manager vs decentralized
• Guardrails: layered defense + human intervention
  • Types of guardrails (a practical menu)
  • How to build guardrails (the iteration loop)
  • Optimistic execution + tripwires (the deep cut)
  • Plan for human intervention
• Putting it all together: a build sequence that works
• Conclusion + next steps
• Starter checklist (screenshot this)

What is an agent (and what isn't)

At a high level:

• A workflow is a sequence of steps that must be executed to reach a goal (resolve a support issue, book a reservation, ship a code change, generate a report).
• An agent is a system that can run that workflow on the user's behalf with a high degree of independence.

The easiest way to make this concrete is to draw the boundary around "control of execution."

Agents do this

An agent:

1. Uses an LLM to manage execution and make decisions:
   • It decides what step comes next.
   • It recognizes when the workflow is complete.
   • It can correct itself when it notices a mistake.
   • If the workflow can't proceed safely, it can stop and hand control back to the user.
2. Has access to tools to interact with external systems:
   • Tools gather context (read data).
   • Tools take actions (write data, trigger side effects).
   • The agent chooses tools dynamically based on current state, while staying inside guardrails.

Not everything with an LLM is an agent

These are useful, but they're not agents in the workflow sense:

• A simple chatbot that answers questions but doesn't control execution.
• A single-turn LLM call (prompt in, text out) with no looping or tool use.
• A classifier (sentiment, intent, relevance) that doesn't decide the next step in a workflow.

The mental model you'll keep coming back to

An agent is basically a while-loop with judgment.

That's the whole game: you build a loop that keeps going until an exit condition, and you give it the right tools, instructions, and guardrails so that the loop is reliable.

When should you build an agent

Agents shine when deterministic automation starts to feel like you're fighting reality.

In plain terms, prioritize workflows that:

1. Need complex decision-making (nuanced judgment, exceptions, context-sensitive calls).
2. Have rules that are painful to maintain (the rule engine is now a fragile monster).
3. Depend heavily on unstructured data (natural language, documents, messy user inputs).

A quick scoring table

Score each category from 0 to 2. If you're mostly 0s, you're probably better off with conventional automation.

The "fraud investigator" analogy (why agents are different)

A traditional rules engine feels like a checklist: "If X and Y, then flag."

An LLM-based agent is closer to a seasoned investigator: it can weigh context, notice subtle patterns, and make a call even when no single rule was violated. That kind of reasoning is exactly what helps in messy real-world workflows where the "right answer" depends on context.

When not to build an agent (yet)

If the workflow is stable, deterministic, and you can express it cleanly with normal code, do that first. Agents add a new kind of complexity:

• Non-determinism (the same input can produce different steps).
• New failure modes (tool misuse, prompt injection, weird edge cases).
• The need for guardrails and evaluation.

Use agents where that tradeoff buys you something real.

Agent design foundations: model + tools + instructions

At the most fundamental level, an agent has three building blocks:

• Model: the LLM that powers reasoning and decisions
• Tools: functions/APIs the agent can use to fetch context and take actions
• Instructions: explicit guidelines that define behavior and boundaries

You can picture this as a triangle. If one corner is weak, the system feels unreliable:

• Great model + weak tools = the agent can't actually do anything.
• Great tools + weak instructions = the agent can do damage quickly.
• Great instructions + weak model = the agent follows poorly, misunderstands, or fails to handle nuance.

Let's break down each corner.

Models: pick capability, then optimize

Different models have different strengths and tradeoffs: capability, latency, and cost. In agent workflows, you might use different models for different steps (a cheap model for retrieval/triage, a stronger model for hard decisions).

But the trap is optimizing too early.

The baseline-first approach (it works for a reason)

1. Prototype with the most capable model for every step.
2. Establish a performance baseline (how often does the agent do the right thing).
3. Swap in smaller/faster models step-by-step and see where quality holds.

This keeps you from prematurely limiting the agent and gives you a clean way to diagnose failures: "This step fails only when we downgrade the model" is actionable.

Model selection checklist

Use this as a quick gut-check:

• I have an eval set (even a small one) that represents real user cases.
• I picked a model that meets the accuracy target on that eval set.
• I measured latency across the full workflow, not just one call.
• I only downgraded models where the eval results stayed acceptable.
• I know which steps are "cheap" (retrieval, intent) vs "expensive" (high-stakes decisions).

Deep cut: in agents, model choice isn't just "smarter is better." A too-smart model with unclear tools and weak guardrails can be more dangerous than a smaller model in a constrained sandbox. Capability magnifies whatever system you built.

Tools: extend the agent into your systems

Tools are how your agent touches reality.

In modern systems, tools usually mean calling APIs. For legacy systems without good APIs, agents can also interact through UIs (web/app) using computer-use style capabilities, basically "doing what a human would do" in the interface.

Either way: treat tools like you would any production API surface area.

Standardize tool definitions (future you will thank you)

As tool count grows, ad-hoc tools become a maintenance nightmare. Standardization buys you:

• Discoverability (people know what exists)
• Reuse (multiple agents can share the same tools)
• Version management (you can evolve tools without breaking everything)
• Fewer duplicate implementations

The three tool types you should name explicitly

Agents generally need three kinds of tools:

1. Data tools: fetch context needed to do the work
   • examples: query a database, read a PDF, search the web, fetch CRM record
2. Action tools: make changes or trigger side effects
   • examples: send email/text, update a record, create a ticket, issue a refund
3. Orchestration tools: agents acting as tools for other agents
   • examples: "refund agent", "research agent", "writing agent"

Naming these categories matters because it forces you to talk about risk. Data tools are usually lower risk; action tools can be high risk; orchestration tools amplify whatever you let them do.

Tool design rules (keep it boring)

• Give tools descriptive names that match user intent (not internal jargon).
• Define parameters clearly; avoid "blob of text" inputs when structure works.
• Document side effects ("this sends an email", "this charges a card").
• Make output types stable and machine-readable (structured objects beat prose).
• Add tests, especially for edge cases and failure states.
• Handle reversibility: can this action be undone, and how.
• Include idempotency keys for actions when possible (prevents duplicate charges/tickets).

Tiny illustrative tool signatures (framework-agnostic)

get_customer_record(customer_id: str) -> CustomerRecord
initiate_refund(order_id: str, amount_usd: float, reason: str) -> RefundResult
create_support_ticket(subject: str, body: str, priority: str) -> Ticket

Notice what's missing: ambiguity. A good tool signature makes it hard for the agent to "almost" do the right thing.

Instructions: reduce ambiguity on purpose

High-quality instructions matter for any LLM app, but agents are special: they run multi-step workflows. Ambiguity compounds across steps.

Good instructions are basically LLM-friendly SOPs.

Instruction best practices (what actually works)

• Use existing documents: SOPs, support scripts, policy docs, knowledge base articles.
• Ask the agent to break down tasks into smaller steps.
• Make each step correspond to a clear action or output.
• Capture edge cases with conditional branches ("if missing X, ask for X").

A reusable instruction-writing template

Use this template when you turn a workflow into agent instructions:

1. Goal: what "done" looks like.
2. Inputs to collect: what info must be gathered from user or tools.
3. Numbered steps: each step maps to one action (ask user, call tool, summarize, etc).
4. Branching rules: what to do when:
   • required info is missing
   • tool errors occur
   • the user asks a new question mid-flow
5. Stop conditions: when to end the run and respond to user.
6. Safety rules: what the agent must never do; when to escalate.

Here's a short example (illustrative, not copied from the PDF):

Routine: "Refund request"

Goal: Either (a) successfully initiate a refund or (b) hand off to a human with a clear summary.

Steps:
1) Ask for order_id if missing.
2) Call get_order_details(order_id).
3) If order is not refundable, explain policy and offer alternatives.
4) If refund amount is > $200 (high risk), ask for confirmation and escalate to human.
5) Otherwise call initiate_refund(order_id, amount, reason).
6) Confirm to the user, including expected timeline.

If any tool fails:
- Retry once.
- If it fails again, stop and escalate with the error details and context.

Using models to generate instructions (a very practical trick)

If you already have strong SOPs/policy docs, you can use advanced models to convert them into clean, numbered agent instructions. The key is to demand "no ambiguity" and "explicit actions" in the output. Think of it as drafting a first pass that humans then review and harden.

Orchestration: how the whole thing runs

Orchestration is the control plane: how your agent executes steps, chooses tools, handles errors, and decides when to stop.

The biggest mistake is jumping straight to a complex multi-agent architecture because it sounds cool. In practice, teams usually succeed faster by starting incremental and keeping the orchestration simple until they have proof they need more.

Single-agent systems (the default)

A single agent can handle a lot. You extend it by adding tools and refining instructions, without introducing coordination overhead.

Every orchestration approach needs a concept of a run: typically a loop that executes until an exit condition is reached. Depending on your framework, the loop may pause at tool calls (yielding control to an orchestrator) or execute tool calls inline. Common exit conditions include:

• The agent emits a tool call (if your runner pauses on tool calls)
• The agent invokes a "final output" tool / returns a structured final result
• The model returns a user-facing message with no tool calls
• An error occurs
• A maximum number of turns is reached

The run loop (Mermaid)

Framework-agnostic pseudocode

state = {messages: [...], context: {...}}

for turn in 1..MAX_TURNS:
  step = model.decide_next_step(state)

  if step.type == "tool_call":
    tool_result = tools.execute(step.tool_name, step.args)
    state = state.with(tool_result)
    continue

  if step.type == "final_message":
    return step.message

return "I couldn't finish safely. Here's what I tried... (handoff)"

If you only internalize one thing: the loop is the product. Everything else (tools, guardrails, instructions) is there to make the loop act sane.

Prompt templates (complexity without prompt sprawl)

As you add use cases, it's tempting to create a different prompt for each scenario. That gets messy fast.

A better strategy is to use a flexible base prompt that accepts policy variables. You maintain one template and vary the inputs.

A small template example

You are a support agent for {{product_name}}.
Tone: {{tone_guidelines}}
Policy: Refunds allowed within {{refund_window_days}} days for {{eligible_reasons}}.

Follow this routine:
1) Identify intent.
2) Gather missing required fields: {{required_fields}}.
3) Use tools to verify eligibility.
4) Take the safest action that resolves the user request.
5) If action is high risk ({{high_risk_thresholds}}), escalate to a human.

This makes maintenance sane: policy changes become variable updates, not prompt rewrites.

When to split into multiple agents

The general guidance is simple: maximize a single agent's capabilities first.

Multiple agents can give nice separation of concerns, but they also introduce:

• coordination overhead
• more failure modes (handoff mistakes, context loss)
• harder evaluation (which agent caused the issue)

Split when you have clear signals that the single-agent approach is hitting a wall. Two practical triggers:

1. Complex logic: the prompt has so many conditionals that it becomes unscalable.
2. Tool overload: not just "too many tools", but too many similar/overlapping tools that confuse selection.

"Go multi-agent?" checklist

• The base prompt is an if/then forest and hard to maintain.
• The agent frequently selects the wrong tool among similar tools.
• Tool naming/params/descriptions are already clear but confusion persists.
• The workflow naturally splits into distinct domains with minimal shared state.
• You can evaluate each sub-agent independently with its own success metrics.

If most of these are false, stay single-agent longer.

Multi-agent patterns: manager vs decentralized

Multi-agent systems can be modeled like graphs: nodes are agents, edges are either tool calls or handoffs.

Two broadly useful patterns:

1) Manager pattern (agents as tools)

One central "manager" agent keeps control, delegates to specialist agents via tool calls, then synthesizes results. This is ideal when you want one consistent user experience and one place where the workflow is controlled.

Even if your real use case isn't translation, the shape generalizes: the manager is the orchestrator; specialists are narrow.

2) Decentralized pattern (handoffs)

In a decentralized pattern, agents are peers. One agent can hand off execution to another agent, transferring the latest conversation state. This is great for triage: route the user to the right specialist and let that specialist take over.

Declarative vs code-first graphs (why this matters)

Some frameworks force you to define the whole workflow graph up front (every branch, loop, and conditional). That can be visually nice, but it gets cumbersome as workflows become dynamic.

A code-first approach lets you express workflow logic with normal programming constructs (if/else, loops, functions) and build orchestration that adapts to state at runtime. For agent systems that evolve quickly, that flexibility is a big deal.

Which pattern should you pick

• Pick manager when you want one agent to control execution and maintain a unified "voice."
• Pick decentralized when triage is the core problem and specialists should fully take over.

You can also hybridize: triage hands off to an agent that acts as a manager for sub-tasks inside its domain.

Guardrails: layered defense + human intervention

Guardrails are how you keep an agent safe, predictable, and on-brand in the real world.

Think of guardrails as defense-in-depth. One guardrail is rarely enough. Multiple specialized guardrails together make the system resilient.

Also: guardrails aren't a substitute for basic software security. You still need strong authentication/authorization, strict access controls, and normal security hygiene.

Types of guardrails (a practical menu)

Here's a concrete menu of guardrails you can layer. The point isn't to implement all of them on day one; it's to know what levers exist.

The key idea from the PDF: rate tools by risk (low/medium/high) based on read vs write access, reversibility, required permissions, and financial impact. Then use those ratings to decide when to pause for checks or escalate.

Guardrails in the workflow (Mermaid)

This shows the shape of layered checks around tool use:

How to build guardrails (the iteration loop)

The PDF's heuristic is practical and worth copying as a mindset (not as text):

1. Start with data privacy and content safety.
2. Add guardrails based on real-world edge cases and failures you observe.
3. Balance security and user experience as you iterate.

In other words: you don't design perfect guardrails in a vacuum. You evolve them from incidents.

A production-friendly way to iterate

• Log guardrail triggers with enough context to debug (without logging sensitive data).
• Track top failure modes weekly (wrong tool selection, off-topic, unsafe input, etc).
• Convert each recurring failure into:
  • a new guardrail
  • a clearer instruction step
  • a tool redesign (better params, more structured outputs)
  • or a new human escalation rule

Optimistic execution + tripwires (the deep cut)

One interesting pattern in the PDF: treat guardrails as first-class and run them concurrently with the agent's work.

The idea is "optimistic execution":

• Let the primary agent proceed to generate outputs or plan actions.
• Run guardrails in parallel (relevance, safety, policy checks).
• If a guardrail detects a violation, it triggers a "tripwire" that interrupts execution (think exception).

Why this is powerful:

• It can keep the user experience fast (you don't block every step on a serial chain of checks).
• It centralizes enforcement: the agent can be helpful by default, but policies still win.

Design detail that matters: your tripwire behavior should be user-friendly.

• If you block, say what to do next ("Try rephrasing", "I can help with X").
• If you escalate, say what will happen ("I'm handing this to a human because...").

Plan for human intervention

Human intervention is a critical safeguard early in deployment. It helps you improve real-world performance without wrecking user trust.

Two triggers are especially important:

1. Exceeding failure thresholds
   • The agent retries too many times.
   • It fails to understand intent after multiple attempts.
   • It hits max turns without progress.
2. High-risk actions
   • Sensitive, irreversible, or high-stakes operations:
     • canceling orders
     • authorizing large refunds
     • making payments

A concrete escalation flow

When escalation triggers, make it deterministic:

1. Agent stops the run.
2. Agent produces a structured handoff packet:
   • user intent summary
   • known facts (tool outputs)
   • what it tried
   • what it needs next (missing fields / permissions)
   • why it escalated (threshold hit, high-risk tool, safety concern)
3. Human reviews and takes action (or messages the user).
4. Agent resumes only if the human explicitly hands control back.

This keeps the UX coherent and gives you clean training/eval data for the next iteration.

Putting it all together: a build sequence that works

This is the "start small, validate, grow" approach distilled into a concrete sequence.

Step 1: Pick one workflow that fits the criteria

Pick a workflow that scores high on at least one of:

• complex decision-making
• brittle rules
• unstructured data

Do not start with your most business-critical, highest-risk workflow unless you already have guardrails and human oversight ready.

Step 2: Define success, failure modes, and an eval baseline

Write down:

• what "done" means (for the user)
• top failure modes you expect (wrong tool, policy violation, hallucinated claim, etc)
• a small set of test cases (10-50 real-ish scenarios is enough to begin)

Then run your first version against that set. You need a baseline before you can improve.

Step 3: List tools (data vs action) and assign risk ratings

Create an inventory:

• Data tools: what context do we need.
• Action tools: what changes do we need to make.

For each action tool, assign:

• risk: low/medium/high
• reversibility: reversible/partially/irreversible
• required permissions
• financial impact

Then decide your policy:

• high-risk tools require confirmation + human review (at first)
• medium-risk tools require confirmation or extra validation
• low-risk tools can run automatically

Step 4: Write instructions as a numbered routine with branches

Use SOP-style instructions:

• numbered steps
• explicit actions per step
• "if missing X, ask for X"
• clear stop conditions

Then do a pass where you remove any ambiguous language like "handle appropriately."

Step 5: Implement a single-agent loop with exit conditions

Keep orchestration simple:

• max turns
• tool call boundaries
• structured output / final response
• error handling and retry rules

Measure end-to-end latency and success rate, not just individual calls.

Step 6: Add layered guardrails + human escalation

Start with:

• deterministic limits (length, allowlists/blocklists where appropriate)
• relevance/safety checks for user input
• tool safeguards (risk ratings)
• a clean human escalation path

Add more based on what breaks in real usage.

Step 7: Iterate with real users, then consider multi-agent

Only after you have:

• stable tool interfaces
• clear instructions
• guardrails that catch common failures
• an eval loop that measures progress

...then decide whether multi-agent is worth it.

Multi-agent is an optimization for complexity. Don't pay that cost until you have to.

Conclusion + next steps

Agents represent a shift in workflow automation: systems that can reason through ambiguity, take actions across tools, and run multi-step tasks with real autonomy.

If you want reliable agents, keep the order straight:

1. Foundations first: capable models + well-defined tools + clear instructions.
2. Orchestration second: start simple, scale patterns as needed.
3. Guardrails always: layered defenses, tool risk controls, and human intervention where it matters.

Start small, validate with real users, and grow capabilities over time. That's how you get real value without shipping chaos.

Starter checklist (screenshot this)

Use this as your "are we ready to ship a v1" list.

Use case

• Workflow needs judgment, messy data, or brittle rules
• Clear definition of "done"
• Known failure modes listed

Model

• Baseline established with a capable model
• Evals exist (even small) and are representative
• Downgrades only where quality holds

Tools

• Tool contracts are explicit and structured
• Side effects documented
• Tests cover success + failure cases
• Action tools have risk ratings and reversibility noted

Instructions

• Numbered routine with explicit actions
• Branches for missing info and common edge cases
• Stop conditions clearly defined

Guardrails + operations

• Input safety/relevance checks in place
• PII/moderation handled where needed
• High-risk actions require confirmation and/or human approval
• Failure thresholds trigger escalation
• Logging supports debugging without leaking sensitive data