A practical guide to AI agents in production

"AI agent" has become a marketing noun. In production, an agent is just software that can choose tools and take steps toward a goal — under constraints you define.

If you ship without those constraints, you get demos that impress and systems that scare your ops team.

What an agent is (and is not)

An agent is useful when:

• The next step depends on context you cannot hardcode
• Multiple tools or systems are involved
• Partial automation still beats pure manual work

An agent is the wrong default when:

• The workflow is a fixed checklist (use a regular automation)
• Mistakes are irreversible and rare (prefer human confirmation)
• You cannot define success in measurable terms

Design for narrow jobs

Production agents win at jobs, not roles.

• Good job: "Triage inbound support email and draft a reply in our voice"
• Bad role: "Be the customer success team"

Narrow jobs are easier to evaluate, cheaper to run, and safer when they fail.

Give tools, not free rein

Every tool should answer three questions:

1. What can it read?
2. What can it write?
3. What is forbidden without approval?

Examples of hard guardrails:

• Can draft emails, cannot send without human click
• Can create CRM notes, cannot delete contacts
• Can refund up to $50, anything higher escalates

Permissions beat prompts. Prompts are suggestions. Permissions are law.

Instrument every run

If you cannot answer these after launch, you are flying blind:

• How many runs succeeded / failed / escalated?
• Where did the agent spend tokens and time?
• Which tool calls failed most often?
• Which outputs did humans rewrite?

Event → Agent run → Tool calls → Result
↓
Logs + metrics
↓
Weekly review → tighten tools / prompts / scope

Shadow mode before autonomy

Run the agent in parallel with the human process. Compare outcomes. Only expand autonomy when:

• Agreement with human decisions is high enough for your risk
• Escalation rate is manageable for the team
• Failure modes are understood and recoverable

Autonomy is earned. It is not a launch checkbox.

Cost and latency are product features

An agent that takes 45 seconds and burns $0.40 per ticket may still win — or it may lose — depending on volume and value. Put numbers next to the architecture:

• Expected runs per day
• Acceptable p95 latency
• Max cost per successful outcome

Otherwise you optimize for cleverness instead of margin.

A sane launch checklist

• Job statement in one sentence
• Tool allowlist with write limits
• Escalation path with owner
• Logging for inputs, tool calls, outputs
• Kill switch
• Shadow period with review cadence
• Success metric tied to business outcome

Ship agents like you ship any other production system: scoped, observable, and reversible.