Agentic AI demos beautifully and breaks quietly in production. This guide covers how to ship an agent that survives past the demo: bound the scope to a narrow task, name a human owner for every action class, and stand up an evaluation set before you let the agent touch anything. It walks a procurement-intake agent from an 82 percent demo to a governed 96 percent production system, with an autonomy-tier table, a launch sequence, the failure modes that surface only at scale, and a checklist for the first two weeks.
Why agents that demo well fail in production
An agent that chains tool calls and takes actions is a different risk object than a model that returns text. The demo shows the happy path on ten curated inputs, and it looks like magic. Production sends it 4,000 messy inputs a week, several of which fall outside anything the demo covered, and the agent acts on all of them with equal confidence. The uncomfortable property of a well-built agent is that it fails silently and plausibly: it does not throw an error on an out-of-scope input, it produces a confident, wrong action that looks exactly like a right one. That is why the volume matters, because even a small error rate across thousands of weekly actions becomes dozens of confident mistakes. The gap between an 82 percent demo pass rate and a system you can trust unattended is not a better model. It is bounded scope, named ownership of every action, and an evaluation harness that catches regressions before users do. A model upgrade might lift the demo pass rate a few points, but it does nothing about the fact that nobody scoped what happens when the agent meets an input outside its training, nobody decided which actions it may take without a human, and nobody built the test set that would catch a regression before it reached production. Those three gaps are organizational, not technical, and no model release closes them.
The failure pattern is consistent. A team wires an agent with broad autonomy because the framework makes it easy, ships it on the strength of a good demo, and then spends the next quarter firefighting edge cases nobody scoped. Deployment discipline inverts that order. You decide first what the agent is allowed to do without a human, you draw the boundary tight, and you widen it only as the evaluation set proves each expansion is safe. The agent earns autonomy; it is not granted it by default.
The cost asymmetry is what makes this discipline non-optional. An agent that takes 4,000 actions a week at 96 percent reliability still takes roughly 160 wrong actions in that week, and if any of those actions moves money, changes a record, or emails a customer, the downside of a single error can dwarf the value of the other 3,840. Reliability that is acceptable for a text suggestion is unacceptable for an unattended action, which is why the same model can be safe in one tier and reckless in another. The tier system exists precisely to sort actions by the cost of their errors, not by how often the demo got them right.
Assign every action to an autonomy tier
List every distinct action the agent can take and place each into one of four tiers. The tier decides whether a human is in the loop and what evidence the agent must attach. Start almost everything in tier 2 or 3, then promote an action to full autonomy only after 200 or more clean production instances and an evaluation score above your bar.
| Tier | Action class | Human role | Eval gate to promote | Owner |
|---|---|---|---|---|
| 0 Suggest | Drafts output, takes no action | Human executes everything | Baseline, no gate | Product owner |
| 1 Approve | Proposes an action, waits for a click | Human approves each action | 90 percent approval rate over 100 items | Workflow lead |
| 2 Notify | Acts, then alerts a human who can reverse it | Human audits and can roll back | 95 percent eval pass, under 2 percent rollback | Operations lead |
| 3 Autonomous | Acts unattended within hard limits | Human reviews the weekly exception report | 98 percent eval pass over 200 live items | Accountable executive |
Take a procurement-intake agent that read supplier request emails, extracted line items, matched them to catalog SKUs, and opened a purchase requisition. In the demo it passed 82 percent of 50 cases. In production, ambiguous unit-of-measure fields and unlisted suppliers dropped it below trust. The team split its actions: extraction and SKU matching moved to tier 2 with a rollback window, but opening a requisition above 5,000 dollars stayed at tier 1 requiring a buyer's click. They built a 300-case evaluation set from real historical emails, labeled by two buyers. Extraction cleared 96 percent on that set within three weeks and was promoted to notify-and-reverse. The high-value requisition path stayed human-approved, which is exactly where the residual 4 percent of errors would have been expensive.
The lesson is that a single agent should almost never sit at a single autonomy level. Splitting its actions let the team ship the safe, high-volume work unattended while keeping a human on the small number of high-consequence actions, so the agent delivered most of its value in week four without exposing the firm to a costly automated mistake. Six weeks later, once the requisition path had accumulated 200 clean approvals under human review and cleared the tier-3 gate on the eval set, that path too was promoted, but only on evidence and only under a hard dollar cap. Autonomy widened action by action, never all at once.
The deployment sequence
- Week 1: enumerate every action the agent can take and assign each a tier. If an action cannot be named and tiered, the agent should not be able to perform it. Draw hard limits, such as a dollar cap or a record-count cap, for anything above tier 1.
- Week 2: build an evaluation set of at least 200 real historical cases with human labels, covering the edge cases the demo skipped. This set, not the demo, is the thing you ship against.
- Weeks 3 to 4: run the agent in shadow mode against live traffic with all actions at tier 0 or 1, and compare its proposals to what humans actually did. Log every disagreement.
- Weeks 5 to 6: promote only the actions that clear their eval gate, one tier at a time, and keep high-consequence actions human-approved regardless of score.
- Ongoing: re-run the evaluation set on every prompt or model change and treat any drop below the tier gate as a blocking regression, not a note for later. Version the eval set alongside the agent so you can always answer what it was tested against on the day an action went wrong.
Failure modes that only surface at scale
- Granting broad autonomy because the framework defaults to it. The agent acts on inputs nobody scoped. Fix: every action starts at tier 0 or 1 and is promoted only on evidence.
- Shipping against the demo set. Ten curated inputs hide the long tail that breaks production. Fix: build a 200-plus case eval set from real historical data before launch.
- No rollback path for tier 2 actions. When the agent acts and there is no way to reverse it, a single bad action becomes an incident. Fix: no notify-tier action without a defined reversal within the audit window.
- Silent prompt edits. Someone tweaks the prompt, nobody re-runs the eval set, and quality drifts for weeks. Fix: gate every prompt change behind a passing eval run.
- One owner for the whole agent. When everything rolls up to a single name, high-consequence actions get the same casual review as trivial ones. Fix: assign owners per action tier, with an accountable executive on autonomous actions.
Apply this in the first two weeks
- List every action the agent can take and assign each one of the four autonomy tiers.
- Set hard limits, such as a dollar cap or record-count cap, on any action above tier 1.
- Build a 200-plus case evaluation set from real historical inputs with human labels.
- Run the agent in shadow mode against live traffic and log every disagreement with the human decision.
- Name an owner per tier and require a passing eval run before any prompt or model change ships.