The easiest way to build a bad support agent is to make it try to answer everything. The easiest way to build a support agent nobody trusts is the opposite — escalate everything and let humans do all the real work anyway. The hard, valuable middle ground is an agent that knows precisely where its competence ends.

Designing the escalation boundary first

Before we wrote a single response template, we mapped every category of incoming request and rated it on two axes: how well-defined is the correct answer, and how much does getting it wrong cost. Simple account questions with a clear right answer got full autonomy. Billing disputes and anything involving a customer's frustration or a policy exception got routed straight to a human, immediately, with zero delay from the agent trying anyway.

The three-tier system we landed on

  • Tier 1 — Full autonomy. Status checks, password resets, order lookups. High volume, low ambiguity, cheap to get wrong and cheap to fix if it happens.
  • Tier 2 — Autonomous with a policy envelope. Refunds under a set amount, delivery date changes, plan downgrades — the agent acts, but only within pre-approved limits, and logs every action for audit.
  • Tier 3 — Immediate human handoff. Anything involving anger, legal language, safety, or a request outside every defined policy. The agent doesn't attempt an answer here — it collects context and hands off in under a second.

Why "knowing when to stop" was the hard engineering problem

Classifying a request into the right tier reliably — before responding, not after a failed attempt — took more iteration than the response generation itself. We trained the classification step on real historical tickets, not synthetic examples, because the edge cases that matter are exactly the messy, real ones a synthetic dataset smooths over.

The agent's most important skill isn't answering well. It's recognizing, in under a second, which of the three tiers a request belongs to.

The outcome

68% of all tickets resolved end-to-end with no human touch, running continuously across every time zone the customer base spans. Just as important: the escalations that did reach a human arrived with full context and a tier-3 flag, so the humans spent their time on judgment calls instead of triage.