Support deflection with an audit lane (L5)

The steps

1. 01

   Cluster 90 days of tickets and classify the top 20 intents three ways

   Tool: Zendesk

   Export 90 days of tickets. Cluster into intents by first-contact reason, then rank by volume. Mark each of the top 20 as one of: answerable-from-docs, needs-account-data, needs-a-human-decision. Only answerable-from-docs is in scope for phase one. A password reset explanation is in scope; "why was I charged twice" needs account data; "can I get a refund" needs a decision. Mixing these is how the first deployment fails publicly. • Owner: Support lead • Tool options: Zendesk, Freshdesk or Intercom reporting, exported to Google Sheets • Pitfall or what breaks: mixing needs-account-data or needs-a-human-decision intents into phase one scope. • Definition of done: a ranked intent list exists with the three-way classification and a volume percentage on each row.

2. 02

   Rewrite the top five deflectable articles answer-shaped

   Tool: Zendesk Guide

   Five articles, rewritten to a strict shape: one question per article, the answer inside the first two sentences, steps numbered, current screenshots, a visible last-reviewed date, and a named owner. Delete competing older articles rather than leaving them to be retrieved. A retrieval system is only as good as the corpus; if your docs are 18 months stale, the agent will be confidently wrong at scale, and it will be wrong in front of customers. • Owner: Support plus product marketing • Tool options: your help center • Pitfall or what breaks: leaving duplicate older articles in place instead of deleting them, so the wrong one gets retrieved. • Definition of done: five articles are rewritten and dated, each answers its question in the first 40 words, and duplicates are removed.

3. 03

   Deploy on those five intents only, with a visible escape hatch

   Tool: Intercom Fin

   Scope the agent to five intents. Any other intent hands to a human immediately with the full transcript attached, so the customer never repeats themselves. "Talk to a person" stays visible in the agent's first message, always, not buried after three failed attempts. Set the refusal behavior explicitly: when confidence is low, the agent says it does not know and routes, rather than approximating. Do not over-assume people want the automated version; give them the choice and most will still take the fast answer. • Owner: Support lead plus GTM engineer • Tool options: Intercom Fin, Ada, Sierra, Decagon, or your desk's native AI • Pitfall or what breaks: burying the human handoff option after several failed attempts instead of keeping it visible from the first message. • Definition of done: live on five intents, human handoff is one click, and three real customers have tested the escape path end to end.

4. 04

   Stand up the audit lane on day one, not month three

   Tool: Google Sheets

   A human reads 20 random AI conversations every day. Score each: correct, incomplete, wrong, or should-have-escalated. Anything scored wrong triggers a same-day fix to the article or the scope, logged with what changed. Twenty a day is roughly 45 minutes; that 45 minutes is the entire difference between an autonomous system and an ungoverned one talking to your customers. • Owner: Support QA • Tool options: a daily random sample export plus a written rubric, results tracked in Looker, Tableau or Power BI • Pitfall or what breaks: dropping the audit lane in month two when it feels like overhead, which is exactly when you stop being able to tell it is failing. • Definition of done: 10 consecutive days of 20-conversation reviews are logged with an attached fix log.

5. 05

   Publish containment and satisfaction on the same dashboard

   Tool: Looker

   Containment rate never ships alone. Four numbers, one screen: containment rate, CSAT for AI-handled conversations, reopen rate, and escalation rate. High containment with falling CSAT means you built a wall, not a service, and reopen rate is where that shows up first. • Owner: Support lead • Tool options: one dashboard in Looker, Tableau or Power BI • Pitfall or what breaks: letting containment become a target on its own, which invites hiding the escape hatch to hit it. • Definition of done: all four numbers are on one dashboard reviewed weekly by CS leadership.

6. 06

   Expand one intent every two weeks, gated on quality

   Add the next intent only after two consecutive weeks of stable CSAT and a wrong-answer rate under 2%. One intent per two weeks, maximum. If the gate fails, you fix content rather than adding scope. Slow is the design, and it is what keeps this from becoming the thing customers complain about publicly. • Owner: Support lead • Tool options: the intent list from step 1 • Pitfall or what breaks: adding scope instead of fixing content when the quality gate fails. • Definition of done: intents six through ten are live under the same gate, or expansion is deliberately paused with a written reason.

7. 07

   Instrument the result

   Track containment rate, always beside AI-conversation CSAT and reopen rate. Never containment alone. • Where it breaks: stale docs produce invented answers and support volume goes up because customers now have to correct a robot before reaching a person; containment becomes a target and the escape hatch quietly gets buried to hit it, with churn showing up two quarters later; or the audit lane gets dropped in month two when it feels like overhead, exactly when it starts failing. • Visual guidance: a decision tree from inbound message to one of three ends, answered, escalated, flagged in the audit lane, with the audit lane drawn as a loop back into the knowledge base. That loop is the design.

Tools in this playbook

• Zendesk
• Zendesk Guide
• Intercom Fin
• Google Sheets
• Looker

Next playbooks

• CSM call summaries via Gainsight + AI (L2)
• AI-generated QBR decks (L3)
• Intercom Fin AI for support deflection (L3)
• Support deflection with AI search (L3)