Design an AI customer support agent

30-second answer

• A two-stage intent classifier (fast embedding model for 80% of traffic, LLM fallback for ambiguous cases) routes customer messages to resolution workflows in under 50ms average, compared to 200ms+ for a single LLM call on every message.
• Multi-signal escalation scoring (intent confidence, retrieval quality, customer sentiment, policy rules) replaces naive "I don't know" detection and reduces unnecessary escalations from 45% to 15-20%.
• RAG over help articles, past ticket resolutions, and live account data gives the response generator grounded context, keeping hallucination rates below 2% on factual queries.
• At scale (100K tickets/day), AI auto-resolution at $0.05-0.15 per ticket versus $15-25 per human-handled ticket saves over $1M per week at 70% deflection.
• The key design point: the AI agent that "resolves" a ticket is meaningless unless it can actually execute actions (process refunds, reset passwords, update orders). Response generation without tool calling is a fancy FAQ page.

Requirements and assumptions

Functional requirements

• Customers can submit support messages via chat (web, mobile) and receive an AI-generated response within 5 seconds.
• The system classifies each customer message into an intent category (order status, refund request, password reset, billing question, general FAQ, complaint) and routes to the appropriate resolution workflow.
• The AI agent retrieves relevant context (help articles, past ticket resolutions, customer account data) to generate grounded, accurate responses.
• The system executes actions on behalf of the customer (process refunds, reset passwords, check order status) when the resolved intent maps to an executable workflow.
• When the AI agent's confidence is below threshold or the issue type requires human judgment (billing disputes, account recovery, emotionally distressed customers), the system escalates to a human agent with full conversation context.
• Human agent resolutions are logged and fed back into the system to improve classifier accuracy and response quality over time.

Non-functional requirements

• First response latency: P95 under 5 seconds for AI-resolved tickets (versus 4-8 hour average for human queue).
• Throughput: 100,000 tickets per day (approximately 1,200 messages per minute at peak).
• Auto-resolution rate (deflection): 70%+ of tickets resolved without human involvement.
• Response accuracy: less than 2% hallucination rate on factual queries (order status, pricing, policy information).
• Cost per AI-resolved ticket: $0.05-0.15, compared to $15-25 per human-handled ticket.
• Customer satisfaction (CSAT): AI-resolved tickets score 4.0+ out of 5.0 (human baseline is 4.5/5).

The hardest engineering problem here: knowing when the AI should stop trying. An overconfident agent that gives wrong answers damages customer trust faster than a slow human queue. An underconfident agent that escalates everything defeats the purpose of automation. The escalation threshold is an important calibration point, and it depends on signals from multiple systems (classifier, retriever, sentiment analyzer) that each have their own failure modes.

Assumptions

• The AI may read account data through typed tools, but write actions such as refunds or account recovery require authorization, idempotency, and business-rule checks outside the model.
• Product and policy content has an owner and a freshness signal. The response generator must be able to say that it lacks a verified answer instead of filling a gap from model memory.
• A human queue is a first-class dependency. Escalations carry the transcript, evidence, attempted actions, and reason so the customer does not have to repeat the issue.

5-minute approach

Model the support agent as a guarded workflow, not a single prompt: classify the issue, retrieve current evidence, decide whether an action is allowed, answer or escalate, and record the outcome.

• Separate read-only retrieval from write-capable tools and require a typed action plan before execution.
• Combine intent confidence, retrieval quality, sentiment, policy, and contact history for escalation; keep high-risk intents conservative.
• Keep conversation context bounded with a recent window and summary, while grounding factual claims in current help content or account data.
• Log the decision and human outcome so thresholds, prompts, and knowledge sources can be improved without guessing.

The entities and endpoints below make those workflow boundaries explicit.

Core entities

SupportTicket

• ticket_id, customer_id, channel (web_chat, mobile, email), status (open, ai_resolved, escalated, human_resolved), created_at, resolved_at, resolution_type, csat_score

CustomerMessage

• message_id, ticket_id, content, sender (customer, ai_agent, human_agent), timestamp, intent_classification, confidence_score

IntentClassification

• classification_id, message_id, intent (order_status, refund, password_reset, billing, faq, complaint, unknown), confidence, classifier_used (embedding, llm), sub_intent, requires_action (boolean)

RetrievalContext

• retrieval_id, message_id, sources (array of doc IDs), relevance_scores, retrieval_latency_ms, context_token_count

EscalationDecision

• decision_id, ticket_id, escalated (boolean), score, signals (intent_confidence, retrieval_quality, sentiment, policy_override), reason, decided_at

ActionExecution

• action_id, ticket_id, action_type (refund, password_reset, order_lookup, account_update), parameters, status (pending, executed, failed), executed_at

ResolutionFeedback

• feedback_id, ticket_id, original_ai_response, human_correction, correction_type (wrong_intent, wrong_answer, missing_context, tone_issue), used_for_training (boolean)

API design

POST /v1/support/message - submit a customer message and receive an AI response

Request: {
  "ticket_id": "tkt_abc123",
  "customer_id": "cust_456",
  "message": "I ordered a laptop 3 days ago and it still says processing. Can I get a refund?",
  "channel": "web_chat",
  "metadata": {
    "page_url": "/orders/ORD-789",
    "previous_messages": 2
  }
}
Response: {
  "message_id": "msg_def789",
  "response": "I can see your order ORD-789 is currently in processing. It typically ships within 3-5 business days. Would you like me to process a refund, or would you prefer to wait for shipping?",
  "intent": "multi_intent:order_status+refund",
  "confidence": 0.87,
  "actions_available": ["process_refund", "check_shipping_eta"],
  "escalated": false,
  "sources": ["help-article-102", "order-ORD-789"],
  "response_latency_ms": 1840
}

The response includes available actions so the frontend can render action buttons. Multi-intent detection (order status + refund) avoids forcing customers to ask one thing at a time.

POST /v1/support/action - execute a customer action (refund, password reset, etc.)

Request: {
  "ticket_id": "tkt_abc123",
  "action_type": "process_refund",
  "parameters": {
    "order_id": "ORD-789",
    "reason": "customer_request",
    "amount": "full"
  },
  "confirmed_by_customer": true
}
Response: {
  "action_id": "act_ghi012",
  "status": "executed",
  "result": "Refund of $1,299.00 initiated for order ORD-789. Expect 5-7 business days for processing.",
  "executed_at": "2026-04-11T14:32:00Z"
}

POST /v1/support/escalate - manually escalate or system-triggered escalation to human agent

Request: {
  "ticket_id": "tkt_abc123",
  "reason": "customer_requested_human",
  "conversation_summary": "Customer asked about order ORD-789 refund. AI offered refund but customer has additional billing concerns.",
  "priority": "medium",
  "escalation_signals": {
    "intent_confidence": 0.62,
    "sentiment_score": -0.4,
    "retrieval_quality": 0.55,
    "policy_override": false
  }
}
Response: {
  "escalation_id": "esc_jkl345",
  "assigned_agent": "agent_sarah",
  "estimated_wait_minutes": 12,
  "queue_position": 4,
  "handoff_context_included": true
}

GET /v1/support/analytics - deflection metrics and system performance

Response: {
  "period": "2026-04-11",
  "total_tickets": 98420,
  "ai_resolved": 69094,
  "escalated": 19684,
  "pending": 9642,
  "deflection_rate": 0.702,
  "avg_ai_response_ms": 1840,
  "avg_csat_ai": 4.21,
  "avg_csat_human": 4.48,
  "cost_savings_usd": 1036410,
  "top_intents": [
    { "intent": "order_status", "count": 31200, "auto_resolved_pct": 0.92 },
    { "intent": "refund", "count": 18600, "auto_resolved_pct": 0.71 },
    { "intent": "password_reset", "count": 14100, "auto_resolved_pct": 0.97 }
  ]
}

45-minute interview approach

Keep the conversation about designing a safe support system. The interesting part is the boundary between automated resolution and human judgment, not the choice of one particular model.

• 0–5 min — Clarify scope: identify channels, supported intents, read versus write actions, identity requirements, languages, and which issues must reach a person.
• 5–10 min — Requirements and estimates: state ticket volume, response SLO, deflection and CSAT goals, hallucination tolerance, escalation SLA, and cost target.
• 10–18 min — Interfaces and data model: define message, intent, evidence, tool call, escalation ticket, and feedback records; include idempotency for actions.
• 18–28 min — High-level design: draw ingress and PII handling, classification, RAG/account tools, response generation, action validation, streaming, and human handoff.
• 28–38 min — Deep dive: explain multi-intent routing, context management, confidence scoring, action guardrails, and learning from human resolutions.
• 38–42 min — Scale and operations: cover queueing, rate limits, knowledge freshness, tracing, per-intent quality, escalation load, and cost allocation.
• 42–45 min — Trade-offs and close: state conservative defaults, failure behavior, rollback paths, and the follow-up that would most change the architecture.

High-level design and data flow

The system splits into two pipelines: the real-time resolution pipeline (handles incoming messages) and the offline learning pipeline (improves the system from human corrections).

The real-time pipeline receives a customer message, classifies intent, retrieves context from multiple sources, generates a response, scores confidence, and either sends the response or escalates. The entire flow completes in under 5 seconds end-to-end. A single monolithic LLM call ("just send everything to GPT-4o") may work for a demo, but it becomes difficult to control for cost, latency, grounding, and actions as volume grows.

The offline pipeline collects human agent corrections, retrains the intent classifier weekly, updates the RAG knowledge base with new resolutions, and A/B tests updated models before full rollout. This feedback flywheel is what separates a static FAQ bot from a system that actually improves.

The architecture separates read-heavy operations (intent classification, retrieval) from write-heavy operations (action execution, ticket logging) so they scale independently. The vector database handles semantic search over help articles and past resolutions, while PostgreSQL stores structured ticket data and customer records.

Draw this diagram in two passes: first the happy path from message to AI response, then the escalation branch and learning loop. The incremental drawing keeps the core flow legible.