When an AI voice agent should not replace a human: sensitive calls
Being able to handle a call does not mean an agent should handle it. This guide explains why the decision is made at design time, the five groups of calls that should stay with people, a four-level escalation matrix, what to do at night and at weekends when no one is available, and how to check the matrix.
The short answer
An AI voice agent should not replace a human operator on calls where the cost of an error is high, the decision requires judgement, or the caller is in an emotional state. These fall into five groups: medical conditions and emergencies, financial transactions and advice, formal complaints and disputes, sensitive personal situations (bereavement, illness, debt), and decisions that need authority the scenario does not grant.
That does not mean the agent never touches these calls. Often its role is narrow and clear: recognise the call, take the minimum information needed, and get it to a person without losing time. The key question is not "can the agent do this?" but "should the agent do this?"
Why this decision is made at design time
Handoff rules that fire during a call — the caller insists, the agent fails to understand twice — matter, but they are reactions. For some call types a reaction is too late: in a medical emergency the caller should not be handed over after two minutes of talking to an agent. So which call types are never given to the agent has to be decided in advance, before the scenario is written.
It is also a question of accountability. Who owns which answer, and who fixes it when it is wrong — these are management questions, not technical ones, and they need leadership, legal and quality teams involved.
Group one: health and emergencies
On clinic and pharmacy lines, the agent can answer questions such as opening hours, address and booking requests. But describing symptoms, medication doses or "is this normal?" is medical advice and lies outside the agent's boundary. When emergency signs are heard — shortness of breath, fainting, severe pain — the agent must not try to diagnose; it should direct the caller to the emergency number straight away or hand over to the person on duty.
Group two: financial transactions and advice
In banks, insurers and lenders, general information — branch hours, required documents, the status of an application — may suit an agent. Transferring money, changing a limit, unblocking a card, or advice such as "which loan is best for me" requires either strict authentication or the judgement of a licensed person. On these calls the agent's role is to recognise the request and route it to a safe channel.
Group three: complaints and disputes
In a formal complaint, a compensation claim or a contract dispute, the caller wants to be heard, and any promise made may carry legal weight. The agent can say "I have recorded your complaint", but it must not promise compensation, talk about fault or make a decision. The best design: write a short summary of the complaint, tell the caller honestly who will contact them and when, and get the call or the note to the responsible person.
Group four: sensitive personal situations
Closing an account after a relative's death, postponing a payment because of serious illness, financial hardship — on these calls, taking information is easy, but finding the right tone is hard. An agent's mechanical politeness can sound cold here. When such topics are recognised from key phrases, the call should go straight to a trained member of staff.
The escalation matrix
For every call type, assign the agent one of four roles. This table becomes the basis for the scenario, the knowledge base and the handoff rules.
- Full answerThe agent completes the call itself: opening hours, address, general price, list of documents.
- Collect and hand overThe agent collects name, number and reason, then hands over or writes a note for a callback: bookings, technical problems.
- Recognise and hand over immediatelyThe agent only recognises the call type and hands over without collecting details: complaints, sensitive situations.
- Not given to the agentA rule sends the call straight to a person: emergency line, VIP customers, legal notices.
Illustrative example: an insurance line
Not a real customer case. An insurer mapped its call types like this: branch address, document list and claim status — "full answer"; a quote request for a new policy — "collect and hand over" (name, number, vehicle or property type); dissatisfaction with a claim payout and calls from the scene of an incident — "recognise and hand over immediately"; a separate number for legal notices — "not given to the agent".
In the pilot's first month, the review showed that a call from the scene was sometimes recognised as a "new policy" request, because the caller first read out the policy number. Phrases such as "accident", "incident" and "I'm on the road right now" were added to the scenario, and those calls were routed straight to the person on duty.
What to do when no one is available
The weakest point of the matrix is nights and weekends: a call you marked "recognise and hand over" arrives at 11 p.m. and there is no one to hand it to. In that case the agent must be honest: say when operators work, record the request, and say who will call back in the morning. For emergencies, the company must define in advance an on-call number or the official emergency service to direct callers to.
How to check the matrix
- Every month, review a sample of calls at the "not given to the agent" and "hand over immediately" levels: did the agent recognise them correctly?
- Separate out calls the agent answered that later turned into complaints
- Ask operators which handed-over calls could have stayed with the agent, and which should have reached them sooner
- Review the matrix whenever a new product or rule is introduced
Limits
Recognition by key phrases is not always accurate: a caller may describe a sensitive situation indirectly, and audio quality can distort words. So the matrix should be written cautiously, and handoff should be preferred in doubtful cases. This article is not medical, legal or financial advice; check sector requirements with specialists.
How these boundaries are set up in Vexvon
In Vexvon AI Call Center the agent's behaviour is written in the scenario: what it says, which topic it stays on and what information it collects. When a question gets difficult or the caller insists, the call passes to a live member of staff. The "not given to the agent" level is built with routing rules: for example, a VIP number list goes straight to a department and never reaches the agent; the rule can be checked in the simulator before it goes live. Every call's transcript stays in the panel for checking the matrix.
In-call handoff triggers are covered in AI-to-human call handoff; the selection matrix shows which call types to start with.
First step
Take your list of call types and write one of the four levels next to each. Confirm the "not given to the agent" and "hand over immediately" rows together with your legal and quality leads. More articles are in the voice agent strategy section; to discuss your matrix, get in touch.