BPO quality assurance: how the client should check agents

Short answer

To check an outsourced contact center's quality, the client company needs four things: criteria written with the BPO and with agreed definitions, regular joint calibration, an independent audit where the client — not the BPO — selects the sample, and a rule for resolving disputes over results. Without them, the "quality 92%" in the BPO's report cannot be verified — it was measured with the BPO's own criteria on calls the BPO chose.

The goal is not to catch the BPO out. It is for both sides to look at the same call and reach the same result. If a gap remains, that is a criteria problem, not a contract problem, and it is solved together.

Why the BPO's report is not enough

• The BPO wrote the criteria and may interpret them differently from your brand
• The BPO chooses which calls are reviewed — the selection may not be random
• The BPO's QA team scores its own agents — there is a conflict of interest
• The report shows an average; critical errors and their type are invisible
• When the form changes, old and new figures are not comparable but appear on one chart

Step 1: write the criteria together

The most important work happens before the contract, or at the latest at the start. The following are agreed:

1. 1FormsSeparate forms by call type — sales, support, complaints. AWS's evaluation-form documentation also describes different forms per business unit and queue, and NiCE recommends tailoring the scorecard to call type and team.
2. 2DefinitionsWhat yes, no and not applicable mean for each criterion — with examples.
3. 3Critical errorsWhich errors are counted separately from the average and reported to the client immediately.
4. 4Brand requirementsGreeting, prohibited phrases, language choice — rules specific to your brand.
5. 5VersionThe form's date and version; changes only with both sides' agreement.

Step 2: who picks the sample

• The client selects the audit sample, or the selection rule is agreed in advance — random, split by call type and shift
• The BPO does not "pick and send" calls; the contract gives the client access to recordings at an agreed volume
• Complaint-related calls are a separate, targeted sample and are not blended into the overall quality figure
• Calls from newly hired agents are checked more in their first weeks

Step 3: joint calibration

Calibration means both sides scoring the same calls independently and comparing results. The rhythm below is an illustrative example:

1. 1PreparationOnce a month, 5–10 calls are chosen: different call types, one or two difficult cases.
2. 2Independent scoringThe client's and the BPO's QA specialists score without seeing each other's answers.
3. 3ComparisonCriteria with different answers are discussed: is the cause the criterion's wording, or interpretation?
4. 4DecisionThe criterion's definition is tightened, an example is added, a new version is written.

Illustrative example: 92 and 78

The example below is illustrative. The BPO's monthly report shows an average score of 92. The client independently selects 30 calls from the same month, scores them with its own team, and gets an average of 78.

Analysing the gap usually turns up three causes: the BPO counts "partial" as "yes", uses "not applicable" differently, or its internal sample has more simple calls. None of these on its own means anyone lied — but each requires fixing the criterion definitions and the sampling rule.

Comparing the BPO's teams

• Teams are compared only within the same call type and the same form version
• The number of calls is shown for each team — a percentage from few calls is read with caution
• Differences in shift, language and customer mix are taken into account
• If there are several BPOs, comparing them is meaningless without the same criteria, the same sampling rule and joint calibration

Dispute and appeal rules

1. 1Appeal windowHow many days the BPO has to dispute an audit result.
2. 2EvidenceAn appeal names the specific criterion and a transcript line or audio excerpt.
3. 3Second reviewA representative of each side listens to the disputed call together.
4. 4RecordThe decision and reasoning are written down; if a criterion's definition changes, a new version is issued.

A monthly governance rhythm

• Weekly: the list of critical errors and the response to them
• Monthly: the independent audit result, joint calibration, criteria changes
• Quarterly: trends, coaching plans, form version updates, a review of contract metrics
• Every meeting's outcome is recorded in writing and checked at the next one

Quality clauses in the contract

Most quality checks turn into disputes after the contract is signed, because nothing was written down in advance. The clauses below are an illustrative list; draft the wording with a lawyer.

1. 1Access to recordingsAt what volume, for how long and through which channel the client receives call recordings.
2. 2Criteria as an annexThe evaluation forms and their versions are an annex to the contract; the change procedure is written down.
3. 3Audit and calibrationMonthly audit volume, calibration frequency and participants.
4. 4AppealsThe appeal window, the evidence required and the second-review procedure.
5. 5Data protectionHow long recordings are kept, restricted access, and deletion when the contract ends.

Typical mistakes

• Accepting a figure the BPO produced with its own criteria without checking it
• Letting the BPO choose the sample
• Calibrating only when a problem arises
• Continuing to compare with old figures after the form has changed
• Tying penalties and bonuses to an AI score without an appeal process

Limits

• Transferring call recordings to the client is subject to the contract, consent and data protection requirements
• In calibration the human answer is not treated as flawless either; the goal is agreement, not a winner
• A quality score does not reflect the BPO's workload, shift planning or system problems
• An AI result should not be used as the sole evidence in a contract dispute

What Vexvon Audio Analyzer offers

• The client can upload call recordings shared by the BPO to its own account and analyse them against its own call standard — recordings and results are stored separately per company
• Call standards are versioned: each call keeps the text and version it was judged by
• Each step comes with a met, partial, missed or not-applicable status, a comment and transcript lines — usable directly in calibration and disputes
• No shared BPO–client portal is offered; sharing results is your process

More: Vexvon Audio Analyzer. Internal calibration is covered in AI call scoring calibration, fair team comparison in fair agent performance comparison, and building an evaluation form in call center scorecard.

First step

Ask the BPO for 30 calls from last month, selected by your rule, and score them with your own team. Compare the result with the BPO's report and make the three criteria with the biggest gap the subject of your first joint calibration. This article belongs to the agent QA by industry section. To try it on your own recordings, get in touch.