Choosing call center quality assurance software: demo questions

The short answer

The most reliable way to choose call center quality assurance software is to test it on your own anonymised calls, not on the vendor's prepared examples. A demo has five areas to examine: how accurate the transcript is in your language and your audio quality, how agent and customer are separated, whether you can write the standard with your own criteria, whether every score is explained with evidence, and how your data is stored and deleted.

Price and interface come later. A system that cannot show evidence is no bargain for being cheap — someone will still have to check its results by hand.

Why a demo can mislead

A vendor's demo calls are usually cleanly recorded, held in one language, with clearly separated speakers. Your calls are different: mobile lines, background noise, two languages mixed in one sentence, a call the agent transferred to another department, moments where the customer talks over the agent. A demo can look successful and still produce something entirely different on your calls in the first week.

So split the demo in two: the vendor's example to get to know the product, and your own test set to make the decision.

How to prepare a test set

1. 1Choose for varietySales and support calls, every language you use, stereo and mono recordings, noisy calls, very short calls, calls with a transfer. Do not pick only the "good" ones.
2. 2AnonymiseBefore sending anything to a vendor, remove the parts with customer names, numbers and payment details, or choose calls with consent for testing. Check the legal requirements.
3. 3Write your own answers firstHave an experienced specialist evaluate the same calls against your criteria. That becomes the reference for comparison.
4. 4Give everyone the same standardGive every vendor the same criteria text so the results can be compared.
5. 5Compare by criterionLook at how closely the system matches your reference on each criterion, not at the overall score.

Questions to ask in the demo

Group the questions and get the answers in writing:

• Transcript and language — which languages are supported; what happens when languages mix in one call; are poorly recognised lines shown
• Speaker separation — how it works on stereo and mono recordings; what the system shows when it is not sure; can a wrong separation be corrected
• The standard — can I write criteria in my own words; are there "not applicable" and "partial" statuses; can there be different standards by call type
• Evidence for the score — which quote or transcript location each criterion's result rests on; can I show that to the agent
• Versions — what happens to old scores when I change a criterion; can old calls be re-evaluated against the new criterion, and is that charged extra
• Data governance — where recordings are stored, for how long, who has access, how deletion works; is our data used to train models
• How recordings arrive — file upload, API or a direct telephony connection; which of these works today and which is only on the roadmap
• Pricing model — per minute, per call or per user; are re-analysis and translation charged separately

Red flags

• An accuracy percentage is quoted, but not on which calls, in which language or how it was measured
• The vendor claims to read "emotion", "tone" or "the agent's personality" from a transcript
• "100% compliance" or "fully automatic redaction" is promised
• A score is shown, but it is impossible to see which sentence it rests on
• There is no "not applicable" status — every criterion applies to every call
• The sample size one vendor recommends for its own product is presented as a universal rule

None of these is an automatic reason to reject a vendor, but each one needs a written explanation.

A vendor comparison table

Give each vendor 1–3 points on each of the eight groups above: 1 — absent or not demonstrated, 2 — partial or only on the roadmap, 3 — shown to work on your test set. Set a separate minimum for the "evidence" and "standard" groups: a system scoring 1 on either should not pass, however well it does elsewhere. That ties the decision to actual results on your calls rather than to the best presentation.

What the trial agreement should say

Agree a few terms in writing before giving a vendor your test set. They protect both the data and the objectivity of the decision:

• Deletion of recordings and transcripts when the trial ends, with confirmation
• Test calls are not used to train models or for any other purpose of the vendor
• The success criterion: which agreement on which criteria counts as a pass — set before results arrive
• The trial's duration and who is responsible on the vendor's side
• No commitment after the trial — moving to a paid phase is a separate decision

Limits

A small test set will not show every risk: rare call types, seasonal peaks and new products may not appear in the trial. A separate pilot phase is needed after selection. Legal requirements for storing call recordings and passing them to a third party differ by country — check the vendor's answers with your lawyer.

Ask Vexvon the same questions

Short, verified answers about Vexvon Audio Analyzer:

• Recordings are uploaded as files (mp3, wav, m4a and other common formats, up to 10 files at a time, up to 200 MB each); the system analyses recorded calls and does not work in real time
• Agent and customer are separated by channel on stereo recordings and automatically on mono; an uncertain line stays "unknown", and poorly recognised lines are marked
• The company writes the standard in its own words; steps get "met", "partial", "missed" and "not applicable" statuses
• Every status comes with a comment and references to transcript lines; no seconds in the audio are shown
• Changing the standard creates a new version, and an old call keeps its version; an existing transcript can be re-evaluated
• The transcript is translated into chosen languages, and the summary is written in the report language

For anything not named here — retention period, access, pricing — ask us directly. How to check a test set is explained in call center QA sampling, and the cost calculation in speech analytics ROI.

First step

Before your next demo, prepare a test set of 20–30 calls and a hand-evaluated version of them. Send the eight groups of questions to the vendor in advance, so the demo answers your questions rather than following the vendor's script. To run a trial on your test set, get in touch; more pieces are in the call QA strategy section.

Further reading on this topic: speech analytics pilot, multilingual call quality assurance, speech analytics accuracy testing.