Conversation analytics software checklist: 30 procurement questions

Short answer

When choosing a conversation analytics platform, the key question is not "how smart is the AI?". What matters is: which conversations the system sees, how it works in your language, who controls the categories, whether every number has evidence behind it, whether the unanalysed part is shown, how data is protected, and whether all of this can be tested in a pilot on your own data.

Below are 30 questions in seven groups. Send them to vendors in writing, check the answers in a demo on your own conversations, and score each answer as "shown", "stated but not shown" or "missing".

How to use it

1. 1Write your own question firstDecide which decision you will use the platform for. The checklist questions should serve that decision.
2. 2Send it in writingSend the questions before the demo so the answers are prepared.
3. 3Ask to be shownFor every important answer, say "show me on screen". A promise on a slide and a working feature are different things.
4. 4Three-point scoring2 — shown, 1 — stated but not shown, 0 — missing or unclear.

1. Sources and channels

• Which channels' conversations are analysed: calls, chat, messengers, social media messages, email?
• For calls, is the transcript produced by the platform itself, or must it be brought in from outside?
• Are internal notes and staff-to-staff comments excluded from analysis?
• Are test and trial conversations separated so they do not inflate statistics?
• Can past conversations (the history) be analysed?

2. Language

• How does it work with your customers' language, including mixed scripts, typos and dialect?
• How are conversations that mix languages analysed?
• In which languages can results and reports be shown?
• Can language quality be tested on your own conversations, or is the demo in English only?

3. Control over categories and fields

• Who builds the categories and fields: you, the vendor, or the system automatically?
• Does AI pick from a closed list or create free labels? Free labels break trends.
• Can you write a definition and examples for a field?
• Where do answers that fit no value appear ("other")?
• What happens to old results when a field or value changes — are they mixed, or re-analysed?
• Can one conversation have several values for a field?

4. Evidence and coverage

• Can you go from every number to the conversations behind it?
• Is the customer's own sentence shown next to the result?
• Does the report show how much of the period has been analysed?
• Are conversations whose analysis failed visible?
• If the data asked about was never collected, does the system say so, or answer with something close?

5. Reliability

• If the vendor quotes an accuracy figure, on what data, in what language and how was it measured?
• Is there a workflow to check results by hand and flag errors?
• Who computes the number: a database query or a language model? When a model counts, numbers can vary.
• Is the way analysis runs (which messages, when) fixed across all customers?

6. Privacy and security

• Where is the data stored, and to which service providers is it passed?
• Who sees what: aggregate numbers, conversation text, customer record — is access role-based?
• What are the retention period and deletion procedure?
• How is sensitive data (card numbers, health) prevented from being extracted as a field?
• Which legal and certification claims can the vendor confirm in writing?

Legal requirements depend on jurisdiction. In Azerbaijan, the Law on Personal Data applies; companies with EU customers should consider GDPR principles (purpose limitation, data minimisation, storage limitation). Check the vendor's answers with your own lawyer.

7. Pilot terms

• Can the pilot run on your own data, and for how long?
• Do data and fields carry over after the pilot, or does everything start from zero?
• Can results be exported (PDF, Excel, API)?
• Who sets up the fields and helps with calibration during the pilot?

How to structure the pilot itself is covered in conversation analytics pilot.

Who to involve in the evaluation

When one person picks the platform, the questions are one-sided. A small evaluation group works better: the business lead who will use the results (sales, marketing or support) checks the decision question and the meaning of the value lists; IT looks at integration, access rights and data flow; a lawyer or the person responsible for data protection reads the privacy section; and an agent or support employee who knows the conversations well says whether the values AI picked on sample conversations are right. Each scores only their own section, and the business lead makes the final decision.

Red flags

• There is an accuracy figure but no explanation of how it was measured.
• The demo only uses prepared data, never your conversations.
• "AI finds everything itself" — no control over fields and values.
• No way to go from numbers to conversations.
• Promises of "automatic compliance" or "full privacy" without written detail.
• A partially analysed period shown as complete.

An illustrative evaluation

This example is illustrative. A company compares two platforms. Platform A shows channels and languages very well, but there is no link from numbers to conversations and the vendor builds the categories. Platform B supports fewer channels, but fields are under the company's own control, and every number has a conversation list and a coverage rate underneath.

The total scores are close. The company looks at its decision question: to discuss objections with the sales team, evidence matters, because the sales lead wants to check every number against a few conversations. B is chosen, and a written roadmap is requested for the missing channel.

How Vexvon answers these questions

In Vexvon the company builds the fields and values itself; AI picks from a closed list and anything that does not fit falls into "other". Every number links to conversations, the report shows the share of the period analysed, internal notes and test conversations are excluded from analysis, and numbers are computed by a database query. Results can be written in Azerbaijani, English, Russian and Turkish. For privacy, retention and legal questions, see the security page or send us a written request.

Next step

Trim the 30 questions to fit your decision question and send them to every vendor in the same form. We can test them for Vexvon on your own conversations in a demo; the analytics capabilities are described on the analytics page.