Chatbot troubleshooting: fixing errors in a live bot

«The bot isn't working» is not a diagnosis

A salesperson writes: «The bot gave a wrong answer again.» The head of support: «The bot never answered the customer.» Marketing: «The bot is silent on Instagram.» All three are true and all three are different problems. An investigation that starts with «the bot isn't working» usually runs for hours, because nobody knows which class of problem it is.

This article is a debug checklist for finding and fixing errors in a live chatbot: five error classes, the symptom of each, where to look, how to fix it, test cases and prioritisation. Pre-launch testing and regular quality audits are separate processes; here the subject is what to do when a specific complaint comes in.

Five error classes

1. 11. No answerThe customer wrote, the bot stayed silent. Symptom: nothing after the customer's message.
2. 22. Wrong routingThe bot answered but did the wrong job: sent a support question to sales, answered itself when it should have handed over, or the reverse.
3. 33. Stale or wrong informationThere is an answer, but it is factually wrong: an old price, a changed rule, an invented detail.
4. 44. Integration errorThe bot should have brought live data — an order status, a new lead — but could not get it, or it was not written.
5. 55. Tone and formThe answer is correct but long, cold, out of place or in the wrong language.

1. No answer: where to look

• Is the channel connected and active — has the page token been renewed, for example
• Was the bot stopped in that conversation by an agent — the bot goes silent once an agent joins
• Was the stop symbol used — when an agent writes, the bot pauses for a set time
• Is the message type supported — a comment reply or a reaction the channel does not process
• Limits — the website widget has per-minute and per-day message limits

In this class the error is often not in the bot but around it: the bot was silent where it should be, or the message never reached it.

2. Wrong routing

• A trigger word or rule is written too broadly — «price» fired inside a complaint
• The escalation rule is missing a phrase — the customer wrote «a real person» instead of «manager»
• The category was set wrongly — a lead went to support instead of sales
• Two rules fit the same case and the unexpected one wins

The fix is in the rule itself: add the phrase, narrow the scope, change the priority. After every fix, keep that conversation as a test case.

3. Stale or wrong information

The most common class. In Vexvon each answer shows which knowledge fragment it used, and that separates the causes at once:

1. 1The fragment is out of dateThe material is wrong — it is updated in the knowledge base and its owner told.
2. 2The fragment is correct but from another topicRetrieval found the wrong fragment — topics need separating, the fragment shortening, or question variants adding.
3. 3There is no fragmentThe bot said something not in the material — check the answer boundary and the «I don't know» behaviour. When no material is found, the bot should give a fallback, not invent.
4. 4Two fragments contradict each otherThe material owners decide which one stays.

The deeper side of retrieval quality is covered in RAG chatbot accuracy.

4. Integration error

• Did the company's API respond, or did it time out?
• Was the request well formed — order number, identity field?
• Has the API key expired or had its rights changed?
• The response came back but the bot misread it — a new status code, a renamed field
• A lead was created but never reached the CRM or the notification channel

What matters most in this class is that the error is not silent: when an integration fails the bot should answer the customer honestly and hand the conversation to a person. Model calls, latency and errors are logged in the platform — that is where the investigation starts.

5. Tone and form

• The answer-length setting — short or long answers
• Language — the customer wrote in Azerbaijani, the bot answered in Russian; language detection and the allowed-language list
• Tone guidance — is the brand's agreed voice written down
• Repeated stock phrases — «I understand» in every answer

Test cases: so a fix doesn't break again

Every fixed error becomes a test case: the real customer message, the expected behaviour, the date. Over time this list becomes the bot's regression set.

• When material changes — a price, a rule — the related test cases are rerun
• When a rule changes — escalation, routing — all routing tests are rerun
• When a new channel is connected — the core test cases are checked there too
• Once a month — the whole set

How to build the pre-launch test set is covered in the chatbot test checklist.

Prioritisation

1. 1Critical — todayA wrong price or promise, a missed escalation, total silence on a channel, personal data shown wrongly.
2. 2High — this weekAn integration error, a frequent routing mistake, stale material.
3. 3Normal — next auditTone, length, incomplete answers on rare topics.

Example: investigating one complaint

1. 1ComplaintA salesperson: «The bot told a customer delivery is free, but it's 5 AZN.»
2. 2ConversationToday at 11:40, Instagram. Customer: «How much is delivery to Sumgait?»
3. 3Class and sourceStale information. The fragment used: last month's «free delivery within Baku» promotion — never deactivated, and Sumgait not written separately.
4. 4FixThe promotion is deactivated, regional delivery prices are written as a separate entry, owner — the head of sales.
5. 5Test case«How much is delivery to Sumgait?» is added to the test set with the expected answer.

Debug checklist

• The specific conversation is found — date, channel, customer
• The error class is identified — one of the five
• The knowledge fragment used, or the rule that fired, has been checked
• Integration calls and errors have been reviewed
• The fix is made and its owner recorded
• The conversation is saved as a test case
• The same error has been searched for in other conversations

Limits

Not every error can be reproduced: an AI answer to the same question may not be word for word the same each time, so after a fix check it several times. Problems on the channel's own side — a platform API change, an account restriction — do not depend on the bot and are solved under the channel's rules.

Conclusion: the error class is half the fix

Chatbot problems fall into five classes, each with its own place: channel and pause, rules, material, integration, tone. Starting from the specific conversation, naming the class, looking at the fragment or rule used, and saving each fix as a test case — this sequence turns a «the bot isn't working» complaint from hours into minutes.

The regular quality audit is covered in the chatbot answer quality audit; the rest of the section is in this category. To investigate your bot's errors together, get in touch.

Frequently asked questions

1. 1Why does the bot sometimes go silent?Most often: an agent joined and the bot is paused, the channel connection dropped, or the message type is not supported. Look at the specific conversation.
2. 2How do we fix a wrong answer quickly?Find the knowledge fragment used: if it is out of date, update it; if it is correct, look at retrieval and how topics are split.
3. 3How do we stop the same error coming back?Save every fixed conversation as a test case and rerun it when material or rules change.