Your bots now resolve a share of customer contacts on their own, and your agents get the conversations that still need a person: the complex and sensitive ones, and the ones the bot couldn't finish. That's a hybrid contact center.
Traditional manual QA has always sampled a few calls per agent each month and scored them, and for a long time that was all it had to do. The idea of checking what a chatbot told a customer, or why it handed the conversation over, is a relatively new QA demand. To meet these new demands, you need to define one quality standard for bots and agents, judged on customer outcomes, and ensure human checks on the AI doing the scoring.
Speaking on Call Centre Helper's 8 October webinar, How to Tackle the Biggest Barriers to Quality Assurance, ScorebuddyCX CEO Derek Corcoran set out six barriers he's seeing when organizations try to hold humans and bots to one standard of quality, and what to do about each.
Six barriers to hybrid quality
| The barrier | What to do |
|---|---|
| Your knowledge base isn't ready for humans and bots | Put policy around it: an owner, an approval process, and updates that reach bot and agent together |
| Escalations fail out of sight | Score the whole conversation, the bot's side and the agent's |
| Agents are heading into a storm | Coach for complexity, and have a person review AI coaching tips before agents see them |
| Traditional QA is no longer fit for purpose | Move evaluator time from scoring to acting on the data |
| Nobody is QA-ing the bots | Give bots scorecard criteria of their own |
| Deflection is the wrong measure | Measure bots on customer outcomes |
Your knowledge base isn't ready for humans and bots
A lot of what agents know lives in their heads, or in documents they keep close to hand. A bot can't read either. As knowledge moves into a central repository that feeds both bots and agents, one source of truth gets harder to hold and matters more. If the bot and the agent work from different information, your customers get different answers.
What to do: whatever the technical fix, it needs policy around it. Decide who owns the knowledge base, how changes get approved, and how quickly an update reaches both the bot and the agent's desktop.
Escalations fail out of sight
When a bot hands a conversation to an agent, you often can't see why. Did it repeat itself? Give a wrong answer? Loop the customer for minutes before letting go? Contradict what an agent told the same customer yesterday? If you can't see into the bot side of the conversation, handover failures don't show up anywhere.
What to do: score the whole conversation, the bot's side and the agent's, so you can see where it escalates and why.
Agents are heading into a storm
Self-service pushes contact volumes down. The conversations that reach your agents get harder: complaints, vulnerable customers, retention calls, failed bot attempts, compliance-sensitive cases. Handle time goes up. So does the emotional load, and with it the risk of burnout, sick leave and churn.
Customers still want a person, too. 87% say companies using generative AI for customer service must give them a way to reach a human agent (Gartner, 2026). Human agents aren't going anywhere, and their job is only going to matter more.
What to do: hire and coach for emotional intelligence, complex problem-solving, policy awareness and AI oversight. AI can suggest coaching for each agent, but have a team leader or evaluator review it before it reaches them. Skip that step and agents can start rejecting the tips, then disengage from the process altogether.
How does your QA program compare? What Contact Centres Are Doing Right Now, our 2026 research report with Call Centre Helper, shows how contact centers monitor quality, what they want to fix, and how far they trust AI. Download the report.
Traditional QA is no longer fit for purpose
Sampling 2% of interactions made sense when people did all the scoring. AI can now score up to 100% of conversations across humans, bots and channels, which shows you where inconsistency starts. But your QA team needs new skills and processes to make use of all this coverage.
Reviewing a sample once a month also means you find problems late. For one of our customers, a UK consumer lender, QA used to be a monthly look in the rear-view mirror: at-risk customers surfaced weeks after the conversation, which left the lender open to regulatory exposure. Now an AI evaluator scores each conversation within an hour of it ending and flags vulnerable customers to a response team ahead of any complaint, with a strong audit trail to show any regulator.
What to do: move evaluator time from scoring to acting on what the data shows.
Nobody is QA-ing the bots
Bots are designed to behave almost like people, which means they can make mistakes if nobody's watching. A standard generative AI deployment won't always match your brand tone or follow the policy and compliance detail your practitioners know matters. Someone has to check what the bots say.
That's where one of our customers, a B2B software company, found itself. It had deployed an AI chatbot with no structured way to QA it, and wanted to score the bot's conversations at volume to sharpen the instructions behind it. At first, the AI evaluator's scores were only 50% aligned with the QA team's. Three testing rounds later, each separated by a calibration round with the QA team, they were aligned.
What to do: give bots scorecard criteria of their own, and send the results to whoever owns the bot's design so they can keep improving it.
Deflection is the wrong measure
Bot vendors talk about containment and deflection rates. A bot could deflect 70% of conversations and still drive repeat contacts and frustrated customers.
What to do: measure bots the way you measure people, on customer outcomes. Did the customer's problem get solved?
Keep the AI evaluator honest
Scoring every conversation with AI only helps if you can trust the scores. An AI evaluator will always return a score, whether or not your human evaluators would agree with it.
Contact centers are still figuring out what level of human checking is required for AI scores. In our AI Reality Check report, 14% of those using AI to evaluate interactions say a human always reviews or challenges the scores. For 47%, that happens sometimes, rarely or never.
Checking also means looking below the total. When one of our customers, a large US financial services company, compared AI and human scores, it found that matching the final number isn't enough: an AI evaluator and a human can reach the same total while scoring individual questions differently.
In the first phase, the AI matched the human evaluators on eight questions with no calibration at all. At question level the gaps showed, so the team reran the same conversations through several calibration rounds, and the scores moved closer each time. Its bar for trusting the AI: aggregate scores within 2–3% of the human benchmark.
Getting aligned once doesn't settle it. Change a scorecard, adjust the criteria, or let the AI model update behind the scenes, and the scores can drift out of line again. That makes calibration part of change management: test and calibrate again after every update.
In the webinar Q&A, Derek said that in his experience AI evaluators handle soft skills well, because tone, repetition and word choice are all language. They have a harder time with policy and regulation: did the agent follow the refund process, or hand out a discount they shouldn't have?
What to do: benchmark AI scores against human scores before you rely on them. Ongoing calibration with humans in the loop is how you catch drift.
One standard, two workforces
Hold bots to the same quality standard as people, judged on customer outcomes, sentiment and consistency.
That shared standard also gives QA something to say to the rest of the business. Once you score bot and agent conversations side by side, the cause of a problem can turn out to sit outside the contact center: a back-office process, a confusing web page, a delivery team. The QA team that can see humans and bots together is the one that can tell those teams what to fix. Derek makes the longer version of that case in From QA to CX Intelligence.
Watch the webinar replay for the full discussion.
ScorebuddyCX scores agent and bot conversations in the same platform. AI Auto Scoring evaluates up to 100% of them, and you can calibrate its scores against your human evaluators, so you get a bird's eye view of every conversation, human and bot. Book a demo to see it in action.
