Six QA Barriers That AI Scoring Won't Fix

The QA scorecard hardly changed for 25 years. Nerys Corfield first got her hands on one in 1998, and in her experience nothing much moved until about three years ago, when AI scoring made it possible to evaluate up to 100% of conversations.

That's a big opportunity. But AI scoring sits on top of the QA program you already have. And if that program is just chasing quotas or your agents don't trust it, then scoring more conversations isn't going to fix it.

Nerys has spent her career inside contact center QA. She worked for 18 years in contact center outsourcing, much of it assessing scorecards and calibration for big brands, then went independent in 2015. Since then she has audited around 120 contact centers in the UK and US, and her work now focuses on bridging the gap between technology and operations.

Speaking on Call Centre Helper's 8 October webinar, How to Tackle the Biggest Barriers to Quality Assurance, she sets out six barriers that get in the way of effective QA, and what to do about each.

Six barriers to effective QA

QA becomes a quota to hit

In Nerys's hypothetical example, she has to tell 12 family members she loves them every month: three phone calls and one text each. By the 28th she's behind, so she rings round saying "I love you" as fast as she can. Her scorecard is green. "But do my family actually feel loved?"

Anyone who has worked to an evaluation quota will recognize it. Nerys points to Goodhart's law: when a measure becomes a target, it stops being a good measure. If success means "assessments completed," QA turns into box-ticking, and the people doing the scoring rarely get the time to do it well.

Scorecards go stale the same way. Nerys recommends reviewing them quarterly but, in the course of her work, she has come across some that are ten years old. If your success criteria haven't changed in a decade, they probably don't describe success anymore.

What to do: define success as improvement, with CSAT, employee satisfaction and quality scores moving up together. Build the scorecard around what matters to the customer, the business and the agent. Once AI scoring covers the volume, the quota is handled. The value is in what you do next.

A culture that counts clean plates

Take Nerys's example of two restaurants. In the first, everyone cares about the food and asks for feedback, because that's the culture. In the second, staff count four clean plates and call it quality. Both serve good food, but of the second she says, "There's no real passion for it. It's just a functional delivery mode."

Culture is slow to change. Her test for where yours stands: who in your C-suite asks how you're doing on quality, and what do they ask?

What to do: find out who in your leadership team asks about quality, and give them more than NPS. As Nerys points out, NPS is becoming a smaller sample, it tends to capture "the ranters and the ravers," and it isn't relevant to every organization.

Diagnosis and treatment need different skills

In a hospital, the person who reads your scan usually isn't the person who treats you. Nerys sees QA the same way: scoring is diagnosis, coaching is treatment, and they call for different skills. Sometimes team leaders do both. Sometimes QA sits apart and passes scores to someone else. Either way, whoever delivers the feedback needs training to do it well, and examples of good coaching feedback are a useful place to start.

She learned that the hard way. Years ago, after listening to a dreadful call, she went straight to the agent and gave the feedback without preparing. "She stood up, and she walked out of the contact center never to be seen again." Nerys has also worked in operations where every team leader was trained to level five in coaching, and she describes the impact on quality, motivation, shrinkage and attrition as significant.

What to do: train whoever delivers coaching, and protect one-to-one feedback time in your workforce plan. As QA teams take on more of the coaching, invest in their coaching skills too.

Want to see how other QA teams are handling this? Download What Contact Centres Are Doing Right Now, our 2026 research report with Call Centre Helper, and benchmark your program against your peers.

QA has a PR problem

QA can affect commission, performance improvement plans and careers. With that much power, it's easy for agents to see the QA team as the people who hand out red cards. A QA manager at one insurance organization told Nerys plainly: "They hate us. Absolutely hate us." AI scoring can make this worse, because it surfaces every reason an agent might be shown that red card. And agents already trust AI scores less than their managers do: in our AI Reality Check report, 57% of agents trust AI-generated QA scores enough to make decisions about performance, against 76% of managers.

What to do: balance the red cards by celebrating the wins too. Introduce QA properly at induction, and have new agents score some calls themselves so they understand the criteria they'll be measured on. Then invite agents to challenge their scores. In a supportive culture those conversations build understanding. In a combative one, Nerys warns, agents leave because they don't think it's fair.

Taking the coaches off the bench

Nerys is seeing organizations decide they no longer need coaches, because AI is doing so much of the scoring. She disagrees: "You can't not have coaches on the bench." Someone still has to celebrate good work, spot knowledge gaps and feed back to trainers. And the AI isn't the one sitting down with the agent afterward.

What to do: keep people in the loop, and move evaluator time from ticking boxes to coaching and insight.

Data everywhere, and not a drop to drink

Scoring every interaction creates a flood of data, and Nerys warns that it's easy to get lost in it. The upside is that QA can now support quantifiable root cause analysis across the business: tracing a dip in customer sentiment to the delivery drivers, say, or to a page on the website that leaves people confused.

That takes data analysis skills, and standing with the C-suite. In Nerys's words, it's a chance to "take the walls down from the contact center." We looked at the same idea from the bot side in Who's QA-ing Your Bots?

What to do: give someone in QA ownership of root cause analysis, and a route to the teams that can fix what they find.

How many people does QA need?

In the audience Q&A at the end of the webinar, Nerys offers a rough way to work out QA headcount. Of course, this is her rule of thumb, and every operation's numbers will differ.

1. Decide how many interactions per agent need a review. If scoring is automated, count the ones your team will still review themselves.
2. Multiply by the average end-to-end minutes per review, including the feedback session.
3. Allow for shrinkage, as you would for agents. She uses 30%.
4. Divide the hours by 1,950 to get your FTE.

It ties back to her first barrier: set a quota without the capacity to meet it, and the quality of the reviews is what suffers.

Foundations first

AI scoring will score every interaction against whatever your program says good looks like. Fix the foundations above first, and up to 100% coverage becomes a way to improve faster. Skip them, and AI scoring won't fix what's broken in your QA. It applies the same flawed criteria to every conversation it scores, so the problems grow with the coverage. One audience tip during the webinar summed it up: don't confuse 100% coverage with 100% understanding.

Watch the webinar replay to hear Nerys's six barriers in full.

ScorebuddyCX brings scoring, coaching and learning into the same platform, so evaluators can spend the time AI scoring saves them on coaching and insight. See how ScorebuddyCX connects QA and coaching.