A voice agent can be fluent, on-brand, and confidently wrong in the same sentence. On a chat interface, the caller can scroll up, screenshot the fabricated policy, and forward it to your CS team. On the phone, the invented claim decays into the air the moment it's spoken. If your QA never surfaces it, the caller carries it away as fact.
This is the failure mode voice teams underweight the most, because it doesn't crash the call, doesn't show up in a latency dashboard, and doesn't fail a scripted UAT. The agent said something. The transcript reads cleanly. The customer hangs up. Weeks later, a support ticket lands complaining that "your bot told me the deductible was $250," and no one has any idea whether it did.
Voice agents hallucinate at two layers
Any post about hallucination in text LLMs describes one layer: the model generates a claim that isn't in its context. Voice agents inherit that layer, plus a second one text agents don't have. The ASR itself invents words.
The Cornell and University of Washington "Careless Whisper" study found hallucinations in roughly 1% of Whisper transcriptions of aphasic speech, where the model produced fluent, made-up sentences during moments of silence. That's not a rounding error at contact-center volume. A Montreal AI Ethics Institute summary of the same work reports that nearly 40% of those hallucinations were actively harmful, inserting invented references to violence, race, or non-existent medications.
The paper also finds the pattern is worse for speakers with longer non-vocal durations, disproportionately harming people with aphasia and older callers. If your production traffic includes callers who pause, mumble, or take breaths mid-sentence (that is: all callers), you have an ASR-layer hallucination surface as well as an LLM-layer one.
The LLM layer is the more familiar failure. A survey of LLM reasoning and agent evaluation reports that state-of-the-art models without grounding reach only about 34% accuracy on the Comprehensive RAG Benchmark, simple RAG lifts that to 44%, and industry-leading grounded systems max out around 63%. At voice-agent volumes, "industry-leading" still means a lot of confidently wrong answers per shift.
Sampled QA can't find them
Traditional call-center QA reviews 1 to 2% of calls. That sampling model was designed for human agents whose errors were roughly random. AI hallucinations aren't. The Gistly hallucination guide for contact centers puts it plainly: AI errors are deterministic, the same input conditions produce the same hallucination every time, and sampling 2 to 5% of interactions misses systematic AI errors.
At voice-agent scale, the math is unforgiving. If the LLM hallucinates on 3% of calls (the low end of published estimates) and you review 2% of calls at random, you catch roughly 0.06% of the hallucinations that happened. The caller who was told the wrong deductible has already hung up.
Getting from 2% coverage to 100% requires two things you can't buy off a QA-outsourcer's rate card: automated grounding checks that run on every call, and simulation-driven pre-launch tests that shake out the systematic failures before they touch a live line.
Grounding, in one sentence
Grounding is the property that every factual claim the agent speaks traces back to an approved source: a tool result, a retrieved document chunk, a policy rule, or a prior turn in the conversation. Retell's grounding overview reduces it to a useful test: does every claim in the answer trace back to the retrieved source? If yes, the answer is grounded. If no, it's a hallucination, whether or not it happens to be true.
Two measurable properties fall out of that definition:
- Faithfulness (or groundedness): for each factual claim in the agent's response, is that claim supported by evidence the agent had access to?
- Citation coverage: what share of responses contain claims traceable to a real source, versus claims the model freelanced?
Neither metric appears on your telephony provider's dashboard. Both have to be scored by an evaluator that sees the transcript, the tool calls, the retrieved chunks, and the audio.
What to test before launch
Grounding tests belong in your simulation harness, run against the agent over real phone calls before a single production caller reaches it. A useful pre-launch grounding suite covers four categories.
Known-answer scenarios. Set up personas who ask questions with a single ground-truth answer verifiable against your source data. "What's my copay for a specialist visit?" against a mocked benefits API. Score each response for whether the number the agent spoke matches the number the tool returned, and whether the agent added unsupported qualifiers ("but that only applies in-network" when your tool result said nothing of the sort).
Out-of-scope scenarios. Ask the agent things your source data can't answer. A well-grounded agent says it doesn't know and offers to escalate. A hallucinating agent guesses. Score refusal-when-appropriate as a first-class metric, not a nice-to-have.
Underspecified prompts. Personas that ask ambiguous questions ("what's my rate?" without saying which policy) test whether the agent asks for clarification or guesses. Research on agentic RAG evaluation for call centers treats correct escalation as a successful outcome. So should you.
Silence and disfluency scenarios. Personas with long pauses, backchannels, and aphasia-like speech patterns to exercise the ASR-layer hallucination surface. If the transcript comes back with words the caller never said, and the LLM builds an answer on top of them, your grounding layer never gets a chance to catch anything.
Each of these needs to run against your real agent, over a real telephony path, before launch. Text-mode simulation misses the ASR failure mode entirely, and that's precisely the mode most likely to inject a fake medication name or a false address into a call.
What to score on every live call
Pre-launch tests catch the systematic hallucinations you can imagine. Production monitoring catches the ones that only appear at scale: new SKUs the KB doesn't cover yet, edge cases in tool outputs, drift when your model provider ships an update behind the "latest" tag.
The goal is 100% coverage. Every call, scored on the same grounding metrics you tuned against in simulation, with an issue filed automatically when a call fails a threshold. That means storing the audio, the transcript, the tool calls, the retrieved chunks, and the full trace, so an evaluator can reconstruct whether each spoken claim had evidence behind it.
Patterns worth flagging as first-class issues:
- The agent quoted a number no tool returned.
- The agent quoted a policy detail with no matching chunk in the retrieved context.
- The agent affirmed a caller's claim ("yes, that's right") without any evidence the claim was correct.
- The agent's spoken answer contradicted the tool result it received one turn earlier.
- The transcript contains phrases the caller almost certainly didn't say, flagged as candidate ASR-layer hallucinations.
The Caylent writeup on grounding evaluation in agentic RAG is honest about why per-turn evaluators fall short in agentic flows: meaning is distributed across turns, and a claim in turn 6 might be grounded in a tool result from turn 3. Your evaluator has to see the whole call, not one turn at a time.
Turn every caught hallucination into a regression test
Finding a hallucination once is a win. Letting the same one recur after the next prompt tweak is a self-inflicted wound. Every caught hallucination should become a repeatable test that runs against every future version of the agent.
The mechanism that scales is production call replay. Capture the original call, promote it into your regression suite, and replay it against the updated agent every time you change a prompt, swap a model, or ship a new tool. If the fix held, the test passes. If a later change regressed the same failure, the test catches it before the new version reaches a live caller.
This is the discipline the industry imports from software engineering, and it's the missing half of most voice-agent QA programs. Prompt changes get shipped without a regression pass. Model providers retire the "latest" tag and behavior drifts overnight. Without a growing regression suite of real, previously-broken calls, every fix is temporary.
The bill of materials for a grounding program
If you want to lift the playbook out of this post, it's four pieces working together:
- A simulation harness that dials your agent over real phone calls with personas covering known-answer, out-of-scope, underspecified, and disfluent inputs.
- Audio-native scoring that runs on every production call, evaluating faithfulness and citation coverage against the tool outputs and retrieved context the agent actually had.
- Automatic issue-filing when a call fails a grounding metric, so nothing depends on a human happening to review the right 2% of calls.
- A production-replay pipeline that turns each caught hallucination into a permanent regression test.
Where Roark fits
The playbook above is what Roark is built to run. Simulations dial your agent over real phone calls (PSTN and WebRTC) with personas that stress ASR, ambiguity, out-of-scope requests, and disfluent speech, in 45 languages and accents. Every production call is scored against your grounding metrics using audio-native models that read the sound of the call, not just the transcript. Failures become filed issues automatically, and captured production calls turn into replayable regression tests you rerun on every prompt or model change. The SDKs and integrations plug into Vapi, Retell, LiveKit, Pipecat, Bland, and ElevenLabs, so the harness sits alongside the agent you already built.
You don't need Roark to do this. You do need to do it. The teams that catch hallucinations before their callers do are the ones running grounding as a first-class metric on every call, and treating each caught failure as the seed of a test that never has to be caught the same way twice.
