Two things happened in the last few weeks that, taken together, mark a shift for anyone running a voice agent in production. On June 29, Retell shipped Conductor, a review layer that sits inside the agent's workflow graph and refuses to change anything without a human approval. And on May 1, the Five Eyes cybersecurity agencies published joint guidance telling organizations to assume agentic AI will misbehave and to deploy it "incrementally, beginning with clearly defined low-risk tasks."
The through-line is not subtle. A voice agent is agentic AI: it plans, it calls tools, it takes actions with real customers on real phone numbers. The industry has decided, all at once, that shipping one is no longer a prompt-and-pray exercise. This post is a launch-readiness checklist for the product managers, ops leads, and compliance owners now on the hook for that.
Why "just launch and monitor" stopped working
The intuitive playbook, still common, is to write a prompt, wire up a few tools, run a handful of hand-tested calls, and go live. Then a small QA team listens to a sample of production calls each week and files tickets. It is a chatbot process ported to voice.
It breaks for three reasons.
First, sampling misses almost everything. At production volume, human QA can only cover a sliver of calls: as AssemblyAI's 2026 landscape review puts it, "Traditional QA teams can review maybe 1-2% of calls." When Retell alone now handles more than 55 million AI phone calls per month, a 1% sample is theatre.
Second, small changes have blast radius. Retell's own framing is worth quoting: "a single change can affect thousands of live calls." A one-word prompt tweak can change how the agent behaves on a dozen edge cases you did not think to check. Without a scored regression suite, you find out from a customer.
Third, the compliance floor moved. The Five Eyes joint guidance, co-authored by CISA, the NSA, and their peers in the UK, Canada, Australia, and New Zealand, organizes agentic AI risk around five categories: privilege, design and configuration, behavior, structural, and accountability. It is voluntary today. Forrester expects private-sector adoption anyway, the same way "Essential Eight" migrated from Australian government mandate to enterprise baseline.
Translated to voice: your legal and security counterparts will start asking questions your current process cannot answer.
The five categories, translated to voice
The Five Eyes taxonomy is generic. Here is what each category looks like when the "agent" is a voice agent on the end of a phone number.
Privilege. Your agent has tools. It can look up policies, transfer to a human, take a payment, book a slot, write to your CRM. Every tool is a permission. The guidance is blunt on this: construct each agent as a distinct principal with its own identity, and mutual TLS between the agent and every downstream service. On voice this maps to: what can your agent read and write, and what happens if a prompt-injected caller tries to convince it to do something it should not.
Design and configuration. The prompt, the graph, the tool schemas, the retrieval sources, the model version. Every one is a config that can regress. The question at launch is whether you can point at a passing test suite for the exact build you are shipping.
Behavior. This is the voice-native one. Does the agent interrupt callers? Does it stall for 4 seconds after a question? Does it read back PII it should not? Does it hallucinate a policy detail when retrieval is empty? Does it handle accents in the languages you claim to support? None of these show up on a text transcript alone.
Structural. How your agent depends on other systems: the telephony provider, the STT and TTS vendors, the LLM, the tool endpoints, the knowledge base. A voice stack routinely stitches together three APIs, speech-to-text, a language model, and text-to-speech, each with its own failure modes. Every dependency is a way for the agent to break without changing.
Accountability. When something goes wrong at 3am, who owns it, and can you reconstruct what happened? CSA's analysis of the guidance is clear that logs of tool calls are not enough on their own: "the reasoning chain that produced a given action... may be invisible or uninterpretable within existing security information and event management infrastructure." On voice, "reasoning chain" also includes the audio: what did the caller actually say, what did the agent actually output, and how long did the pause between them last.
The launch-readiness checklist
Everything below is table stakes, not aspiration. If you cannot answer yes to a line, you are not launch-ready, you are pre-launch. Skip nothing.
1. Scenario coverage
- You have written down the top 20 to 50 scenarios the agent must handle. Not "happy path" as one bucket. Real, specific tasks: "caller wants to add a driver to a policy mid-term," "caller is calling on behalf of an elderly parent," "caller starts in Spanish then switches to English."
- Each scenario has a clear pass condition, written before you build the agent.
- For each scenario, you have adversarial variants: interruptions, background noise, hostile tone, out-of-scope requests, prompt-injection attempts embedded in what the caller says.
- For each scenario, you have accent and language variants that match your actual caller distribution.
2. A simulation suite that dials real calls
Reading the transcript of a simulated conversation is not simulation. The audio pipeline is where most voice agents fail. You need scenarios that run as actual phone calls, with a synthetic caller on the other end, through the same PSTN or WebRTC path a customer would take.
- Simulations exercise your telephony path, not just the LLM.
- Simulations run automatically on every candidate build, not once before launch.
- Failures surface as specific scored scenarios, not "the demo felt worse."
This is what Roark's simulation testing is built for: it dials your agent over real phone calls (PSTN) and WebRTC, runs a scored suite of scenarios, and returns pass/fail per scenario before you promote a build.
3. Metrics that match how voice actually fails
Text metrics do not catch voice failures. If your evaluation is "did the LLM produce a sensible answer," you will miss the agent that interrupted the caller three times, mispronounced the customer's name, or paused for 4 seconds before responding. These are the failures customers remember.
- You are scoring pronunciation, pace, and pauses, not just semantic correctness.
- You are detecting interruptions and measuring who is talking over whom.
- You are catching emotion and vocal stress on the caller side, because that is the fastest signal that the agent is failing.
- Your metric set covers your compliance obligations (disclosures spoken, PII not read back, transfers offered when required).
Roark ships 64+ built-in metrics plus unlimited custom metrics, and its audio-native models measure pronunciation, emotion, vocal stress, pace and pauses, and interruptions directly from the recording. That is the vocabulary voice needs.
4. Production replay before promotion
Simulations catch known unknowns. Real callers generate unknown unknowns. The build you are promoting should be tested against the calls that broke the last build.
- You capture full call recordings from production with the right consent posture.
- You can select the calls that failed, or scored badly on a metric, and replay them against a new agent version.
- Regressions show up as scenario-level failures on a specific commit or config change, not as a vibes-based rollback.
Roark supports production call replay: capture the real calls, replay them against updated agent logic. The 2am incident becomes a permanent test case.
5. Language and accent coverage
If you support a market, you support the accents in that market. A US healthcare receptionist that fails on South Asian or Latin American accents is not launched; it is deployed to a subset of your callers.
- You have simulation coverage across every language and accent you claim to serve.
- Your metrics are scored in those languages, not just English.
- You know the accuracy delta between your best and worst accent bucket, and it is inside your tolerance.
Roark supports 45 languages and accents across simulation and scoring.
6. Compliance and security posture
The Five Eyes guidance is a security document, and the security controls it recommends flow through to your evaluation platform, not just your agent. Anything that touches call recordings or transcripts is now part of your compliance surface.
- Your evaluation vendor is SOC 2 audited.
- For healthcare or other regulated verticals, you have a signed BAA or equivalent.
- Third-party pen-test evidence is on file.
- Access to call recordings is scoped, logged, and reversible.
Roark is SOC 2 Type II certified, pen-tested, and offers a HIPAA BAA. If you are in healthcare or another regulated space, this is the floor.
7. Rollback and containment
The Five Eyes guidance calls out resilience and reversibility as the priorities, not efficiency gains. For voice this is concrete.
- You can roll back to the previous agent version in minutes, not hours.
- You have a defined off-switch that routes calls to humans if certain metrics degrade in production.
- You have a runbook for the "agent is behaving oddly" call, not just the "agent is down" call.
What "governed launch" looks like in practice
The gap between the old and new process is smaller than it looks. Most teams already have half of this. The difference is that the artifacts stop being tribal knowledge and start being evidence.
The category is starting to converge on this shape. Retell's Conductor already generates around 70% of its own simulation tests and executes about half of its internal agent edits, which tells you where the labor is going. Nobody serious is still hand-listening their way to production.
The one design decision worth flagging: keep your evaluation independent from your agent runtime. If the same platform builds your agent, scores your agent, and decides when it is safe to ship, you have no second opinion when it matters. Treat evaluation the way you treat auditing. It is where the reviewer sits.
Where Roark fits
Roark is the evaluation, simulation, and observability layer that lives beside your agent runtime, not inside it. It has one-click integrations with Vapi, Retell, LiveKit, Pipecat, Bland, and ElevenLabs, plus Node and Python SDKs and a client.call.create ingestion path if you are running something custom.
Practically, that means the checklist above is a workflow, not a project:
- Author scenarios once, in Roark.
- Every candidate build gets dialed over PSTN or WebRTC and scored on your metric set.
- Real production calls flow in via the integration; the failures become new test cases.
- Compliance and product see the same pass/fail evidence, per build, per scenario.
The pricing page has the plan detail. Talk to us if you want a working suite on your agent inside a week.
The short version
Voice agents crossed a line this year. They are agentic AI, they carry the risks the Five Eyes agencies just named, and the shipped-a-prompt way of running them is over. The teams that will still be in production twelve months from now are the ones that treat launch as evidence: a scored suite, replayable production calls, real audio metrics, a clean rollback. Everything else is a bet against your own callers.
