Backed by Y Combinator

Test, monitor and improve your voice agents.

Roark Evals runs your agent against hundreds of simulated callers before launch, then scores every live call and files what breaks. Works with any agent, on Vapi, Retell, LiveKit, Pipecat or Roark.

Before launch

240 simulated callers

92%

Task completed, with 3 failures caught before a customer heard them

On every live call

Example: every live call is scored as it ends, and calls that fail are filed as issues.

Example data

Used by voice AI teams at

Google
AT&T
BCG
Spectrum
Aircall
Podium
radiantgraph
Google
AT&T
BCG
Spectrum
Aircall
Podium
radiantgraph

Teams ship faster when they can hear what breaks.

Roark has reduced the time we spent manually testing by a few hours per week, allowing us to focus on improving the product.
Elaine Phoon, Operations Manager (AI Enablement), SEEK Pass
Nearly 4,000
simulated calls completed since February 2026
3.7 to 4.7 out of 5
coverage of required questions between March and October 2026

Production voice AI scored on every conversation: pronunciation, empathy and resolution across their support and sales calls.

radiantgraph

Healthcare calls evaluated for disclosures and identity checks: compliance scoring on every conversation, automatically.

Client voice agents validated in simulation before they go live: evidence that a build is ready, not a hunch.

Simulation testing

Break it in staging, not in production.

Send hundreds of simulated callers through your agent, in different places and as different people, and see exactly where task completion drops.

  • Personas, environments and scripted flows
  • 45 languages and accents
  • Regression, load and red-team runs
Explore simulation testing
Simulation run: booking-testsIllustrative

Task completion by condition

0 of 240 simulated calls

  • Quiet line, calm caller0%
  • Where the caller is

    Airport background noise-6 pts0%
  • Speakerphone in a moving car-8 pts0%
  • Who is calling

    Strong regional accent-4 pts0%
  • Impatient caller who interrupts-12 pts0%
  • Older caller who speaks slowly-2 pts0%

Running simulated callers through your agent

Post-call analysis

Every production call, scored.

Roark scores every call on 500+ audio-native metrics and files what breaks, with the audio and transcript attached.

  • Issues filed automatically
  • Alerts and dashboards
  • Traces for every call
Explore post-call analysis
platform.roark.ai/calls/c_8f42

+1 (415) 555-0134 → support_v2

Today 14:32 · 3m 42s · Vapi

91

Caller: I was told the refund would arrive by Friday…

Agent: Let me check that for you, one moment.

Dead air: 3.8s before the agent responded.Issue #482 filed

Metrics

Pronunciation
94
Empathy
84
Instruction following
97
Response time
88
Disclosures ✓Identity check ✓+ 60 more scored

Human review

Check the scores against people who know.

Your reviewers label real calls and set the ground truth, so every metric matches how your team would judge the call.

  • Compare model scores with your labels
  • Correct disagreements in place
  • Tune metrics on what you label
Explore human review
Review · refund_call_04283 of 3 labeled
MetricModelYou
Identity verifiedPassFail
Empathy3 / 54 / 5
ResolutionResolvedResolved

transcript + audio in view · two reviewers · disputes settled inline

Self-improvement

From caught issue to proven fix.

A failed call becomes a test, a drafted fix and a check on live calls. Your team still decides what ships.

  • Fixes drafted from real failures
  • Every change explicit and diffed
  • Confirmed on live calls
Explore self-improvement
Your fix, diffedsupport_v3 v4
PromptModelTool
− Issue the refund right away.
+ Verify the caller’s identity, then issue the refund.

Everything in Roark Evals.

Before launch

Simulation testing
Scenarios and personas
Hundreds of simulated callers built from your real call types
Red teaming
Adversarial callers probing for jailbreaks, prompt injection and off-policy answers
Multilingual testing
45 languages with native accents, code-switching and noise
Load testing
Hundreds of callers at once, before launch day does it for you
Health checks
Always-on probes that catch an outage before a customer does
Regression testing
Every change compared with your last green run, so nothing quietly breaks
CI/CD gates
Every prompt or model change runs the suite before it merges

In production

Post-call analysis
500+ metrics
Audio-native, conversational, compliance and latency, on every call
Pronunciation testing
Drug names, SKUs and people, checked from the audio itself
Custom metrics
Your rubric, scored on every call automatically
Human review and ground truth
Label real calls, set the right answer and see how each metric agrees
Issue tracker
Failures filed and grouped, tracked across deploys
Alerts and monitors
Thresholds on any metric, sent to Slack or a webhook
Tracing and dashboards
OpenTelemetry spans for every turn, and score trends by agent and version

Always improving

Self-improvement
Prompt optimizer
Prompt edits drafted from the calls that failed, with the evidence attached
Ground-truth tuning
Tune a metric on your labels until it agrees with your team
Model, voice and infrastructure fixes
Suggested swaps and settings when the wording was never the problem
Fix verification
Replay your fix against the exact callers that broke it
Live verification
The metric watched from the first call after your deploy

Define a simulation in code. Run it from the CLI.

Keep your agents, flows and metrics in your repo, apply them, and run the plan. Evals works on its own, so it fits the agent you already have.

New to Roark? This walks you through your first simulation.

roark get-started
Read the guide

roark.config.ts

const bookAppointmentTestFlow = defineImprovFlow({  name: 'book-appointment',  agents: ['frontdesk'],  happyPath: {    persona: 'new-patient',    environment: 'Quiet line',    prompt: 'You call to book a cleaning for next week.',    expectations: ['Agent offers a real opening and confirms the time'],  },  edgeCases: [    {      name: 'changes-the-day',      prompt: 'You change your mind about the day halfway through.',      expectations: ['Agent rebooks without starting over'],    },  ],})const bookingTestSuite = defineSimulationPlan({  name: 'booking-suite',  direction: 'INBOUND',  agentEndpoints: [{ agent: 'frontdesk' }],  metrics: ['agent_responsive', 'call_ended_cleanly'],  flows: [{ flow: 'book-appointment' }],  maxDurationSeconds: 300,})

Terminal

roark config apply roark.config.tsroark simulation run --data '{"planId":"<plan-id>"}'

Works with

VapiBlandRetellLiveKitPipecatElevenLabsKore.aiGoogle

Which voice stack is best? We call them all and find out.

300+
voice stacks ranked
8,000+
scored calls

See how your agent really does.

Connect your agent or send a recording, and get $50 in credit with a work email.

SOC 2 Type II. BAA available for HIPAA workloads.

Rather have us build it?Our team builds it, connects it to your systems and tests it on Roark.See Built by Roark