On November 4, OpenAI promoted its GPT-Live outbound telephony API to general availability, letting developers dial out through any SIP trunk provider with HD Voice and bidirectional DTMF. The same week, OpenAI's changelog opened outbound SIP calling to all organizations through the Live API. That one shift puts a lot more voice agents in a position very few of them have been built or tested for: dialing another automated system and getting past it.
Outbound IVR navigation is not a flashy feature. It is the part of the call that happens before the real conversation starts: the agent listens to a menu, decides which key to press, waits for the next prompt, and tries not to hang up on a human halfway through. It is also the part most teams ship without a real test harness. This post is about what to test, how to test it, and why the common shortcuts fall apart the first time a downstream menu changes.
Why outbound IVR navigation is a distinct problem
Testing a voice agent that answers calls is one problem. Testing a voice agent that calls someone else and talks to a machine is a different one. The agent is now both a client of the telephony stack and a client of whatever IVR picks up on the other end. It has to:
- Recognise that a menu is playing, not a human greeting.
- Parse options from a prompt that may list them in any order, with synonyms ("for billing, press or say two").
- Send the right digit, in the right format, at the right moment.
- Handle branches that the prompt never advertised, like "we are experiencing higher than normal call volume."
- Decide when to abandon and retry versus wait patiently on hold.
Each of these is a new failure surface. DTMF testing validates voice agents that depend on keypad input for menu navigation, PIN entry, and account number collection, and the outbound direction tests something stricter: whether the external system accepts the digits and the call reaches the intended destination. That is end-to-end behaviour, not a unit check that a digit appeared in a log.
Where outbound navigation actually breaks
Most agents ship with a DTMF tool wired up, a system prompt that says "navigate the menu", and a hope. Here is what we see fail once real traffic starts.
1. Prompt detection vs. voice activity
The agent needs to know that the remote end finished speaking before it presses anything. Early presses land before the IVR is ready to collect digits and get dropped silently. Late presses cause the IVR to replay the menu or time out. This is the same endpointing problem a normal voice agent has, except the "speaker" is a recorded prompt that never pauses to breathe, and your turn-detection model was trained on humans.
2. Transport mode mismatch
The protocol layer still bites. The industry-standard mode supported by Twilio, Plivo and other modern SIP carriers is RFC 2833, which sends DTMF as out-of-band telephone events in RTP. In-band DTMF, where the tone is encoded into the audio itself, degrades badly through compressed codecs. SIP INFO is still used by some legacy enterprise PBXs. If your voice platform is configured for one mode and the downstream IVR expects another, you can see DTMF that looks fine in your local softphone test pass through production and fail silently. Twilio's own docs are explicit that RFC 2833 is what their SIP interface supports for send and receive.
3. Tone cadence
LiveKit's send_dtmf_events tool, for example, sends events sequentially with a 0.3-second delay between each. That default is safe for most IVRs. Some older systems need more; some newer ones will accept faster bursts and some will swallow digits if there is no gap at all. The agent that works against Twilio Studio will not necessarily work against a 20-year-old hospital switchboard.
4. Menu drift
An IVR you tested against last Tuesday is not necessarily the one answering today. Companies reorder options for seasonal promos, add COVID-style announcements, insert surveys, move "representative" from 0 to 9. Your agent's instructions ("press 2 for billing") are now wrong, and the only signal is that calls started failing to route.
5. The human in the middle
Every outbound IVR flow ends when a human picks up, or should. The agent has to switch modes: stop pressing digits, start conversing. We have seen agents that confidently press 0 three more times once the rep says hello.
6. Dead ends and loops
"I didn't understand that." "Please try again." Some IVRs loop you back to the main menu after two bad inputs. Others hang up. Without an explicit retry budget and a loop detector, your agent will burn five minutes in a menu it cannot escape.
What the test suite needs to cover
A real outbound-navigation test suite answers six questions per scenario:
- Did the agent reach the right branch? Not "did it press the right digit" — did it land on the expected prompt or human on the other side.
- How long did it take? Time-to-goal is a scoreable metric. A navigation that works but takes 90 seconds is still broken for most use cases.
- How many digits did it send? Extra, mistimed, or duplicate presses indicate retries that shouldn't have happened.
- What did it do when the menu drifted? Inject a scenario where "press 2 for billing" is now "press 3." Does the agent re-listen, or stubbornly press 2?
- Can it detect a human? Mid-menu, swap the recorded prompt for a human-sounding "hi, this is Sam." The agent should stop sending DTMF and start a conversation.
- Does it exit cleanly? Dead-end menus, three-strike rejections, voicemail. There should be a bounded retry policy and a graceful hangup, not an infinite loop.
Building the harness
The practical problem with testing outbound navigation is that the thing on the other end is a real phone system. You cannot mock it convincingly in a unit test because the whole point is the transport, the codecs, and the timing. You have three options.
Replay against recorded IVRs. Capture audio from the real IVRs you need to navigate and replay them from a synthetic peer. This is cheap and reproducible but goes stale fast; a menu change breaks the fixture and you only notice when the fixture starts lying.
Live calls against staging numbers. Some vendors offer test IVR phone numbers that you can call repeatedly without consequences. Most do not. For partners, you can sometimes negotiate a staging line. This is high-fidelity but limited in coverage.
Simulated callees over real telephony. The most robust option: a synthetic caller platform that acts as the IVR, placing or receiving real phone calls so your agent exercises its full SIP, RTP, and DTMF stack, but with a scripted menu your test owns. You control what gets said, when, and how the menu drifts, while the transport is real.
This is where ecosystem tools are catching up. LiveKit publishes an IVR navigator recipe that listens for spoken menu options and sends DTMF codes with a cooldown, and their Agent Console exposes a DTMF pane that simulates a phone keypad for IVR testing. These are good building blocks for happy-path development. What they don't do is run a suite of scenarios on a schedule, score the outcomes, and tell you when a menu changed under you.
Scoring: what "pass" means for navigation
The temptation is to score IVR navigation by assertion: did the right digit get sent? That misses almost everything that matters. A better score sheet, for every scenario:
| Metric | What it catches |
|---|---|
| Reached-branch accuracy | Agent landed on the expected prompt or the expected human |
| Time to goal | Wall-clock from dial to intended branch |
| Digits sent | Extra presses = a confused agent |
| Barge-in restraint | Did the agent press while the menu was still speaking? |
| Human-detect latency | Seconds between "hello" and the agent stopping DTMF |
| Loop/retry exits | Did the agent give up at the policy bound, or spiral |
All of these need audio-level signals, not just transcript or tool-call logs. A "sent digit 2" in the log is not evidence that the IVR received it; a telephone-event packet made it onto the wire is. If your harness cannot score what actually happened on the audio and the RTP, you are measuring intent, not behaviour.
Regression testing when the menu changes
The hardest part of operating an outbound-navigation agent is that the external systems are moving targets. The most useful signal is a scheduled suite that calls each important downstream IVR daily, records the audio, and flags anything that drifted. When it drifts, you want to:
- Keep the audio of the failed run so you can see what the menu now says.
- Promote that recording into a regression test, so the next version of your agent has to handle both the old and new menu.
- Alert the people who maintain the destination prompts in your agent (the actual "press 2 for billing" strings), because those are now wrong.
This is the pattern behind production call replay for inbound agents, applied in the outbound direction. You turn real failures into repeatable scenarios so they stop being surprises.
Where Roark fits
Roark simulates voice agents by dialing them over real PSTN and WebRTC, with personas that include voices, languages, and background-noise environments. For outbound navigation testing, the same primitives run in reverse: Roark acts as the IVR your agent is calling, with a scripted menu tree you own, and scores the agent on the audio-native signals that matter here: how long it waited before pressing, whether it kept pressing after a human answered, whether it looped. The same 64+ built-in metrics that score inbound calls score these ones, and failed runs become replayable regression tests the next time the menu changes. You wire it into CI over HTTP so a release can't ship until the navigation suite passes. For agents that have to talk to Twilio, Vapi, Retell, LiveKit, Pipecat, Bland, and ElevenLabs stacks, the one-click integrations mean you don't rebuild the telephony substrate just to test a menu.
That substrate is the whole point. GPT-Live's outbound GA is going to put a lot of agents into IVR-navigating situations whose complexity their builders have not internalized yet. The ones that hold up will be the ones whose tests cover the boring stuff: tone cadence, menu drift, human-detect latency, retry bounds. The ones that don't will spend a lot of money pressing 0 on hold.
