Microsoft shipped a streaming transcription model on October 1 that returns first hypotheses in just over 100 milliseconds and ranks first on Artificial Analysis for both partial and final text. Three weeks later Mistral dropped Voxtral 3 with its own real-time transcription, TTS, and voice agents API. In between, Decagon launched Voice 3, which covers 70+ languages and can switch language mid-sentence. The whole industry just made partial transcripts faster.
That is good news for latency and bad news for anyone who built an agent that reacts to those partials. The faster the first hypothesis, the less acoustic context the model had when it emitted it, and the more it will revise before it finalizes. If your pipeline fires a tool call, starts a preemptive LLM generation, cancels TTS, or triggers a turn-end on a partial, you have a test problem. And it is not one your text-level unit tests can see.
What a "partial" actually is, and why it churns
Streaming STT exposes two classes of transcript. In Deepgram's API, interim results are marked is_final: false and finalized text is marked is_final: true; LiveKit normalizes those to INTERIM_TRANSCRIPT and FINAL_TRANSCRIPT, with a third PREFLIGHT_TRANSCRIPT event for providers like Deepgram Flux and Cartesia ink-2 that emit eager end-of-turn predictions. The interim is a running best guess. The final locks text so downstream can act on it.
Why do interims revise? Streaming decoders extend a beam search as audio arrives. The acoustic score of what came before can change in light of what comes after, and the top beam can swap. One USPTO filing on streaming ASR describes the mechanism directly: a first partial comes from the top-scored hypothesis of a first beam search, a second partial comes from a second beam search extended with more audio, and the two need not agree. For agents, that means every interim is a conditional statement: "given what I had heard so far, my best guess was X." Reacting to X before the final is a bet.
Where agents actually eat partials in production
Modern voice-agent stacks eat partials in several places, each one earning real latency and each one a potential failure site.
Preemptive generation. LiveKit Agents enables preemptive_generation by default and ships a PREFLIGHT_TRANSCRIPT path for providers that emit eager end-of-turn signals. The agent starts the LLM call on the preflight so the first TTS byte can arrive sooner.
Prefetch tool calls. LiveKit's latency guides recommend classifying intent in on_user_turn_completed and firing the likely tool call as an asyncio task, then awaiting the future when the LLM asks for the tool, worth a documented 200 to 400ms at p95.
Interim-driven turn start. Pipecat's TranscriptionUserTurnStartStrategy defaults use_interim=True, so an interim transcription frame alone can declare a user turn has started.
Backchannel suppression and barge-in. Interim text is used to decide whether a short noise is a backchannel ("uh-huh") or a real interruption that should cancel TTS.
Every one of these is correct engineering. Every one of these is also a bet that today's partial is close enough to tomorrow's final.
The failure modes you need to test for
This is where tests earn their keep. Everything below is a real pattern surfaced in the last few months in the agent-framework repos themselves. These are the bugs the people writing the frameworks are fixing on main. Your pipeline has them unless you have proven otherwise.
Preemptive generation that doesn't cancel on resumed speech. A LiveKit issue filed three days ago describes exactly this: with preemptive_generation on and an STT that emits PREFLIGHT_TRANSCRIPT, a preemptive reply starts on the preflight. When the user keeps talking, Flux sends TurnResumed, which the plugin forwards as INTERIM_TRANSCRIPT. Nothing in AudioRecognition or AgentActivity reacts to that interim, so the preemptive generation keeps running until the next preflight replaces it. The stale reply never plays, but the cost is wasted LLM tokens, wasted TTS, and (if preemptive_tts=True) audio that has to be silently discarded. Multiply by thousands of calls.
Final transcripts dropped by their own turn's interruption. A Pipecat issue from last month documents a frame-ordering bug: when an interim-driven turn start fires, Pipecat broadcasts an InterruptionFrame that flushes queued frames. If a TranscriptionFrame was sitting in a queue behind the start frame, it gets deleted by the interruption of its own turn. The documented result: "The turn opens with no text (or with only part of what the user said), the LLM runs on that or never runs, and the user gets silence or a wrong answer". That is a user asking "what is my balance" and getting answered as if they said "one".
Turn-start on a spurious interim. VAD oversensitivity triggers on background noise, and the STT returns false short-word transcriptions like "okay" or "yes". If use_interim=True is the turn-start trigger, the agent commits to a turn that nobody started.
Trigger words in a revised partial. The caller says "transfer me to… billing, actually sales." An intent classifier fires prefetch(route="billing") on the first interim. The final is "transfer me to sales." Your tool returned a billing agent, your prefetched future is wrong, and either your wrapper throws the result away (wasted RTT) or the LLM happily uses the stale output.
Code-switch mid-sentence. Decagon Voice 3 can switch language mid-sentence across 70+ languages, and Microsoft's MAI-Transcribe-2-Streaming covers 60 with automatic language detection. The interim can be in one language and the final in another. If any part of your pipeline keyed off interim-language, that key is stale by final time.
Why text-level unit tests do not find any of this
The bugs above do not live in your prompt, your tool schema, or your LLM. They live in the choreography between audio arrival, VAD events, interim deltas, final transcripts, LLM streaming, and TTS cancellation. A text-level eval that feeds "transfer me to sales" into the agent and checks the tool call will pass every one of these failures, because the audio timing that produced them never existed in the test.
To reproduce them you need:
- Real audio with real timing, not scripted text.
- A real STT in the loop, because the churn pattern is model-specific.
- The actual framework surface (preemptive generation, turn strategy, interrupt handling) exercised end-to-end.
- Observability on the audio side: did TTS actually cancel, was there dead air, did the agent speak over the user. Not just the final transcript.
That is a simulation harness, not a unit-test fixture.
What a partial-churn test suite looks like
Call this scenario family "the second-guess" and build a battery around it. The scenarios that catch the most bugs are mundane:
- Self-correction. The caller starts a sentence, catches themselves, and corrects within 400ms. "Transfer me to billing, I mean sales." "The last four of my card are one two three four. Wait, four three two one." Expect: the final tool call matches the corrected text, no stale prefetch survives, no premature confirmation.
- Mid-utterance filler. "I need, uh, I need to, uh, reschedule for Thursday." Interims flicker as fillers come and go. Expect: a single turn, no spurious turn-start on the filler, no preemptive generation that stalls.
- Code-switch inside the turn. Caller starts in Spanish, switches to English mid-sentence, or vice versa. Expect: language-dependent logic settles on the final language, not the interim.
- Barge-in during preemptive TTS. Agent starts speaking a preemptive reply; caller barges in with new content. Expect: the preemptive TTS cancels, the preemptive LLM cancels, the new turn uses the new text.
- Noisy interim. Background chatter triggers a short false partial while the agent is speaking. Expect: no turn start, no wasted LLM call, agent keeps talking.
- Keypad entry and speech on the same tick. Caller says a word and hits a DTMF key within the same frame. Expect: both land in the committed turn.
Every one of those scenarios needs a measurable expectation on the audio timeline, not on the final transcript alone. "No stale generation" is a timing claim. "No dead air greater than 1.5 seconds" is a timing claim. "TTS cancelled within 200ms of barge-in" is a timing claim.
Metrics worth watching
For every call in the suite, score at least:
- Partial churn rate. Edit distance between the latest interim and the final for each utterance, averaged over the call. High churn with no agent reaction is harmless. High churn with action taken on the early partial is a bug waiting to produce a user-visible failure.
- Interim-to-final delay. How long after the last interim does the final arrive? That window is where premature actions do damage.
- Wasted preemptive generations. Count LLM streams that were started on a preflight or interim and later discarded. Each one is dollars and latency with no return.
- Barge-in response time. From the first frame of caller audio over the agent's TTS to the moment TTS actually stopped. The threshold a caller feels is around 200ms.
- Dead air. Any gap above 1.5s between the end of the caller's turn and the start of the agent's response, with the final transcript already in hand.
Transcript-only tooling can see the first and partially the second. The rest need to score the sound of the call, not the sound-of-the-call-as-rendered-by-a-text-field.
How Roark fits
Roark exists for exactly this test problem. It runs simulations by dialing your agent over real phone calls (PSTN and WebRTC), driving the conversation with a persona defined by voice, language, accent, pace, emotional register, and background-noise environment. Simulations run in 45 languages and accents, so code-switch and multilingual churn scenarios are not an afterthought. The partial-churn scenarios above become scenarios you codify once and run on every build, triggered automatically over HTTP so a bad main-branch commit cannot pass CI.
Because Roark's metrics are audio-native, they score what the call sounded like, not just what the final transcript said: pronunciation, pace, pauses, and interruptions. Barge-in latency, dead-air gaps, and bot-over-user overlap are first-class metrics, not things you reconstruct from logs after the fact. Sixty-four built-in metrics ship by default and anything missing becomes a custom metric.
Post-launch, every production call is scored against the same metric suite, failures are filed as issues automatically, and production call replay lets you take a real failed call and turn it into a repeatable regression test. The LiveKit preflight bug above and the Pipecat frame-ordering bug above both become single scenarios in your suite, not whitepapers you re-read when the next model vendor pushes an update. Roark ships one-click integrations for LiveKit, Pipecat, Vapi, Retell, Bland, and ElevenLabs, so wiring it against your existing stack is a config change, not a rebuild.
The point
Streaming ASR partials are not going to get more stable. The incentives all push the other way: faster first hypotheses sell models, and every provider that ships this quarter is trying to beat the last. The agent frameworks respond by giving you more ways to react to those partials: preflight transcripts, preemptive generation, interim-driven turn start, prefetch classifiers. Each of those is a lever you can pull for a couple of hundred milliseconds of perceived latency, and each is a scenario your test suite needs to cover.
If your QA process stops at "did the final tool call match the final text," your agent is probably doing one of the things in this post somewhere. And you will not see it until a caller does.
