Testing voice AI agents for self-triggered echo

On November 3, Soniox released STT v6 with two linked features aimed squarely at voice agents: stream the agent's outgoing text into the recognizer, and the model will filter out the agent's own voice from the transcript. In their own words, it is the first speech model that knows what your agent is saying and filters its own voice, and the pitch is "fewer false interruptions, fewer clarification loops, more reliable conversations."

That is a real improvement, and it is also a reminder of a failure mode that quietly sinks production voice agents: the agent hearing itself, treating the playback as the caller, and derailing the turn. Self-triggered echo is not an edge case. It is what happens on every speakerphone call, in every car, on every SIP leg with a bad handset, every time the AEC warmup window has not finished. If you are not testing for it explicitly, your evals do not cover the thing that breaks the most calls.

What self-triggered echo actually is

The loop is mechanical. The agent speaks, the TTS plays through the caller's speaker, the mic picks it up, acoustic echo cancellation (AEC) subtracts the reference signal, and whatever survives lands in the STT stream as if the caller had said it. The agent, now "hearing" itself, either interrupts its own next sentence, generates a confused reply to its own last sentence, or enters a loop where every turn creates a new turn.

A real LiveKit iOS SDK regression this year made the symptom unmistakable. The reporter described the agent's audio output being picked up by the microphone and interpreted as user speech, producing an infinite echo loop where the agent responds to itself endlessly. That bug was in a client AEC path, but the mechanism is the same everywhere: AEC is imperfect, residual energy is never zero, and TTS audio is spectrally consistent enough that recognizers readily transcribe it.

The reason this keeps happening even with "echo cancellation on by default" across every major framework is that AEC quality is a function of hardware, driver stack, double-talk behavior, and reference-signal alignment, none of which your agent controls. Standard WebRTC AEC is strong in a browser on a laptop. It is weaker over Bluetooth. It is often missing entirely on the SIP leg of a phone call, where there is no client-side echo cancellation because a SIP participant has no client.

Why the Soniox v6 moment matters for testing

Soniox v6 is the clearest public example of a new defense layer: the recognizer gets the agent's text as it goes to TTS, then uses that context to both improve short-reply accuracy and filter residual agent audio that acoustic echo cancellation left behind. Their own docs are careful about what the feature is and is not: agent-voice echo cancellation is designed to clean up what AEC leaves behind, not replace it. You still need AEC. You still need VAD tuning. You still need an interrupt path that cancels TTS within one audio frame.

AssemblyAI shipped similar agent-context carryover earlier in the year, and the clearer differentiator Soniox is highlighting is the model-level residual echo filter. Expect this pattern to generalize fast across the ASR vendors your stack uses.

The testing implication is the inverse of what most teams assume. New defenses do not reduce the need to test for self-triggered echo; they shift the failure surface. The failures that remain are:

• Scenarios the recognizer-side filter cannot see (SIP tandem echoes, speakerphone at high gain, car handsfree with long tail).
• AEC warmup windows. LiveKit's Agents framework explicitly ignores audio during the first few seconds of agent speech on inbound calls via an aec_warmup_duration setting that ignores audio interruptions for the first 3 seconds. That is a deliberate tradeoff that kills barge-in during the warmup.
• False positives. If the filter is too aggressive, it eats legitimate short answers that overlap with the tail of the agent's utterance.
• Wrong reference signal. If your TTS stream and the reference frames the AEC or filter receives are misaligned by even 50ms, cancellation collapses.

The failure modes you actually need coverage for

Every one of these has shown up in production stacks this year. Each is testable before launch.

Each row is a different scenario definition. Each needs a persona (device profile, room acoustics, speech timing) and a success metric. Running the same five happy-path calls will never flush these out.

How to test it before launch

There are three honest ways to generate the acoustic conditions that cause self-triggered echo in the first place.

1. Loopback injection in a text harness. You take a portion of the agent's TTS output and inject it back into the input stream at a configurable gain and delay. This exercises the "agent hears itself" code path cheaply. It will not reproduce real acoustic echo (room response, driver-induced spectral coloring, network tail), and anything you fix at this layer should still get validated end-to-end. But it is fast and runs on every PR. A Deepgram community issue filed this week describes exactly this pattern: using TTS word-level timestamps to tell the STT stream when to ignore agent audio, with the same underlying motivation that drives the Soniox v6 filter.

2. Real-phone simulation over PSTN and WebRTC. The only honest test for echo behavior is a real call. Dial the agent through the actual carrier path, have the simulated caller answer through a loud speaker in a noisy room, and let the AEC, VAD, recognizer, and interrupt path interact under the real audio conditions they will see in production. This is the layer most teams skip because it is awkward to build in-house, and it is also the layer that catches the failures that matter.

3. Production replay. When a real production call fails because of an echo loop or a self-interrupt, you want to turn that recording into a fixture you can replay against every subsequent agent build. If your platform can capture real production calls and replay them against updated agent logic, each real failure becomes a repeatable regression test and your coverage grows as a function of real traffic, not just hypotheses.

This is where Roark's simulation testing fits for most voice-agent teams. Roark dials your agent over real PSTN and WebRTC with configurable personas (voice, pace, background noise, language), scores each call with audio-native metrics, and turns production failures into replayable regression tests through production call replay. The reason to lean on a platform for this and not a unit-test loop is that the whole point of echo testing is the real acoustic path, which cannot be mocked in a Python process without losing the thing you were trying to test.

Scenarios worth building explicitly

A minimal echo-testing suite covers the device axis and the timing axis. Treat each entry as a scenario with a persona and a scripted interaction.

• Speakerphone, high gain, quiet room. The pure self-loop test. Agent should complete a 30 second response without any phantom caller utterances.
• Speakerphone, high gain, 10 dB SNR noise bed. Self-loop plus VAD stress. Agent should not interrupt itself; recognizer should not emit phantom partials on residual echo.
• Car Bluetooth handsfree, highway speed. Long echo tail, wind noise, variable gain. This is the hardest real-world environment and the one most likely to find missed barge-in.
• PSTN leg, G.711, cheap handset. No client-side AEC, carrier echo cancellers of varying quality. This is where most production echo lives.
• AEC warmup barge-in. Caller starts speaking at 500ms into the agent's first utterance. Did the agent yield, or did the warmup window swallow the interruption?
• Double-talk overlap. Caller and agent speak simultaneously for 2 seconds. Measure STT fragmentation, interrupt latency, and whether the agent recovered the caller's intent.
• Device swap mid-call. Caller moves from headset to speaker at turn 3. AEC has to re-converge; measure the window where echo leaks through.
• Short-answer suppression. Caller says "yes" at the tail of the agent's last word. With an aggressive filter or VAD, this gets eaten. The LiveKit short-utterances guide warns that noise cancellation can classify a short "yes" as noise, especially on SIP.

Metrics that actually catch echo failures

Pass/fail on transcript equality will not catch any of this. Echo failures live in the audio and in the timing. The metrics that do catch them:

• Self-loop count. Number of agent-attributed utterances in a call that were triggered by residual-echo-induced "user" turns. Target: zero.
• False interruption rate. Agent cancels its own TTS mid-utterance without a legitimate caller speech event. Target: zero per minute.
• Missed barge-in rate. Caller speech event present in reference audio but no yield from the agent within 500ms. Target: zero.
• Interrupt latency (p95). Time from caller speech onset to agent audio cancel. Target: under 500ms once outside the AEC warmup.
• Short-answer response rate. Fraction of one-word caller responses that the agent acknowledges without a re-prompt. Target: 98%+ across clean, noisy, and SIP conditions.
• Phantom-turn rate. Agent utterances per minute during silence or hold music. Target: zero.

These are audio-native metrics: they cannot be computed from a transcript alone because the transcript already reflects the output of the thing you are evaluating. You need to score the sound of the call, including pronunciation, pace, pauses, interruptions, and vocal stress. That is what Roark's metric suite measures, and why it files issues on failures that pure LLM-as-judge pipelines miss. The full metric catalog covers 64+ built-in checks plus unlimited custom metrics for the behaviors specific to your agent.

What to do this week

1. Pick your worst echo environment (almost always speakerphone over PSTN) and run ten real simulated calls through it. Count self-loops, self-interrupts, missed barge-ins, and short-answer failures.
2. Turn on whatever recognizer-side defense your ASR offers. For Soniox stacks that means upgrading to stt-rt-v6 and setting enable_agent_text_input. For other vendors, check for agent-context features shipped this year.
3. Retune the AEC warmup window. The three-second default from LiveKit is conservative for good reason, but it is also the window most likely to swallow a caller's first interruption. Measure both failure modes before you change it.
4. Build the device-variance scenarios above as named, scheduled simulation runs. The goal is a suite that runs on every agent change and produces a comparable score each time.
5. Turn every production echo failure into a replayable fixture. The next incident should fail the test suite before it reaches production.

Self-triggered echo is one of the oldest problems in voice technology and one of the easiest to pretend you have handled because "AEC is on." The ASR-side defenses shipping this quarter are genuinely useful. They do not eliminate the need to test for the thing they are defending against; they move the goalposts. Build the suite, measure the metrics, replay the failures, and your agent stops responding to itself.