Testing voice agents that quote prices

Something quietly shifted this summer. Voice agents built on OpenAI's GPT-Realtime aren't just answering questions anymore, they're holding real sales conversations, and the retailers deploying them are getting the numbers they were promised. Japanese home-appliance chain Yamada Denki ran a public campaign where roughly 30,000 shoppers talked to a 24/7 multilingual voice agent trained on their sales associates' expertise, and 92% of post-conversation surveys came back positive. The average enterprise now runs 13 agents in production, up from 5 in early 2025, per Salesforce's Agentic Enterprise Index. Voice is where a lot of that growth is happening.

That growth comes with a specific new failure mode: an agent that misquotes a price, invents a discount, or agrees to a policy that doesn't exist. On a chat surface, a bad quote is a screenshot. On a phone call, it's a verbal commitment, and regulators and small-claims tribunals have already made clear that "the chatbot said it" is not a defense. This post is about how to test agents that quote prices before they cost you money, and how to catch the ones that quote wrong once they're live.

When "the agent said it" becomes the company's problem

The precedent every voice-AI product lead should know is Moffatt v. Air Canada. A grieving passenger asked Air Canada's website chatbot about bereavement fares. The bot confidently described a retroactive-refund policy that didn't exist, and the airline refused to honor it. The British Columbia Civil Resolution Tribunal ruled in February 2024 that Air Canada was liable, treating the error as negligent misrepresentation and ordering the airline to pay the fare difference. Air Canada's defense was that the chatbot was "a separate legal entity" responsible for its own actions. The tribunal called that submission "remarkable" and rejected it flatly, on the reasoning that the chatbot was still just part of Air Canada's website and the airline was responsible for all the information on that site.

That reasoning doesn't narrow when you swap chat for voice. If anything, it widens: a voice quote lands faster, sticks harder, and is harder to walk back in the moment. The caller hangs up with a number in their head. The recording is either your defense or the plaintiff's exhibit.

Layer on top the fact that, since 2 August 2026, Article 50 of the EU AI Act requires that voice agents interacting with people in the EU disclose that they are AI, with penalties reaching €15 million or 3% of global turnover. That doesn't limit price-quote liability, but it does mean the caller now expects to be talking to a machine, and courts will treat the transcript accordingly.

What "wrong price" actually means on a call

Wrong is not just a made-up number. On a live sales or support call, the failure modes are:

• Fabrication. The agent invents a discount, promo, or SKU. Air Canada style.
• Staleness. The agent quotes a real price that expired last quarter, or a promotion the marketing team ended.
• Bundling drift. The agent adds a feature to the tier that the tier doesn't include, or drops one the tier does.
• Unit confusion. The agent quotes a monthly price when the caller asked for annual, or drops a currency, or forgets tax and shipping.
• Policy hallucination. The agent describes a refund window, a warranty term, or a cancellation policy that doesn't match the contract.
• Silent failure on a tool call. The pricing service timed out and the model filled in the blank rather than escalating.
• Language-switch drift. In a code-switched handoff, the agent quotes the price using yesterday's exchange rate, or misparses "quinze" as "cinquante."

Each of these leaves an audio recording that says the same word to the caller: "your price is X." Each of them is testable if you know what "right" looks like.

Define correct before you test for wrong

The most common failure I see is teams jumping to "we need a hallucination check" without a written source of truth. You cannot detect a wrong quote without a machine-readable version of the right quote.

For a sales agent, that usually means three artifacts, kept in one place and versioned:

1. A price catalog with every SKU, tier, currency, and effective date range.
2. A promotions engine that enumerates every discount, its eligibility rules, and its expiry.
3. A policy document for anything the agent is allowed to speak to (refund windows, warranty, cancellation, delivery, escalation thresholds).

If it isn't in those three, the agent shouldn't be quoting it. If the agent does quote it, that's a test failure, not a rounding error. Getting this right at the artifact level is the biggest quality lever most teams have and the one they most often skip.

The test plan before launch

Once you have a source of truth, simulation is straightforward. You put your agent on a real phone line, and you dial it with a synthetic caller that pushes on every place a wrong quote can appear.

The scenario library that pays for itself:

• The direct ask. "How much is the Pro plan?" Checked against the catalog to the cent.
• The comparison. "What's the difference between Standard and Pro?" Verifies bundling, not just top-line price.
• The negotiation. "Can you do better than that?" Checks that the agent doesn't manufacture a discount to close the call.
• The stale-promo trap. The persona name-drops a discount code that expired last month; the agent must refuse it.
• The unit trap. "So that's per year, right?" The agent should correct or confirm without drifting.
• The currency swap. Caller asks in one currency, agent quotes in another; verify the conversion or the escalation path.
• The tool-call outage. The pricing service returns a 500; the agent should decline to quote and offer a callback, not improvise.
• The policy pressure. "But your website said 90 days." The agent must state the actual policy and not concede.
• The bilingual quote. Start in English, switch to Spanish mid-call, then ask for the price again.

Personas matter as much as scenarios. A skeptical enterprise buyer, a first-time consumer, an angry return-seeker, and a bilingual code-switcher will pull different failure modes out of the same agent. Roark dials agents over real telephony with personas that vary voice, language, accent, speech pace, emotional register, and background noise, across 45 languages and accents. You want your scenarios crossed with those axes, not run against a single default caller reading a script.

Every scenario needs an explicit expected answer: the exact price, tier, term, and any conditions. If your harness only checks "did the agent respond fluently," you will ship an agent that fluently misquotes.

Metrics that catch price errors

Overall pass/fail is not enough. For agents that quote, four families of metric earn their place on the dashboard:

• Quote fidelity. Did the number the agent said match the number in the catalog, for the SKU the caller asked about, on today's date?
• Policy conformance. Did the terms the agent described match the current policy document? Word-for-word is too strict; semantically equivalent is the bar.
• Uncertainty-to-escalation ratio. When the agent didn't know, did it escalate or did it improvise? Improvisation is the single biggest source of Air Canada moments.
• Audio-level signals. Voice-only cues (rushed pace, filled pauses, a small "um" before the number) sometimes correlate with the model's own uncertainty. Scoring these on the audio, not just the transcript, catches quote errors that a text-only eval misses.

That last one matters because a lot of the eval tooling on the market grades a transcript and stops. Voice agents don't read scripts, they speak them, and the quality of the call is not just what was said but how. Roark scores calls with audio-native models that grade pronunciation, emotion, vocal stress, pace and pauses, and interruptions on top of transcript-level checks. A quote delivered at the wrong pace with a hedged tone is a quality signal even when the number happens to be right, and it's exactly the kind of signal that gives you a chance to catch a wobble before a customer files a complaint.

Score every production call, not a sample

Sampling is where price-quote testing usually breaks in production. You cannot QA one call in a hundred and then argue in front of a tribunal that the other 99 were fine. Every call your agent handles that involves a price, a promotion, or a policy statement needs to be scored against the source of truth, automatically.

That means:

• Every call goes through the metric suite the moment it ends, not "when someone flags it."
• A quote that doesn't match the catalog files an issue on the specific turn where it happened, with timestamp and transcript context.
• The dashboard shows quote-fidelity and policy-conformance by SKU, by persona, and by day, so a regression from a prompt change, a model swap, or a knowledge-base update is visible within a shift.

That last point is worth pausing on. GPT-Realtime is now generally available with new voices, remote MCP servers, and SIP calling built in, per OpenAI's own announcement. Model swaps and tool-graph changes are going to be normal, not exceptional, for the next couple of years. If you don't have every production call automatically graded against your source of truth, you will not know when the swap silently changed how your agent quotes prices, until the third or fourth caller says something on Twitter or in a tribunal.

Turn a failing quote into a regression test

The last piece is the loop. When a live call misquotes a price, you don't want to fix the prompt and hope. You want the exact call, with the exact caller audio, dialed again against the fixed agent before you redeploy.

That's what production call replay is for: capture the real call, replay it against the updated agent logic, and confirm the fix holds. Do it every time a wrong quote surfaces, and the second occurrence never happens. Do it as part of CI on any prompt change, model swap, or catalog update via the Roark SDK, and you stop shipping the same class of error twice.

A price-quote regression suite that grows every time production embarrasses you is boring, which is the point. Boring compliance is the goal.

The short version

• Voice agents on the sales floor are past the demo stage. When they quote a price, the business owns it.
• Wrong is not just fabrication. It's stale, mis-bundled, mis-unit, and confidently-improvised-under-outage.
• Define correct before you test for wrong. Catalog, promotions engine, policy document, all machine-readable and versioned.
• Simulate with personas and scenarios that pressure every failure mode. Score the audio, not just the transcript.
• Score every production call. Sampling is not a defense in front of a regulator.
• Replay failing calls against the fixed agent before you redeploy.

The agents that will earn their place on the sales floor over the next year are the ones whose builders treat every quote as a test case. The rest will be tomorrow's tribunal decisions.