Voice agent p95 latency: the slow calls your average hides
By Christophe Paka · Facts checked on 7 November 2026
Your test calls sound crisp. The average turn time looks fine. Then the owner phones you: callers say the agent hangs. You play a dozen calls and hear nothing wrong.
Callers never hear your average. They hear the one slow reply. That is why you track voice agent p95 latency, the slow end, not only the middle. The figure below shows the usual turn, the slow tail, and one sum you must never do. A turn is one exchange: the caller stops talking, the agent answers.
How to read this figure
Find p50 on the curve. Line up a week of turns from fastest to slowest. p50 is the median, the turn in the middle of that line: half are faster, half slower. It tells you how the usual reply feels. One independent test site, Openbenchmarks, measured Vapi's p50 at 1,558 milliseconds, about one and a half seconds, to the agent's first sound.
Move right to p95. p95 is the time that 95 of 100 turns stay under. The turns to its right are the tail, the slow end that callers remember and hang up on. In that test it sits at 2,008 milliseconds, about two seconds. For scale, people answer each other in about a fifth of a second.
Count the dark squares as turns. The five dark squares are the 5 in 100 replies slower than your p95. The figure labels them calls, but p95 counts turns, and a call has many turns. So more than 5 in 100 calls meet at least one slow reply. Read the figure's 25 callers a week as the least it can be.
Read the two bars at the foot. The bars carry no numbers; they show a shape. The top bar adds up each stage's own p95: transcriber, model, voice. The lower bar is the p95 of the whole turn, measured. The top bar is longer, because the stages do not all stall on the same turn.
Key takeaways
- Report p50 and p95 of whole turns from your own call recordings, side by side, after every change.
- Never add the p95 of each stage; measure the p95 of the whole turn instead.
- Remember p95 counts turns: with many turns per call, more than 5 in 100 callers meet the tail.
Try this: measure your own p50 and p95 from recordings
You need an assistant on a real phone number, the dashboard, a spreadsheet and Audacity, a free audio editor that shows sound as a wave.
- In the assistant's Advanced tab, under Recording & Artifacts, turn on Audio Recording in a stereo format and publish. Stereo puts you and the agent on separate tracks.
- Ring the agent 20 times across the day. Ask the same three things each time: a short question, one that needs a lookup, and one where you read out a number.
- Download each recording from the call log and open it in Audacity, with the selection shown in milliseconds.
- Select from your last sound to the agent's first sound. Type each gap into column A of a spreadsheet.
- Use
=MEDIAN(A:A)for p50 and=PERCENTILE.INC(A:A,0.95)for p95. Skip the plain average: one slow outlier drags it where no caller was.
Expected result: Two numbers for your own line, from 60 real gaps. Twenty calls is the fewest that makes a p95 mean anything. Promise a client this measured p95, never a vendor's figure. To cut it, start with the latency budget of one turn.
Questions readers ask
What is a good latency for a voice agent?
There is no safe vendor number to promise. People answer each other in about a fifth of a second, and every voice agent is slower. Vapi's default arithmetic adds to more than two seconds a turn, and one independent test measured less. Measure your own p50 and p95 from recordings, and promise that.
Can I add up each provider's p95 to get my total?
No. Percentiles do not add. The transcriber, model and voice rarely all stall on the same turn, so the sum is a total almost no call produces. No vendor publishes p95 for each stage, so the only p95 you can trust is the one you measure on whole turns.
Why do callers complain when my average latency is fine?
The average hides the tail. If 19 replies in 20 are crisp and the twentieth hangs, the caller who got that reply complains. A call holds many replies, so the slow ones reach more callers than the 5 in 100 suggests. Track p95 to see them.
Sources: Vapi voice agent latency (Openbenchmarks); Understanding latency (Vapi docs).
This figure is from Vapi Made Crystal Clear, chapter 4.
The chapter builds the idea step by step, with recipes you can follow, a Try it to check yourself, and every source.
Get the book
