What does production readiness mean for a voice AI agent?
A voice agent is production-ready when it can end a call safely without ever needing to be clever. That means four things are tested against real audio, not transcripts: it recognises the caller's actual speech, it handles being interrupted, it has somewhere to hand off to a human, and every action it takes against a real system has a limit it cannot exceed.
Test against audio, not transcripts
The most common readiness mistake is validating the conversation on the transcript. A transcript makes a bad call look like a good one: it hides the pause where the caller thought they had been cut off, it silently fixes the number they said badly, and it flatters the agent by removing the hesitation the caller actually heard. Two people reading the same transcript will disagree about whether the call went well, and the audio would have settled it in four seconds.
So the test set is audio, recorded with the conditions the real callers will bring: a speakerphone in a noisy room, a caller who reads their reference number too fast, a mix of Indonesian and English inside one sentence. Indonesian customer traffic switches languages mid-utterance routinely, and a transcriber tuned for one language will lose the words around the switch rather than the switch itself.
The four checks
One: does it recognise the caller's actual speech, including the numbers? A mishearing that changes a total is a support incident with a different name. Two: what happens when the caller interrupts? If the agent talks over them it has to stop mid-word and re-attend, and if it does not, the caller will conclude the system is not listening and start repeating themselves.
Three: where does it hand off? A voice agent without a human path is a wall, and callers escalate to the wall. The handoff has to be reachable by saying a plain word, and it has to carry context, so the human does not make the caller repeat everything the agent already heard. Four: what can it do to a real system, and what stops it? A refund, a block, a change of address — each needs a limit the agent cannot reason its way past.
Then run UAT as if it were a launch
UAT for a voice agent is not a bug hunt; it is a rehearsal. I run it as real scenarios with the limits attached: the difficult cancellation, the angry caller, the one who does not have their details, the one who asks something the agent is explicitly not allowed to do. What I am measuring is whether the system's edges are where the business said they were, and whether the human on the other end of the handoff can finish the job.
That is the part of my work that came from operations rather than from research, and it is the reason I treat launch readiness as a testing discipline rather than a mood. The workflow across transcription, model, speech, and business tools either survives contact with a real caller or it does not, and the only way to find out before the customers do is to call it yourself, in the noise.
Filbert Henrico is a Technical Customer Success Lead at FINETIKS in Tangerang since June 2025, with seven-plus years of customer success and technical sales across Southeast Asian fintech, including Xendit, Peakflo, and Brick. His focus includes API integrations, agentic AI, workflow automation, conversational AI, UAT, and launch readiness. Bachelor of Marketing, Victoria University, Bandar Sunway, Malaysia, 2016–2019.
Stuck on something similar?
These pages describe how I approach the work, not a fixed methodology. If you are scoping something similar and want a second opinion on the trade-offs, send me the problem in the messy state it is actually in.
Keep reading
All questionsWhich AI use case should a team build first?
Build the one where a human already does the task every day, the correct answer is easy to recognise, and a wrong answer is cheap to undo. Those three conditions are what let you finish the project: the workflow gives you data, the recognisable answer gives you a test, and the cheap undo gives you permission to ship.
How do you automate customer support without breaking escalation?
Map every escalation rule a human currently applies, and automate only the work that sits above the rules. Escalation is not a feature you add to an automated flow — it is the boundary of the flow, so the boundary has to exist and be written down before anything is automated. Teams that automate first and define escalation later end up with a system that handles everything confidently, including the things it must not touch.
How should a fintech team evaluate an AI vendor?
Evaluate the vendor's failure behaviour, not their accuracy claim. Ask for the worst ten calls they can show you, what the system did when it was wrong, and who found out. A vendor who can answer that has built something; a vendor who answers with a benchmark number has built a demo.