← Overview
AI
TV
noise
caller
coworker
02 · Voice AI

Realtime voice that never talks over you.

From the phone in a hand to the model and back, supervised, the whole way on one line.

OpenAI RealtimeGemini LiveElevenLabsAnthropicDeepgram

Bring your own model

Swap providers without rewiring. The same call runs on every major realtime voice model, and you can change your mind in production.

Barge-in that actually works

The moment a caller starts speaking, the agent stops. Instantly, and the same way on every provider. No awkward overlaps, no waiting for a sentence to finish.

Perception, on the line

It tells the caller from the TV, a coworker, or background noise, and decides on its own whether to mute, listen through, or clean up the audio. The caller just gets understood.

Agents that do things

Tools, memory and actions run alongside the audio, so the agent can act mid-call instead of only talking. One pipe carries speech up, audio down, and everything in between.

24,000★concurrent calls on one 8-vCPU server, every packet delivered
52%★CPU at that load (pion: 83% at 22,000)
Voice AI · FAQ

Questions builders ask about realtime voice that never talks over you..

LiveKit, Daily and Chime are SFUs built for many-party video calls, priced per participant-minute as if every call were a meeting room. Twilio Media Streams is G.711 over WebSocket: carrier reach, but no codec choice and no fast interruption path. We are built for a caller talking to an AI agent, with the codec, transport and speech detection owned end to end. If you already run on one of them, we ship adapters for Twilio, Vapi, LiveKit, Daily and Chime, so you can move one call path at a time.

Interruption detection lives in the transport, not in each provider's SDK. The moment caller audio crosses the speech-gate threshold we hold the agent's output and send a cancel, on the same code path whether the model behind it is OpenAI Realtime, Gemini Live or a self-hosted llama.cpp. What you hear after you start talking is one packet, not a sentence.

The cancel is one packet on the QUIC line, so the agent stops at once. How soon its new reply starts depends mostly on the upstream model, not the transport. We measure the full histograms on your own traffic during the pilot.

OpenAI Realtime, Gemini Live, ElevenLabs Conversational AI, and a self-hosted llama.cpp worker out of the box. For pipelined stacks (separate STT/LLM/TTS), wire any combination: Deepgram or AssemblyAI in, an LLM such as Anthropic Claude or GPT-4o in the middle, Cartesia or ElevenLabs out. The substrate handles the joins.

Yes. Next to the providers above, we serve open speech models on our own C++ engine, on one RTX A5000 ($0.27/hr). In our own run of Coval's open benchmark method (not an entry on the Coval leaderboard), Qwen3-TTS 1.7B starts to speak in 53 ms p50 / 76 ms p95. The same model on the board takes 63 / 91 ms on Nari Labs and 104 / 132 ms on Baseten. Our text-to-speech run used Coval's public prompt set; the board uses a private one. For speech to text on our streaming route, the final transcript arrives 42 / 62 ms after the caller stops with Parakeet v3 on one H100 (49 / 75 ms on an RTX A5000; Together serves the same model at 49 / 139 ms on the board), and 64 / 122 ms with Whisper large-v3 on one H100 (103 / 206 ms on an A5000; Baseten 112 / 248, Together 108 / 232). Board values were read on 14 October 2026, and our client was about 10 ms from the GPU. Qwen3-ASR 1.7B has a 3.67 % word error rate over the seven English test sets that Coval uses, and one card holds 64 live Qwen3-ASR streams with partial transcripts. One card carries 64 concurrent Qwen3-TTS calls. For Hindi, IndicConformer 600M gives the transcript in 76 / 103 ms (client near the GPU) at 13.1 % WER on FLEURS, against 14.7 % for Qwen3-ASR and 29.0 % for Whisper large-v3.

Most of the time from the caller's first voiced frame to the moment agent audio stops is the speech-gate hangover: we wait one frame to confirm it's not a sneeze. The cancel itself is one packet over QUIC. We measure it on your own calls during the pilot.

Speaker-agnostic on first hear: we enroll the dominant voice in the first ~3 seconds of voiced speech, then use it to gate. Approach is d-vector-based, in the lineage of WeSpeaker / 3D-Speaker. No upfront enrollment flow. For known-caller scenarios (your own employees), you can pre-enroll a print and ship it as a tenant secret.

VAD is a Silero V5-style small recurrent net, frame-rate, integrated into the speech-gate so it can feed barge-in directly. Environment classification (caller / TV / music / coworker / car) sits on top of YAMNet-family acoustic embeddings: that's what tells the agent "this is noise, not interruption" so it ignores a YouTube video next to the mic.

QUIC stream independence keeps audio rendering. The agent's next reply takes a beat longer to arrive but the conversation doesn't lock up. We expose per-call jitter/inter-arrival in the post-call summary so you can see the tail shape for your network.

Yes. On-prem deployments ship with the inference worker co-located: audio never leaves your VPC. The substrate handles transport + perception; the model is whatever binary you point it at.

Put it on one line.

FirstByte ships every modality on the same transport.