How OpenAI delivers Low Latency Voice AI for 900M Users
ByteByteGo Newsletter
OpenAI built a custom WebRTC architecture to deliver voice AI to 900 million weekly users by splitting packet routing into a stateless relay layer and a stateful transceiver layer, avoiding Kubernetes deployment issues with port exhaustion and state stickiness. The system encodes routing metadata into the ICE ufrag field to route the first packet correctly without database lookups, then uses a geographically distributed relay fleet to minimize latency. The architecture serves one-to-one conversations between users and AI models more efficiently than standard SFU (Selective Forwarding Unit) approaches used for multiparty calls.
Why it matters
OpenAI serves WebRTC voice AI to 900M weekly users by splitting the stack into a stateless edge relay for packet routing and a stateful transceiver, sidestepping port-exhaustion and state-stickiness problems on Kubernetes.
