Speeding up agentic workflows with WebSockets in the Responses API

Agent loops are chatty. Every time Codex takes a step, plans a move, or checks a tool result, it's firing another request at the model. Do that over standard HTTP and you're paying a tax on every single call: new connection setup, repeated headers, redundant context getting re-sent again and again. OpenAI's engineers looked at that tax bill for the Responses API and decided it was too high.

Their fix was to open a WebSocket once per session and keep it alive for the life of the agent loop, rather than tearing down and rebuilding a connection for every turn. On top of that, they added connection-scoped caching, so information tied to that specific session sticks around locally instead of getting shipped back and forth with each request. The combination cuts out a meaningful chunk of the round-trip overhead that piles up when an agent is taking dozens or hundreds of steps to finish a task.

The numbers matter here because Codex isn't a chatbot answering one question. It's closer to a loop that reasons, calls tools, reads output, and reasons again, sometimes for minutes at a stretch. Shaving latency off each of those steps compounds fast. A few hundred milliseconds saved per turn turns into real, noticeable speedups once you're running an agent through fifty or a hundred iterations on a coding task.

This is also a quiet admission that the request-response model, the thing the web has run on for three decades, isn't a great fit for how agentic systems actually behave. Agents don't ask one question and walk away. They loop, they backtrack, they hold state. Building infrastructure that assumes a single stateless exchange per interaction was always going to strain under that pattern, and OpenAI's move toward persistent connections is a tacit acknowledgment of that mismatch.