Beyond rate limits: scaling access to Codex and Sora

Scaling a product that millions of people suddenly want to use is a different problem than building the product itself, and OpenAI's latest engineering writeup makes that distinction pretty clear. The company describes moving beyond simple rate limits toward a real-time access system that blends usage tracking, credits, and dynamic limits to keep Codex and Sora running smoothly under heavy load.

The old approach, static rate limits per user or per key, works fine when demand is predictable. It falls apart when a coding agent like Codex or a video generator like Sora gets adopted faster than anyone modeled for. OpenAI says the new system tracks usage continuously and adjusts access in something close to real time, rather than just capping requests at a fixed number and calling it done.

Credits are the other half of the equation. Instead of a binary allowed-or-blocked gate, the system apparently lets access flex based on consumption patterns, so a user or workload that's been light on resources one hour can draw more the next. That's a meaningfully different philosophy than a hard quota reset every 24 hours, and it suggests OpenAI is trying to smooth out the bursty, unpredictable way people actually use generative tools, especially something as compute-hungry as Sora's video generation.

None of this is glamorous work, but it's the kind of infrastructure that determines whether a popular AI product feels reliable or constantly throttled. Anyone who's hit a mysterious "too many requests" error on Codex or watched a Sora render queue stall knows the stakes here aren't abstract. Getting this layer right is arguably as important to user experience as the models themselves.