Meta's AI Storage Blueprint at Scale

Everyone talks about GPUs when they talk about AI infrastructure, but Meta's engineers just published a reminder that memory and storage are the unglamorous chokepoints nobody wants to discuss at dinner parties. Compute performance has tripled roughly every two years. Storage and networking haven't kept pace. And when hundreds of thousands of GPUs need to synchronize every few training steps, one slow storage fetch on a single machine drags the entire fleet down with it.

The old BLOB-storage stack at Meta was built for Facebook and Instagram, not for Llama-scale training runs. It had layers stacked on layers, each with its own metadata store, and a single getObject call could bounce across name layers, volume layers, and container layers before finding actual bytes — sometimes racking up hundreds of milliseconds in the process, occasionally crossing regions along the way. That was fine for HDD-era web traffic. It's a disaster when GPUs expect flash-speed, bounded-latency responses and stall out waiting on metadata lookups that have nothing to do with the data itself.

So Meta ripped out the middle. They collapsed the scattered metadata into one flat schema backed on ZippyDB, cutting lookups down to constant time. They killed the dataplane proxy entirely and shipped a fat client SDK that streams bytes straight from Tectonic storage servers to GPU hosts, which also happens to save power — a real constraint now that datacenters are limited by kilowatts, not square footage. And they made the whole stack deployable regionally, so storage physically sits next to the GPUs doing the training instead of routing traffic across the globe.

Even with a leaner foundation, spikes remained a problem — hundreds of GPUs hammering the same checkpoint file simultaneously, or GPU restarts triggering sudden traffic surges. Meta's fix borrows from a system called Owl: spare memory on GPU hosts becomes a distributed cache, hitting an 80% hit rate in practice, paired with a metadata cache that resolves paths in one to two milliseconds. Add hedged reads for laggard nodes and dynamic concurrency control for egress spikes, and the stall problem mostly disappears.

The more interesting shift, though, is what Meta did for researchers rather than GPUs. Because datasets are increasingly geo-distributed while GPUs are scarce and clustered elsewhere, researchers were burning hours just copying snapshots between regions before a job could even start. Meta's answer borrows straight from operating-system design: treat global storage like a disk, and GPU-host memory and flash like L1 and L2 CPU caches, with a regional flash tier acting as L3. Data hydrates on demand instead of requiring manual copies, turning what used to be an hours-long ingestion ritual into something closer to a background prefetch.