AI’s hidden storage bill: What inference leaves behind
MIT Technology Review Tim Rausch
AI doesn’t just burn compute. Every prompt and video can leave a storage trail behind. That hidden bill can grow faster than the GPU line finance is watching.
Based on reporting by MIT Technology Review, Tim Rausch — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
When executives picture AI costs, they usually see the obvious part: GPU racks, power draw, and the scramble for more silicon. But inference leaves a second bill behind, and it can be easy to miss until the storage line starts behaving badly. The data that piles up after a model answers a prompt has to live somewhere, and most planning still treats that as an afterthought.
That matters because training and inference are not the same kind of expense. Training is a project with a beginning and an end. Inference keeps going. Every request can generate logs, cached context, safety records, renders, backups, and other back-end material that does not disappear when the user gets their result.
WD’s internal measurements show how fast that adds up in AI video. An 8-second clip returned as a 3.97MB 720p file left more than 500MB of associated data behind on the back end. That is roughly 130 times the size of what the user actually saw. Push that workload into longer or higher-resolution video, and the footprint can reach hundreds of gigabytes per clip.
The broader trend points the same way. McKinsey projects AI inference demand will grow at a 35% CAGR through 2030, compared with 22% for training, and says inference will make up more than 40% of total data center demand by the end of the decade. In a WD-sponsored IDC survey of 763 IT and business decision makers, nearly 95% said generative AI increased their data volume over the past 12 months, and about 61% said the growth was 25% or more.
That is why storage belongs in the AI budget conversation, not on the sidelines. A hyperscale cloud product manager quoted in the report put it bluntly: keeping GPUs busy depends on having data ready when they need it, because idle time gets expensive fast. If storage spend keeps rising while compute is planned in neat cycles, the overlooked line item can end up growing into the one that hurts most. For many teams, the real surprise is not that AI costs money. It is that the cheapest-looking part of the stack keeps getting fatter after the model is done speaking.
My take — AI-written commentary, not fact-checked reporting
The storage bill is the part of AI everyone politely ignores until the CFO starts asking awkward questions. That is classic enterprise behavior: spend like the future is mostly GPUs, then act shocked when the archive becomes the real production system. The industry loves talking about inference as if it were just a query problem; in practice, it is also a hoarding problem.
Read more about this at: MIT Technology Review