Intelligence is Free, Now What? Data Systems for, of, and by Agents
BAIR ● Covered by 3 sources
The cost of AI inference has dropped 50x to 900x per year, with GPT-4-class capabilities now under $1 per million tokens compared to $30 in early 2023, making sufficient intelligence for knowledge work effectively free. This shift requires rethinking data systems in three ways: designing systems that handle agents issuing thousands of speculative queries per request, building infrastructure to manage agent swarms with shared memory and coordination across thousands of concurrent agents, and enabling agents to synthesize and verify custom data systems. The changes enable new possibilities like multi-query optimization to reduce duplicate work, structured memory systems for agents to retrieve task-relevant information across multiple dimensions, and systems that proactively guide agents rather than passively execute queries.
Why it matters
... government of the people, by the people, for the people ... — Abraham Lincoln, Gettysburg Address (1863) The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1, and some providers are pushing costs below $0.10. Across benchmarks, inference prices have fallen between 9x and 900x per year, with a median decline near 50x. Even frontier models are getting dramatically cheaper each generation, with open-source models following closely behind. And crucially, even if “Nobel-Prize-winning genius-level” intelligence isn’t here yet, the intelligence that suffices for the vast majority of knowledge work is here today, and getting cheaper by the month. At this rate, we are soon entering the era of virtually free intelligence—the kind that is more than enough for everyday knowledge work. Disclosure: This post is a perspective led by Aditya G. Parameswaran—an Associate Professor of EECS and co-director of the E
