NVIDIA
·
1 week ago
NVIDIA introduced Vera, a CPU designed specifically for agentic AI systems that prioritizes single-threaded performance to execute tool calls and data processing between model calls. Vera delivers 1.8x higher sustained per-core performance than x86 CPUs in agentic workloads, with Perplexity achieving 1.5x faster performance on real coding workflows. This optimized CPU architecture helps AI factories reduce GPU idle time and complete more agent tasks by ensuring each step in the agent loop runs faster.
TLDR
·
1 week ago
Nvidia's Kyber rack system, designed to house its 2027 Rubin Ultra chips in a single cabinet with 144 processors, has been delayed to 2028 due to manufacturing difficulties with a specialized circuit board. The delay pushes back the system's original 2027 launch by more than 12 months, and a proposed backup solution using two current-generation racks was scrapped after cloud providers rejected it as operationally impractical. The setback leaves Nvidia without a proven way to scale up the Rubin Ultra system to larger configurations, potentially opening a market opportunity for AMD and Google's custom chips in high-end AI infrastructure.
BAIR
·
1 week ago
The cost of AI inference has dropped 50x to 900x per year, with GPT-4-class capabilities now under $1 per million tokens compared to $30 in early 2023, making sufficient intelligence for knowledge work effectively free. This shift requires rethinking data systems in three ways: designing systems that handle agents issuing thousands of speculative queries per request, building infrastructure to manage agent swarms with shared memory and coordination across thousands of concurrent agents, and enabling agents to synthesize and verify custom data systems. The changes enable new possibilities like multi-query optimization to reduce duplicate work, structured memory systems for agents to retrieve task-relevant information across multiple dimensions, and systems that proactively guide agents rather than passively execute queries.
