“We love the world where we can use both”: How Nvidia thinks about local and frontier models

Nvidia's Joey Conway has a pretty simple pitch for where AI is headed: stop imagining one massive model that does everything, and start imagining a bench of specialists. Some questions are trivial — Conway's example is a reasoning model chewing through number lines just to figure out that two plus two equals four — and sending those to a frontier model is a waste of money and time. Send the easy stuff to something small and local, save the expensive frontier model for problems that actually need it.

The mechanism that makes this work is routing, and Conway admits it's still early. Nvidia's own play here is lower in the stack than you might expect: its open-source Dynamo software focuses on inference serving, essentially steering a query to whichever GPU last handled a similar one, rather than deciding which model gets the job in the first place. That decision — the actual model-picking — is left to a broader ecosystem of routers, some of which are themselves models weighing cost, latency, and modality. Conway doesn't rule out Nvidia building more of that layer itself down the line.

The results Nvidia is citing aren't hypothetical. A collaboration with LangChain ran its Deep Agents harness on Nemotron 3 Ultra, Nvidia's 550-billion-parameter open model, and reportedly matched closed frontier models on business tasks at up to ten times lower cost. No retraining involved — the gains came purely from tuning prompts, tool descriptions, and middleware around the model. That's a meaningful data point for anyone assuming open models only catch up through brute-force fine-tuning.```

For enterprises, Conway argues the real draw of local models isn't just saving money, though a fine-tuned open model living on your own hardware does mean no surprise token bills. It's control: keeping data and intellectual property in-house, treating a tuned model like an employee that becomes part of the company. Nvidia's DGX Spark, a $4,699 machine with 128GB of unified memory, can run models up to roughly 200 billion parameters without any data leaving the desk, and the pricier DGX Station goes bigger still. To keep those local agents from running wild, Nvidia is also pushing NemoClaw, a sandboxed reference stack for running open agent harnesses with policy controls baked in.

Whatever mix of local and frontier wins out, Nvidia's position is comfortable either way — the company sells the chips underneath both. That's worth keeping in mind when reading its enthusiasm for a 'system of models' future: it's not exactly a neutral bystander in this fight.