NVIDIA AI Releases Nemotron 3.5 Lightning: A 30B Open MoE with 3B Active Parameters, and NeMo Switchyard Model Router
MarkTechPost Asif Razzaq ● Covered by 2 sources
NVIDIA shipped an open 30B model and a router for AI agents. The pitch is simple: save money and lag by not sending every step to a huge model.
Based on reporting by MarkTechPost, Asif Razzaq — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
NVIDIA has pushed out two pieces aimed at one problem: agents waste a lot of time on routine work. Tool calls, validation, and handing tasks between subagents are exactly the sort of steps that can get expensive fast if every move goes to a frontier reasoning model. So the company is splitting the job up instead of pretending one giant model should do everything.
Nemotron 3.5 Lightning is the bigger headline. It’s a 30B mixture-of-experts model with 3B active parameters, built on a hybrid Mamba-2, MoE, and attention setup. NVIDIA says it has a 1M-token context window and was pre-trained on more than 20 trillion tokens with an NVFP4 recipe. The model is open under OpenMDW-1.1, ships with open weights, training data, and recipes, and NVIDIA says it is ready for commercial use.
The speed story comes from two places. One is speculative decoding, including multi-token prediction and two draft models NVIDIA calls DSpark and DFlash. The other is quantization: an NVFP4 checkpoint comes alongside BF16, with support that spans Blackwell, Hopper, and Ampere through W4A16 kernels. NVIDIA claims up to 4x output speed versus similar-sized models, and says Lightning finished 10,000 PinchBench tasks 30% faster than Qwen3.6 35B at comparable accuracy.
This is not just a lab demo. NVIDIA says the model runs on a single modern GPU, including 1x DGX Spark or 1x H100, and can also be served through Baseten, Together AI, or Nebius. Named users already include CrowdStrike, Harvey, CodeRabbit, Fastino Labs, and Lila Sciences, with use cases spanning cybersecurity, legal work, coding, finance, healthcare, and life sciences. It also shows up locally through tools like Ollama, LM Studio, llama.cpp, and Unsloth.
The second release, NeMo Switchyard, is the routing layer that makes the whole idea less wasteful. It sends each step of an agent workflow to the model best suited for it, using options like an LLM classifier with session affinity, a stage router, an escalation router, and a tunable prefill router. NVIDIA says the reference server accepts OpenAI, Anthropic, and Responses API requests. In LangChain’s 145-task benchmark, routing between Lightning and Claude Opus 4.8 cut cost by 74% versus a frontier-only baseline, while sending just 7% of calls to the frontier model.
My take — AI-written commentary, not fact-checked reporting
This is the sensible direction, which is exactly why it will probably be ignored by people who still want one model to cosplay as an entire company. The real money in agent systems is boring routing, not grand “AI coworker” speeches, and NVIDIA knows it. Open weights plus practical deployment is the adult choice here; the hype crowd can keep renting latency.
Read more about this at: MarkTechPost