Nvidia Releases Lightweight Nemotron Model With Smart Routing Technology

Nvidia unveiled two new tools designed to make AI inference cheaper and faster: Nemotron 3.5 Lightning, a compact language model, and NeMo Switchyard, an open-source library for intelligently routing requests between multiple models.

A Compact Model Built for Speed#

Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model that achieves reasoning performance approaching Nvidia's larger Nemotron 3 Super, while delivering up to four times faster output speeds. The model was built collaboratively within the Nemotron coalition and emphasizes speed and customization over raw benchmark performance.

The core strategy is architectural: larger frontier models handle strategic planning and decision-making, while smaller models like Lightning execute specific tasks—potentially after fine-tuning for particular domains or workflows.

Customization Where It Counts#

Rather than relying solely on generic AI benchmarks, Nvidia found that the real advantage lies in post-training Lightning for specialized tasks. Working with partners including CrowdStrike and CodeRabbit, the company demonstrated that a fine-tuned open model could match or exceed larger proprietary alternatives on domain-specific work.

"General agentic benchmarks are just the starting point, but in production, what matters is accuracy on your task — and that's where post-training makes the biggest difference," Kari Briski, Nvidia's leader for the announcement, explained. Early partners reported significant improvements after customization, with Lightning outperforming models enterprises currently rely on.

Nvidia is also releasing a full dataset for agentic reinforcement learning to help teams train coding agents.

Smart Routing Cuts Costs Dramatically#

The companion technology, NeMo Switchyard, tackles a fundamental challenge in multi-model systems: deciding which model to invoke for each request. Written in Rust but accessible via APIs, Switchyard lets developers define model pools and routing policies tuned for quality, latency, or cost.

Nvidia's internal tests showed mixed-model systems using Switchyard maintained frontier-level accuracy while cutting costs to roughly one-third of running a single large model alone. Early partners saw similar wins: LangChain achieved 74% cost reductions on multi-turn agent tasks by routing only 7% of calls to frontier models, and Ramp matched frontier performance on internal benchmarks while cutting costs by 58% and latency by 33%.