NVIDIA Rubin: six chips, an NVL72 rack and what it means for developers
NVIDIA unveiled the Rubin platform, the successor to Blackwell, and the announcement has the usual shape: lots of petaflops, lots of terabytes per second and promises of lower cost. Almost nobody reading this blog will buy a Vera Rubin rack. Even so, it is worth understanding what was announced, because this is the hardware that will shape the price, latency and context size of the model APIs we use from the second half of the year onward.
What was announced
Rubin is not one chip but six, designed to work together:
- Vera CPU, with 88 custom Olympus cores and Armv9.2 compatibility;
- Rubin GPU, with 50 petaflops of NVFP4 compute per GPU, aimed at inference, and a third-generation Transformer Engine;
- NVLink 6 Switch;
- ConnectX-9 SuperNIC;
- BlueField-4 DPU;
- Spectrum-6 Ethernet Switch.
These chips come in two main formats. The Vera Rubin NVL72 is a rack with 72 Rubin GPUs and 36 Vera CPUs, linked by NVLink 6, with ConnectX-9 SuperNICs and BlueField-4 DPUs. The HGX Rubin NVL8 is a server board that connects eight Rubin GPUs over NVLink, designed for x86 platforms.
NVLink bandwidth is 3.6 TB/s per GPU and 260 TB/s per NVL72 rack. The platform also brings third-generation Confidential Computing, and NVIDIA says Vera Rubin NVL72 is the first rack-scale platform to offer it.
The promises, according to NVIDIA
The performance numbers come from NVIDIA itself, compared with Blackwell:
- up to 10x lower inference cost per token;
- training MoE (mixture of experts) models with 4x fewer GPUs.
The “up to” is doing a lot of work in that sentence. Generation-over-generation comparisons tend to pick the most favorable workload, and the real gain depends on the model, batch size and number format. Still, the direction is clear: NVIDIA is optimizing for cheap inference and for MoE models, which is exactly where the big labs are heading.
The part that matters most for developers: context and KV cache
The least flashy item in the announcement is, to me, the most interesting. NVIDIA introduced the Inference Context Memory Storage Platform, built on BlueField-4 and designed to let AI infrastructure share and reuse KV cache data to scale inference context.
Why should someone who just calls an API care? The KV cache is what lets a model avoid recomputing the whole prompt for every generated token. In long contexts, and in agents that return to the same history over and over, storing and reusing that cache is what makes features like prompt caching cheap. A storage layer dedicated to it suggests providers will have more room to offer bigger contexts and longer-lived caches at lower prices.
Together with 50 petaflops of NVFP4 per GPU, the message is that the next generation of infrastructure was designed around long-context inference, not just training.
When it arrives
NVIDIA says Rubin is in full production, but Rubin-based products will only be available from partners in the second half of 2026. Among the first to deploy in the cloud in 2026 are AWS, Google Cloud, Microsoft and OCI, plus NVIDIA Cloud Partners CoreWeave, Lambda, Nebius and Nscale.
In other words, nothing changes on your API bill tomorrow. The effect, if it comes, will show up over the year as model providers migrate capacity.
My take
For software developers, Rubin is indirect news, but not irrelevant. If the cost-per-token promises hold up even partially, the price of large models should keep falling, and long contexts should get cheaper thanks to the focus on KV cache. I would not make any architecture decision based on NVIDIA’s “up to 10x”. But I would make a conservative bet: design systems assuming long-context inference will become more affordable during 2026, and avoid complex optimizations that only exist to work around today’s token costs.