- Who
- vLLM
- What
- reports optimized inference performance DeepSeek-V4.1-Flash on Vera Rubin NVL72, including throughput results compared with GB300 NVL72
- Quote
We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX.
Read next
See the full storyvLLM Reports DeepSeek-V4.1-Flash Throughput on Vera Rubin NVL72We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX.
Across the curve:
🔹 2.52 million tokens/s/GPU at peak throughput (2.7× GB300)
🔹 1.7 million tokens/s/GPU at P90 387 tokens/s/user (58× GB300)
🔹 1,810 tokens/s at one concurrent session with TP
1/4
The story so far
vLLM said it optimized DeepSeek-V4.1-Flash for NVIDIA's Vera Rubin NVL72 and reported higher token throughput than GB300 NVL72 on the SemiAnalysis AgentX benchmark, including results powered by Dynamo and vLLM. In its initial performance curve, vLLM measured 1.7 million tokens per second per GPU at 387 tokens per second per user at P90, or 58 times GB300 NVL72 throughput, along with a peak of 2.52 million tokens per second per GPU (2.7 times GB300 NVL72) and 1,810 tokens per second in a single concurrent session using TP 1/4. In a follow-up post, vLLM said InferenceX's performance ruler showed total token throughput per chip 63.4 times higher than GB300 NVL72 at 400 tokens per second per user and 93.3 times higher at 600 tokens per second per user.
