- 谁
- vLLM
- 引文
We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX.
发生了什么
我们在 vLLM 上针对 Vera Rubin NVL72 优化了 DeepSeek-V4.1-Flash,在 SemiAnalysis AgentX 上达到 58× GB300 NVL72 的吞吐量,速度为 170 万 tokens/s/GPU 和 387 tokens/s/user。 不同负载下: 🔹 峰值吞吐量为 252 万 tokens/s/GPU(GB300 的 2.7×) 🔹 P90 时为 170 万 tokens/s/GPU,387 tokens/s/user(GB300 的 58×) 🔹 TP 时,单个并发会话达到 1,810 tokens/s 1/4
接下来读什么
查看这件事的完整进展vLLM Reports DeepSeek-V4.1-Flash Throughput on Vera Rubin NVL72We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX.
Across the curve:
🔹 2.52 million tokens/s/GPU at peak throughput (2.7× GB300)
🔹 1.7 million tokens/s/GPU at P90 387 tokens/s/user (58× GB300)
🔹 1,810 tokens/s at one concurrent session with TP
1/4
这件事到目前为止
vLLM said it optimized DeepSeek-V4.1-Flash for NVIDIA's Vera Rubin NVL72 and reported higher token throughput than GB300 NVL72 on the SemiAnalysis AgentX benchmark, including results powered by Dynamo and vLLM. In its initial performance curve, vLLM measured 1.7 million tokens per second per GPU at 387 tokens per second per user at P90, or 58 times GB300 NVL72 throughput, along with a peak of 2.52 million tokens per second per GPU (2.7 times GB300 NVL72) and 1,810 tokens per second in a single concurrent session using TP 1/4. In a follow-up post, vLLM said InferenceX's performance ruler showed total token throughput per chip 63.4 times higher than GB300 NVL72 at 400 tokens per second per user and 93.3 times higher at 600 tokens per second per user.
