Skip to content

vLLM Reports DeepSeek-V4.1-Flash Throughput on Vera Rubin NVL72

2 reports 1 reporting sources first reported Updated 3 hours ago

Understand the event

AI overview

vLLM said it optimized DeepSeek-V4.1-Flash for NVIDIA's Vera Rubin NVL72 and reported higher token throughput than GB300 NVL72 on the SemiAnalysis AgentX benchmark, including results powered by Dynamo and vLLM. In its initial performance curve, vLLM measured 1.7 million tokens per second per GPU at 387 tokens per second per user at P90, or 58 times GB300 NVL72 throughput, along with a peak of 2.52 million tokens per second per GPU (2.7 times GB300 NVL72) and 1,810 tokens per second in a single concurrent session using TP 1/4. In a follow-up post, vLLM said InferenceX's performance ruler showed total token throughput per chip 63.4 times higher than GB300 NVL72 at 400 tokens per second per user and 93.3 times higher at 600 tokens per second per user.

Generated by AI from reports · Updated 2 hours ago

Read next

More changes in today’s briefingGemini Connects to Google Wallet for Personal Finance Insights

Models in this story

Developments

2 developments

  1. 11/17 01:01 (UTC+8) · 1 reports
    InferenceX Reports DeepSeek V4.1 Flash Benchmark Results on Vera Rubin
    vLLM · X:DeepSeek V4.1 Flash Runs on Vera Rubin NVL72 in AgentX Benchmark
  2. 11/13 06:31 (UTC+8) · 1 reports
    vLLM Reports DeepSeek-V4.1-Flash Throughput on Vera Rubin NVL72
    vLLM · X:We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX. Across the curve: 🔹 2.52 million tokens/s/GPU at peak throughput (2.7× GB300) 🔹 1.7 million tokens/s/GPU at P90 387 tokens/s/user (58× GB300) 🔹 1,810 tokens/s at one concurrent session with TP 1/4

Report timeline

Follow the reports to explore different sides of the event.

11/17
  1. DeepSeek V4.1 Flash Runs on Vera Rubin NVL72 in AgentX Benchmark

    vLLM reports the first DeepSeek V4.1 Flash results on NVIDIA Vera Rubin NVL72, powered by Dynamo and vLLM on SemiAnalysis’ AgentX benchmark. InferenceX’s performance ruler shows total token throughput per chip 63.4× higher than GB300 NVL72 at 400 tok/s/user and 93.3× higher at 600 tok/s/user.

11/13
  1. We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX. Across the curve: 🔹 2.52 million tokens/s/GPU at peak throughput (2.7× GB300) 🔹 1.7 million tokens/s/GPU at P90 387 tokens/s/user (58× GB300) 🔹 1,810 tokens/s at one concurrent session with TP 1/4

    We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX. Across the curve: 🔹 2.52 million tokens/s/GPU at peak throughput (2.7× GB300) 🔹 1.7 million tokens/s/GPU at P90 387 tokens/s/user (58× GB300) 🔹 1,810 tokens/s at one concurrent session with TP 1/4

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

Current within comparable coverage
9
Peak within comparable coverage
1011/17 02:00 (UTC+8)
24-hour change
–

Trends compare the same continuously observed participants, which may cover fewer sources than current heat. Point or click to inspect hourly heat, or use left and right arrow keys.