5.9K views · 115 likes · 22 reposts (at collection)
vLLM Matches DeepInfer on DeepSeek-V4.1-Flash Across Batch Sizes 8 to 256
- Who
- vLLM
- What
- reports benchmark results vLLM matching DeepInfer across batch sizes 8 to 256 on DeepSeek's decode configuration
- Quote
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.
What happened
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64. DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and
Links in the post: 1Read next
See the full storyDeepSeek Open-Sources DeepInfer; vLLM Reports Matching Its V4.1-Flash Decode PerformancevLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.
DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and https://t.co/0vOvFGa7GG
The story so far
DeepSeek has open-sourced DeepInfer under Apache 2.0, releasing the inference framework it says powers DeepSeek-V4.1-Flash in production; vLLM reports matching its performance on DeepSeek’s decode configuration. That comparison covers batch sizes 8 to 256, with vLLM running at DEP32 against DeepInfer’s EP64DP64. vLLM said DeepInfer was originally built primarily on vLLM. According to Baoyu, DeepInfer currently supports only V4.1-Flash on Nvidia Blackwell SM100 or Huawei Ascend 950 hardware, and the release provides offline inference components that require additional infrastructure for online serving. Leiphone explained that its asynchronous scheduler overlaps CPU scheduling with GPU computation to reduce decoding idle time, coordinating in-flight tokens, sequence-state corrections and resource release across expert-parallel GPUs. DeepSeek also released deepseek-recipe. Baoyu reported that the MIT-licensed companion package provides Python and Rust implementations supporting V4 and V4.1, translating common OpenAI and Anthropic request formats and model outputs to align with DeepSeek’s official API.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-10-31