DeepSeek Open-Sources DeepInfer; vLLM Reports Matching Its V4.1-Flash Decode Performance

1. 22:14
   雷锋网Picks

   How DeepInfer Coordinates Asynchronous Scheduling for DeepSeek-V4.1-Flash Inference

   DeepInfer, the inference engine behind DeepSeek-V4.1-Flash, overlaps CPU scheduling with GPU computation to reduce decoding idle time. Leiphone explains how its asynchronous scheduler handles in-flight tokens, sequence state corrections, deferred resource release, and coordination across expert-parallel GPUs. The open release currently supports offline inference, with some online infrastructure unavailable; DeepSeek has not separately published complete end-to-end speedup measurements isolating the scheduler’s contribution.

1. 01:59
   vLLM · X

   vLLM Matches DeepInfer on DeepSeek-V4.1-Flash Across Batch Sizes 8 to 256

   vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64. DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and https://t.co/0vOvFGa7GG

1. 14:54
   Latent SpacePicks

   DeepSeek releases DeepInfer inference stack with cache-heavy production figures

   DeepSeek open-sourced DeepInfer, the inference engine behind V4.1 Flash, and reported single-cluster throughput of 155 trillion tokens/day with a 96% input-cache hit rate. Latent Space examines the architecture, noting that important KV-cache management and serving orchestration components remain outside the release. It also distinguishes the reported $38 revenue per accelerator-hour from margin estimates and payback projections, while describing vLLM’s self-reported decode throughput within 6% of DeepInfer under a matched configuration.

2. 02:56
   The DecoderPicks

   DeepSeek open-sources DeepInfer, the inference framework powering V4.1-Flash

   DeepSeek has open-sourced DeepInfer, its internal inference framework powering V4.1-Flash, under the Apache License 2.0. Built on vLLM, it includes optimized compute kernels, distributed processing across multiple chips, and faster text generation techniques. DeepSeek is also releasing deepseek-recipe to replicate the official API's token processing in self-hosted setups and says code for future models will also be open source.

1. 22:44
   宝玉 · X

   DeepSeek open-sources DeepInfer inference framework and companion deepseek-recipe components

   DeepSeek open-sourced DeepInfer, its production inference framework for DeepSeek-V4.1-Flash, under Apache 2.0, alongside deepseek-recipe. Baoyu reports that DeepInfer currently supports only V4.1-Flash on Blackwell SM100 or Ascend 950 hardware and releases offline inference components, leaving additional infrastructure necessary for online serving. The MIT-licensed deepseek-recipe provides Python and Rust implementations for V4 and V4.1, translating common OpenAI and Anthropic request formats and model outputs to align with DeepSeek’s official API.

2. 19:43
   Nathan Lambert · X

   DeepSeek open-sources DeepInfer, its production inference framework

   DeepSeek announced that DeepInfer, its in-house inference framework powering DeepSeek-V4.1-Flash at scale, is now fully open source. Nathan Lambert called this a wild way to drive model adoption and great for the community, remarking on the timing given what he described as a government ban on DeepSeek in federal agencies. http://github.com/deepseek-ai/DeepInfer https://twitter.com/deepseek_ai/status/2112476496283775135

3. 17:47
   vLLM · XPicks

   DeepSeek open-sources DeepInfer inference framework, drawing acknowledgment from vLLM

   DeepSeek announced DeepInfer, its in-house inference framework powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. vLLM thanked DeepSeek for releasing it under Apache 2.0, said it was initially built primarily on vLLM, and plans to draw on its ideas to improve inference efficiency. http://github.com/deepseek-ai/DeepInfer

4. 17:30
   DeepSeek · XPicks

   DeepSeek Open Sources DeepInfer, Its In-House Inference Framework

   🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 http://github.com/deepseek-ai/DeepInfer 1/4

Trends compare the same continuously observed participants, which may cover fewer sources than current heat. Point or click to inspect hourly heat, or use left and right arrow keys.