DeepSeek open-sources DeepInfer, the inference framework powering V4.1-Flash
Deepseek open-sources the inference engine that powers its fastest and cheapest AI model
- Who
- DeepSeek
- What
- open-sources the DeepInfer inference framework
- When
- Quote
Deepseek has open-sourced DeepInfer, its internal inference framework, under the Apache License 2.0.
What happened
DeepSeek has open-sourced DeepInfer, its internal inference framework powering V4.1-Flash, under the Apache License 2.0. Built on vLLM, it includes optimized compute kernels, distributed processing across multiple chips, and faster text generation techniques. DeepSeek is also releasing deepseek-recipe to replicate the official API's token processing in self-hosted setups and says code for future models will also be open source.
Why it matters
The release extends open sourcing beyond model weights to serving infrastructure, giving self-hosting teams access to the software DeepSeek uses to pursue lower inference costs.
Reported by 5 sources
The story so far
DeepSeek has open-sourced DeepInfer, the inference framework it says powers DeepSeek-V4.1-Flash at scale, under Apache 2.0, alongside companion components called deepseek-recipe. According to Baoyu, DeepInfer currently supports only V4.1-Flash on Nvidia Blackwell SM100 or Huawei Ascend 950 hardware; the release provides offline inference components, so online serving requires additional infrastructure. DeepInfer includes optimized compute kernels, processing distributed across multiple chips, and techniques to accelerate text generation, The Decoder reported. vLLM said the framework was initially built primarily on vLLM and that it plans to draw on DeepInfer’s ideas to improve inference efficiency. DeepSeek described high throughput and low latency as features of its production system and lower compute costs and energy use as goals of sharing it. Baoyu reported that the MIT-licensed deepseek-recipe offers Python and Rust implementations supporting V4 and V4.1, translating common OpenAI and Anthropic request formats and model outputs to align with DeepSeek’s official API. The Decoder also reported that DeepSeek says it will open-source code for future models.
Keep reading
Latest daily · Issue 27TorchNeuron brings native PyTorch support for Hugging Face models on AWS TrainiumTrending now
Tomorrow at 08:00, a briefing like this arrives in your inbox.
The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue · 2026-10-20