265 views · 0 likes · 0 reposts (at collection)
DeepSeek open-sources DeepInfer inference framework and companion deepseek-recipe components
- Who
- DeepSeek
- What
- open-sources the DeepInfer inference framework
- When
- Quote
DeepSeek 把自家线上服务用的推理框架 DeepInfer 开源了，还一起开源了配套组件 deepseek-recipe。
What happened
DeepSeek open-sourced DeepInfer, its production inference framework for DeepSeek-V4.1-Flash, under Apache 2.0, alongside deepseek-recipe. Baoyu reports that DeepInfer currently supports only V4.1-Flash on Blackwell SM100 or Ascend 950 hardware and releases offline inference components, leaving additional infrastructure necessary for online serving. The MIT-licensed deepseek-recipe provides Python and Rust implementations for V4 and V4.1, translating common OpenAI and Anthropic request formats and model outputs to align with DeepSeek’s official API.
Read next
See the full storyDeepSeek Open-Sources DeepInfer; vLLM Reports Matching Its V4.1-Flash Decode PerformanceReported by 5 sources
DeepSeek 开源自家推理框架 DeepInfer：Apache 2.0 可商用，目前只支持 V4.1-Flash，要用 Blackwell 或昇腾 950 这类数据中心显卡
DeepSeek 把自家线上服务用的推理框架 DeepInfer 开源了，还一起开源了配套组件 deepseek-recipe。
模型文件只是一堆训练好的参数，好比一本菜谱。接单、排队、在显卡上把答案算出来，靠的是推理框架，相当于后厨的出餐系统。出餐系统调度得好，同样的显卡能多接几倍的单，每个人等得也更短。
DeepSeek 说 DeepInfer 就是支撑 V4.1-Flash 大规模服务的那一套，开源是为了帮大家省算力、省电。
它最主要的提速手段叫投机解码。大模型平时一个字一个字往外写，每写一个字都要把整个模型算一遍。投机解码让一个轻量的“草稿手”先猜后面几个字（默认最多 5 个），大模型一次核对完，猜对的直接收下，猜错的重写。
普通用户不会直接用到它。它要跑在英伟达 Blackwell 一代的数据中心显卡（代号 SM100）或华为昇腾 950 上，官方测性能用了 4 到 64 块这样的卡，个人电脑跑不动。模型也只支持 V4.1-Flash 一个，官方说以后新模型的推理代码也会放进来。
用得上的是想自己部署 DeepSeek 的企业和云服务商，可以直接拿 DeepSeek 调好的方案。不过这次放出的只是“离线推理”部分，缓存对话前文的系统、把读输入和写输出拆到不同机器的部署方式、多台机器间的流量分配都没包含，要搭成完整的线上服务还得自己补。
deepseek-recipe 管的是另一个问题：自己部署的 DeepSeek，在提示词怎么拼、工具调用怎么解析这些细节上，可能和官方 API 对不上。
deepseek-recipe 像前台接单员，把 OpenAI 和 Anthropic 的几种常见请求格式统一翻译成 DeepSeek 模型能读的输入，再把模型输出的思考过程、工具调用等翻译回去，逻辑和官方 API 对齐。
DeepInfer 用 Apache 2.0 协议，允许免费商用和修改。deepseek-recipe 用 MIT 协议，同样可以商用，有 Python 和 Rust 版本，支持 V4 和 V4.1。
DeepInfer：https://github.com/deepseek-ai/DeepInfer
deepseek-recipe：https://github.com/deepseek-ai/deepseek-recipe
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 http://github.com/deepseek-ai/DeepInfer 1/4View the quoted post on X
The story so far
DeepSeek has open-sourced DeepInfer under Apache 2.0, releasing the inference framework it says powers DeepSeek-V4.1-Flash in production; vLLM reports matching its performance on DeepSeek’s decode configuration. That comparison covers batch sizes 8 to 256, with vLLM running at DEP32 against DeepInfer’s EP64DP64. vLLM said DeepInfer was originally built primarily on vLLM. According to Baoyu, DeepInfer currently supports only V4.1-Flash on Nvidia Blackwell SM100 or Huawei Ascend 950 hardware, and the release provides offline inference components that require additional infrastructure for online serving. Leiphone explained that its asynchronous scheduler overlaps CPU scheduling with GPU computation to reduce decoding idle time, coordinating in-flight tokens, sequence-state corrections and resource release across expert-parallel GPUs. DeepSeek also released deepseek-recipe. Baoyu reported that the MIT-licensed companion package provides Python and Rust implementations supporting V4 and V4.1, translating common OpenAI and Anthropic request formats and model outputs to align with DeepSeek’s official API.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-10-31