DeepSeek open-sources DeepInfer inference framework, drawing acknowledgment from vLLM

What happened

DeepSeek announced DeepInfer, its in-house inference framework powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. vLLM thanked DeepSeek for releasing it under Apache 2.0, said it was initially built primarily on vLLM, and plans to draw on its ideas to improve inference efficiency. http://github.com/deepseek-ai/DeepInfer

Reported by 3 sources

The story so far

DeepSeek has open-sourced DeepInfer, its in-house inference framework, on GitHub, making available the production stack it says powers DeepSeek-V4.1-Flash at scale. DeepSeek said the framework delivers high throughput and low latency and described the release as an effort to help the community lower compute costs, reduce energy use and make AI more affordable. vLLM thanked DeepSeek for releasing DeepInfer under the Apache 2.0 license and said the framework was initially built primarily on vLLM. The vLLM team plans to draw on DeepInfer’s ideas to improve inference efficiency.

Keep reading

Trending now

1. 2MiniMax Releases M3.1 Flash With Higher Benchmark Score and Lower Task Cost Than M3
2. 3Anthropic Commits 5% of Its Shares to New Public Benefit Fund