DeepSeek open-sources DeepInfer and discloses cache-heavy production inference economics
[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
What happened
DeepSeek released DeepInfer, the inference engine behind V4.1 Flash, reporting 155 trillion tokens/day from one cluster with a 96% input-cache hit rate and $38 revenue per accelerator-hour at off-peak prices. Important KV-cache management and load-balancing components remain unreleased. Latent Space presents margin and payback estimates based on these disclosures, while explaining their cost assumptions; vLLM separately reports decode throughput within 6% of DeepInfer’s published results under a matched GB200 configuration.
Why it matters
The distinction between cached token throughput and newly computed tokens provides essential context for interpreting DeepInfer’s reported scale and the article’s inference margin estimates.
Keep reading
Latest daily · Issue 28Mistral AI releases Voxtral 3 speech models and a voice agent SDKTrending now
- 1Mistral Releases Voxtral 3 Speech Models, Voice Agents API and LiveKit-Based SDK
- 2DeepSeek Open-Sources DeepInfer and API Components, With Model and Hardware Limits
- 3OpenAI Acquires Orchestra Bio to Expand AI Tools for Drug Development
- 4Google Rolls Out Gemini Personal Intelligence to Android Auto, With Google Built-In Cars to Follow
Tomorrow at 08:00, a briefing like this arrives in your inbox.
The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue · 2026-10-21