StudioDeepSeek Open-Sources DeepInfer: Claims 155T Tokens/Day and ~80% Margin with 81-Day Payback
Original title: [AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
DeepSeek has open-sourced DeepInfer, the production inference engine behind V4.1 Flash, claiming a single cluster processes 155 trillion tokens daily and generates $38 per accelerator hour during off-peak times, implying an ~80% gross margin and an 81-day payback period. Latent Space notes that KV cache management and load balancing were not included in the release, and a 96% input cache hit rate means throughput figures should not be interpreted as new token generation. The vLLM team later reproduced results on GB200 hardware matching DeepInfer's configuration, showing less than 6% variance in decode throughput. The report also aggregates various estimates on cache economics, debates the transferability of model-specific optimizations, and covers initial performance metrics for V4.1 Flas…
This article consolidates independent tests and economic calculations following DeepInfer's open-source release, clarifying the boundaries of disclosed metrics like the 96% cache hit rate, allowing readers to independently evaluate the inference stack.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.