StudioDeepSeek Open Sources DeepInfer: A High-Performance Inference Framework for Large-Scale Production
Original title: deepseek-ai/DeepInfer: DeepInfer: High-performance, large-scale inference framework for DeepSeek models
deepseek-ai has open-sourced DeepInfer, a high-performance inference framework used in DeepSeek's large-scale production services. Currently supporting the DeepSeek-V4.1-Flash model, it features optimized kernels (FlashMLA, DeepGEMM, DeepEP), Attention DP/CP and MoE EP parallelism, DSpark confidence scheduling, asynchronous scheduling, batch-invariant inference, deterministic sampling, and structured output. Built on vLLM under Apache License 2.0, it supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Note that KV cache systems and Prefill/Decode separation are not included in this release.
The README lists supported models, parallelism and scheduling features, hardware compatibility, and dependency versions, allowing readers to assess its suitability for their own deployment.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.