DeepSeek Announces Full Open-Source Release of Internal Inference Framework DeepInfer

On October 20, DeepSeek announced it will fully open-source its internal inference framework DeepInfer, and simultaneously release the production-grade Token encoding/decoding component deepseek-recipe to help developers and enterprises optimize throughput, latency, and deployment costs for large-scale model inference.

DeepInfer integrates high-performance operators and a deeply optimized inference engine, supporting features such as distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and is compatible with NVIDIA and Huawei Ascend platforms. DeepSeek stated that it has already deployed high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash based on DeepInfer, and inference code and subsequent optimizations for future models will continue to be updated within this framework.

Additionally, deepseek-recipe will align with the processing logic of DeepSeek's official API, enabling the community to achieve an API usage experience similar to official deployments.