DeepSeek Open-Sources DeepInfer, the Inference Framework Behind DeepSeek-V4.1-Flash

DeepSeek has put DeepInfer on GitHub under Apache 2.0, opening up the inference framework it uses in production. It also released deepseek-recipe, a token encoding and decoding layer under MIT. For developers who like peeking behind the curtain, this is not a demo toy. It is the machinery behind a live system.

DeepInfer supports one model: DeepSeek-V4.1-Flash. Per the model card, that model has a 1M-token context window. The framework is built for NVIDIA SM100 Blackwell GPUs and Huawei Ascend 950 NPUs, and DeepSeek’s own benchmarks used 4 GPUs for prefill and 64 for decode. The company says one cluster can serve 155 trillion tokens per day.

The architecture is very much DeepSeek’s own stack, layered on top of its kernels: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and custom TileLang kernels. Above that sit a few system-level tricks: attention data parallelism, expert parallelism for MoE layers, an expert-parallel load balancer, and a decode path that runs EP64 to EP128 inside a supernode. Prefill uses the smallest expert parallel size that fits in memory, usually EP2 to EP8.

The most interesting piece may be DSpark, DeepSeek’s confidence-scheduled speculative decoding system. It drafts 5 tokens at a time, then a Confidence Head estimates which ones are worth checking. DeepSeek says that cuts verified draft tokens by about 30% while keeping accepted length almost the same. There is also SWA bounded replay, which only recomputes the last 128 tokens on a cache hit, plus asynchronous scheduling so the GPU does not sit idle between steps.

But the release comes with hard limits. This is offline inference only. There is no KV cache system, no P/D disaggregation, and no load balancing across data-parallel replicas. DeepSeek also says the missing pieces are tied closely to its internal infrastructure. So yes, this is a real serving stack. It is just not the whole serving stack.