DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia.
The New Stack Paul Sawers ● Covered by 8 sources
DeepSeek open-sourced the engine behind its API, not just its model weights. It also put Huawei’s Ascend chips alongside Nvidia from day one, which says a lot about where this is headed.
Based on reporting by The New Stack, Paul Sawers — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
DeepSeek has opened up the inference engine it uses in production, a move it previously said wasn’t practical. The new release, DeepInfer, is meant to help developers run DeepSeek’s V4.1 Flash model on their own infrastructure using many of the same optimizations the company relies on behind its API. Support for future DeepSeek models is planned too.
This is a bigger deal than simply publishing model weights. DeepSeek has long allowed developers to download those weights and run them with third-party tools like vLLM. But the software that handles requests, spreads work across chips, and turns tokens into answers is where speed, throughput, and cost really get decided. DeepInfer opens up a lot of that machinery.
The code arrives under the Apache 2.0 license and includes optimized kernels, communication libraries, multi-chip inference support, speculative decoding, asynchronous scheduling, and structured outputs. DeepSeek is also releasing deepseek-recipe, a companion library in Rust and Python that helps reproduce the behavior of its official API when developers run their own deployments. That library was quietly released in September and updated on Tuesday.
The timing is notable. DeepSeek also just updated V4.1 Flash with a version called DeepSeek-V4.1-Flash-1017, saying it improved coding performance and efficiency. That came alongside new Search and Fetch APIs for the company’s in-house web search technology.
What DeepSeek did not release is just as important. The public version of DeepInfer currently supports offline inference and the core features needed for serving at scale, but it still lacks pieces such as KV cache management, cross-machine separation of inference stages, and load balancing across replicas. So this is not a turnkey production system. It is more like the engine block with some of the accessories missing.
The hardware list may be the most revealing part. Nvidia is supported, but only its SM100 architecture, used in Blackwell data center GPUs. Huawei’s Ascend 950 NPU series is also on the list. Older Nvidia chips such as Hopper are not mentioned. That puts Huawei on equal footing with Nvidia in DeepSeek’s public compatibility list, even if feature support differs between the two implementations.
My take — AI-written commentary, not fact-checked reporting
This is the sort of open source move that actually matters: not marketing fluff, but the plumbing that decides cost and speed. The Huawei support is the real tell. DeepSeek is not just sharing code; it is quietly normalising a world where Chinese AI stacks are built to run without Nvidia as the default answer.
Read more about this at: The New Stack
Related stories
DeepSeek-V4-Flash Outshines Pro, The Biggest GitHub Crawl Yet, Engineering System Prompts for Safer Code
The Batch ·
43
DeepSeek made its name with cheap AI models. Now it’s undercutting search API rivals, too.
The New Stack · 1 day ago ·
8