Despite the U.S. Ban, DeepSeek Open-Sources Its Inference Engine
Trending Topics Jakob Steinschaden ● Covered by 7 sources
DeepSeek has open-sourced DeepInfer, the engine it uses to run its models. It works on Huawei and Nvidia chips, right as the U.S. keeps squeezing the company.
Based on reporting by Trending Topics, Jakob Steinschaden — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
DeepSeek is answering pressure from Washington with a public release. The Chinese A.I. lab has put DeepInfer on GitHub under the Apache 2.0 license, opening up the software it uses to run its models at scale. That matters because inference is where the rubber meets the road: it is the layer that turns a trained model into actual responses, and it has a huge say in speed and cost.
DeepSeek says DeepInfer is the framework behind its large-scale production services. At launch, it supports DeepSeek-V4.1-Flash, including the faster version the company released just last Saturday. More model support is supposed to follow. The code pulls together pieces DeepSeek had already released before, including FlashMLA, DeepGEMM and DeepEP, and adds features such as model spreading across many chips, draft-based speedups and structured outputs like JSON.
The hardware support is the part that really stands out. DeepInfer currently runs on exactly two platforms: Nvidia’s latest Blackwell data center chips and Huawei’s Ascend 950 accelerators. Older Nvidia chips, including Hopper, are not on the list. That lines up with DeepSeek’s effort to lean less on Nvidia, and Bloomberg reports the lab plans to use at least 160,000 Ascend chips in a data center in Inner Mongolia.
The timing is awkward for the U.S. government and useful for DeepSeek. This week, the Department of Homeland Security ordered federal civilian agencies to remove DeepSeek from their systems, including third-party products built on it. Reuters also reported that the U.S. is moving against shipments of Nvidia chips to Chinese companies outside China. DeepSeek is clearly trying to show that it can keep its models running efficiently on Chinese hardware while also making them easier for other companies to deploy on their own servers.
But DeepInfer is not finished. Missing pieces include cache management for previously computed requests, splitting work across different machines and load balancing between instances. Some features, like reproducible results, only work on Nvidia chips, and DeepSeek has not published performance numbers or Huawei benchmarks. So the release is real, and useful, but it is still a work in progress.
My take — AI-written commentary, not fact-checked reporting
DeepSeek is doing the classic open-source power move: ship the code, let everyone else fill in the missing plumbing. That’s smart, and a little cheeky. The bigger story is that Chinese A.I. stacks are getting good enough to stand beside Nvidia, even if the benchmark pages are still conveniently blank.
Read more about this at: Trending Topics