Native-speed vLLM transformers modeling backend

Hugging Face just closed a gap that's been annoying model builders for a while. Until now, if you wanted your model to run at full speed inside vLLM, someone had to write a custom implementation just for that purpose, separate from the transformers version everyone actually reads to understand how the architecture works. That's double the work, and it meant only a handful of popular models ever got the fast treatment.

The fix is a set of upgrades to the transformers modeling backend for vLLM, tested across three very different Qwen3 setups: a 4B dense model on one GPU, a 32B dense model split across two GPUs with tensor parallelism, and the massive 235B-parameter FP8 mixture-of-experts model spread over an 8-H100 node using data and expert parallelism. In every one of those cases, the transformers backend now matches or beats vLLM's own native, hand-tuned code. You just add one flag, --model-impl transformers, and everything else about your serving command stays the same.

The trick is that the backend now analyzes the model's computational graph using torch.fx, looking for patterns it recognizes, then rewrites the underlying source code with Python's ast module to swap in vLLM's heavily optimized kernels. Think fused blocks for mixture-of-experts routing, or the MergedColumnParallelLinear and QKVParallelLinear operations that let vLLM infer tensor-parallel and even pipeline-parallel plans automatically. The rewritten models still go through torch.compile and CUDA Graphs exactly like a dedicated vLLM implementation would.

What's genuinely useful here is that this isn't a one-off hack for Qwen3. Any compatible Hugging Face model gets this for free the moment someone writes it in transformers, no custom vLLM port required. And because it's still fundamentally transformers code, the same model can be used for training, evals, or RL rollouts, not just inference — something a hand-written vLLM-only implementation could never offer. Linear attention models aren't supported yet, and Hub-hosted custom code probably won't play nice, but for the mainstream case this collapses two jobs into one.