Nunchux AI Introduces VC-Attention: A Training-Free Low-Bit Attention Kernel That Speeds Up Video Diffusion Transformers
MarkTechPost Asif Razzaq
Nunchux AI launched VC-Attention, a training-free kernel for video Diffusion Transformers. It cuts two bottlenecks at once: value error and slow softmax.
Based on reporting by MarkTechPost, Asif Razzaq — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Nunchux AI has rolled out VC-Attention, a new attention kernel for video Diffusion Transformers that doesn’t need retraining. The pitch is simple enough: make the attention step faster without breaking the output. The company says it tackles two weak spots at once, the errors that creep in when values get quantized and the slow softmax stage that still sits in the middle of the whole thing.
The target is a nasty workload. Video DiTs turn a clip into one long sequence of spatiotemporal tokens and then run full self-attention at every layer. A 5-second 720p Wan2.2-14B clip lands at about 70,000 tokens, and on an RTX 5090 attention takes more than 64% of generation time. The researchers also say attention eats about two thirds of each MiniMax-H3 denoising step on a single B200. That’s the kind of number that makes every micro-optimization look suddenly worthwhile.
VC-Attention’s first trick is called V-Smooth. Prior methods such as SageAttention2 smooth queries and keys, but after that step the value term is still doing most of the damage. The paper says values account for 82% of output error on Wan2.2 after QK smoothing and rotation. V-Smooth instead groups value tokens online with k-means, subtracts the mean inside each 128-token hardware block, and quantizes only the residual. The mean comes back through the row sum that online softmax already tracks, so there’s no extra pass and no extra buffer.
The second trick is ExpCast-FP8, which skips the usual FP32 softmax exponent path for the 8-bit kernel. Instead of computing exponent-then-casting, it writes the E4M3 byte directly from the log-domain score with one fused multiply-add. Nunchux says the direct path matches the normal one on 79.6% of each doubling, and stays within one code elsewhere. Across 204,800 Wan2.2 attention rows, the average measured error was 1.6%. The paper also says hand-written CuTe/CUDA fusion cut one V-Smooth call from 42.2 ms to 4.8 ms on B200. And because NVFP4 doesn’t have the same neat log-to-code map, ExpCast-FP8 is limited to the 8-bit kernel.
My take — AI-written commentary, not fact-checked reporting
This is the kind of work that actually matters: not bigger models, just fewer stupid bottlenecks in the middle. The AI crowd loves talking about capability, but video diffusion still spends a shocking amount of time fighting attention machinery that barely deserves its own electricity bill. Training-free speedups with real constraints are the sane path, not another round of benchmark theatre.
Read more about this at: MarkTechPost