Mamba-3

Together AI · 4 months ago

Mamba-3 is a new state space model designed to prioritize inference efficiency rather than training speed, introducing a more expressive recurrence formula, complex-valued state tracking, and multi-input multi-output variants. On the 1.5B scale, Mamba-3 SISO achieves lower prefill+decode latency than Mamba-2, Gated DeltaNet, and Llama-3.2-1B across all sequence lengths, with the fastest latency at 4.39 milliseconds for 512 tokens versus Mamba-2's 4.66 milliseconds. The team open-sourced custom kernels built with Triton, TileLang, and CuTe to enable wider adoption and experimentation with the new architecture.