Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets
Together AI
Together AI partnered with MiniMax to serve the M3 model efficiently, implementing custom kernel optimizations including sparse attention and paged attention integration. The optimizations achieved 81-125% throughput improvements across different concurrency levels on NVIDIA B200 hardware. This infrastructure enables production deployment of M3's 1M-token context window and multimodal capabilities at scale.
Why it matters
How Together served MiniMax-M3 efficiently with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based multimodal gateway.
.jpg)