Cursor Open-Sources Mixture-of-Kittens (MoK): A Deterministic MoE Training Megakernel for GB300 NVL72 Racks

Cursor Research has put Mixture-of-Kittens, or MoK, out on GitHub under an Apache-2.0 license. It's the mixture-of-experts training kernel that's been quietly running Cursor's own Composer models across tens of thousands of GPUs, and now anyone with the right hardware can grab it. The pitch is simple: MoK fuses every communication and computation step of MoE training into one deterministic kernel, and against the strongest public baseline the team tested, it's up to 2.37 times faster.

That hardware requirement is the catch, and it's a steep one. MoK needs NVIDIA Blackwell SM100 or SM103 chips, which in practice means GB200 NVL72 or GB300 NVL72 racks, plus Python 3.12, PyTorch 2.10, and CUDA 13.0 or newer, with inter-GPU buffers leaning on PyTorch symmetric memory. That rules out anyone running a single node or an eight-GPU box. The realistic user list is short: frontier labs, well-funded model startups, GPU neoclouds, and national computing centers — organizations that already own or rent NVL72-scale capacity.

The backstory explains why this exists at all. Cursor's earlier kernel work — custom MXFP8 and NVFP4 training kernels, plus a warp-decode path for MoE inference — treated compute and communication as separate problems. In production that assumption broke down. The MoE layer alone can eat more than half of total training time, and moving to GB300 NVL72 racks made things trickier still. A rack ties 72 GPUs into one NVLink domain, which opens the door to fine-grained overlap, but the Grace CPUs sitting alongside those GPUs are comparatively slow, so every unnecessary CPU-GPU handoff becomes a tax the team had to engineer around.

Three choices do most of the work here. First, MoK picks communication direction per operation rather than defaulting to push, as tools like DeepEP do — Cursor's own microbenchmarks found push-based transfers leave the reverse NVLink lane mostly idle, while pull-based dispatch lifts NVLink bandwidth utilization by up to 29% under expert imbalance and cuts signalling latency from 103 microseconds to 18, roughly a 5.8x drop. So MoK pulls on forward dispatch and backward combine, and pushes on forward combine and backward dispatch, all driven by a single schedule table that costs under 3% of MoE runtime. Second, it aims for an overlap granularity between the fine-grained approach of Comet and the coarse-grained one of DeepEP, targeting at least two full streaming-multiprocessor waves per expert-grouped GEMM — for the Kimi 2.5 shapes underlying Composer 2.5, that works out to a floor of 2,368 tokens, and measured latency lines up with that estimate. Third, a ring token buffer of a few hundred megabytes cycles at minibatch granularity and gets walked in reverse during backward, removing the CPU from the loop entirely instead of dropping tokens or asking the CPU to size buffers on the fly.

On the numbers, MoK's layer-level tests ran inside a single NVL72 rack at expert-parallelism degree 64, with 2,048 tokens per GPU before routing, against NCCL, DeepEP, and HybridEP baselines across shapes like Kimi K2.7 Code and DeepSeek-V4-Pro. Results ranged from 1.58x for BF16 backward up to that headline 2.37x for MXFP8 forward. Scaled out to 512 GPUs across several GB300 NVL72 racks, throughput per GPU climbed from 760.9 to 1,070.2 tokens per second — a 1.41x gain end to end, and the kind of number that matters a lot more to someone paying for rack-hours than to anyone reading about it from a laptop.