From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon
BAIR
Researchers developed a method to automatically translate GPU kernel optimizations from CUDA to Apple's MLX framework by building an evolutionary search system (K-Search) with a structured translation layer that maps CUDA concepts to Metal/MLX equivalents. The evolved attention kernel achieved 0.97x the speed of Apple's native implementation and the Mamba SSM kernel reached 20x faster prefill throughput compared to the community mlx-lm implementation. This approach enables newer hardware ecosystems like Apple Silicon to leverage decades of accumulated CUDA optimization knowledge rather than rediscovering optimizations from scratch.
Why it matters
Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago. With so much of computing now centered on AI, GPU kernels are a crucial component of its success. These are the low-level programs that run inside the GPU, and writing efficient ones is far from obvious — it takes years of expertise to get right. Transferring a kernel from one vendor’s hardware to another is harder still, and often means rediscovering the same optimizations from scratch. The CUDA ecosystem, for example, has accumulated decades of hard-won kernel expertis
Related stories
[AINews] Megakernels are so dead and so back
Latent Space · 1 month ago ·
47
Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon
MarkTechPost · 2 weeks ago ·
49
Cursor Open-Sources Mixture-of-Kittens (MoK): A Deterministic MoE Training Megakernel for GB300 NVL72 Racks
MarkTechPost · 1 month ago ·
46