TurboFieldfare
GitHub
One dev built a way to run a 26-billion-parameter AI model on an 8GB Mac by streaming only the parts it needs from disk. No cloud, no crazy RAM upgrade — just clever engineering squeezing a big model into 2GB.
Based on reporting by GitHub — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Andrey Mikhaylov had a problem that a lot of on-device AI tinkerers run into: the model he wanted to run, Gemma 4 26B-A4B, needs 14.3 GB just sitting on disk, and conventional wisdom says you load the whole thing into memory before you can generate a single token. His MacBook Air has 8 GB of RAM. Instead of giving up or buying more hardware, he wrote a custom Swift and Metal runtime called TurboFieldfare that keeps only about 2 GB resident at any time — a shared 1.35 GB core plus an FP16 KV cache — and streams the rest, the mixture-of-experts weights, straight from SSD as each token gets generated.
The trick lives in how mixture-of-experts models actually work. Gemma 4 26B-A4B has 26 billion parameters total, but only around 3.88 billion get activated for any given token, routed through eight experts chosen at each layer. TurboFieldfare's CPU takes those router decisions, checks a 16-slot per-layer cache, and fires off parallel disk reads for whatever's missing, timed so Metal can crunch through the resident shared-expert computation while the fetches are in flight. It's a pipeline built around hiding I/O latency behind compute, not a generic wrapper bolted onto llama.cpp or MLX — Mikhaylov wrote this specifically for this model's architecture, down to 4-bit quantized weights with an 8-bit router and custom Metal kernels for attention, GEMV, and RoPE.
The numbers back up the pitch, if modestly. An 8 GB M2 MacBook Air pushes out 5.1 to 6.3 tokens per second, while a 24 GB M5 Pro hits 31 to 35. Neither is going to replace a cloud API for anything demanding, but that's not really the point — the point is that the smaller number exists at all. Mikhaylov backs this with 103 measured experiments covering kernels, caching, prefill and decode behavior, published openly, including the ideas that didn't pan out.
What's shipped so far is deliberately narrow: text-only, Apple Silicon only, one pinned instruction-tuned checkpoint, no tool execution baked into the app itself (though the loopback server can hand off function calls for a client to run). There's a native Mac app, a CLI, and an OpenAI-compatible local server, all built on Swift 6.2 and requiring Metal 4 and macOS 26. The installer itself follows the same philosophy as the runtime — it never stages a full duplicate checkpoint, streaming byte ranges from Hugging Face and repacking them directly into its own format as they land.
It's a solo project, released under Apache 2.0, with the model weights fetched separately and governed by Google's own terms — TurboFieldfare has no affiliation with Google. Mikhaylov's next targets are iPhone and iPad, plus wider Mac benchmarking, especially on smaller 16 GB and 8 GB machines that haven't been tested yet.
My take — AI-written commentary, not fact-checked reporting
This is the kind of project that makes the 'you need 64GB of unified memory to touch real models' narrative look lazy — a single engineer with a Metal habit just proved an 8GB laptop can run a 26B model by being smarter about memory instead of demanding more of it. I'd rather see ten more efficiency hacks like this than another benchmark chart showing a 400B model barely beating the last 400B model; the interesting frontier in local AI right now is squeezing more out of hardware people already own, not chasing parameter counts nobody outside a hyperscaler can afford to serve.
Read more about this at: GitHub