Experiences with local models for coding
martinfowler.com
A developer spent weeks testing small AI models (like Qwen3.6 35B) running locally on a MacBook for coding tasks. Results were wildly inconsistent — same model, same task, different hardware, wildly different success rates.
Based on reporting by martinfowler.com — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
There's a quiet experiment happening on developer laptops right now: can you skip the cloud entirely and get real coding help from a model small enough to run on 48-64GB of RAM? One engineer at Thoughtworks decided to find out, methodically, and the results are messier than the hype around local AI would suggest.
The setup involved an M3 Max with 48GB and an M5 Pro with 64GB, running models like Qwen3.6 35B MoE, Gemma 4 (26B and 31B variants), and Qwen Coder Next 80B through harnesses like OpenCode and Pi. The tasks weren't toy examples — real changes to a bar chart's sorting logic, cumulative percentage labels on an x-axis, that kind of thing. Small stuff, but stuff that requires a model to read code, understand it, and edit it correctly.
The most striking finding wasn't about any single model failing or succeeding. It was the inconsistency. Gemma 4 26B nailed a task in manual testing, then failed 3 out of 3 times in automated runs. Qwen3.6 35B succeeded 2 out of 2 in that same automated test. And in a separate task, the exact same model and settings failed 5 out of 7 times on the 48GB machine but only once on the 64GB machine — same quantization, same everything, just different hardware, and the quality of output shifted dramatically. Nobody has a clean explanation for that one yet.
There were also flat-out crashes. Qwen Coder Next produced working code in under three minutes, then crashed the runtime the moment a follow-up message was sent — technically capable, practically useless. Other combinations spiraled into what the author calls
My take — AI-written commentary, not fact-checked reporting
None of this surprises me, honestly — running a compressed 35B model on a laptop and expecting Claude-level judgment was always a bet on convenience over capability, and the numbers here just make that trade-off visible instead of hand-wavy. What actually interests me is the RAM-dependent quality swing between two Macs running the identical model; if quantized local models are this sensitive to hardware quirks nobody fully understands, "just run it locally" is nowhere near the plug-and-play story open-model advocates like to sell, and pretending otherwise does the open-weights movement no favors.
Read more about this at: martinfowler.com