Experiences with local models for coding

TLDR Dev · 1 week ago

A developer tested small language models running locally on their machine for agentic coding tasks, evaluating models like Qwen 3.6 35B and Gemma 4 across a viability funnel covering RAM usage, speed, tool calling, and code quality. In automated evaluation of a task involving access log visualization, Qwen 35B MoE failed 5 out of 7 times on a 48GB machine but only once on a 64GB machine, suggesting memory significantly impacts output quality despite identical model settings. The experience revealed that task characteristics like required code search, number of files to edit, instruction specificity, and tech stack heavily influence whether local models remain viable alternatives to larger cloud-based models.