You only need the frontier model for one single edit

stencil.so · 1 month ago · 6

A benchmarking study finds that using an expensive frontier model to create a plan before handing off to a cheaper model for execution costs more than using the frontier model alone, because the expensive operation is reading context rather than editing code. The hybrid approach with Claude Opus and Gemini Flash costs $3.18 per task versus $2.78 for Opus alone, while achieving the same 84.6% pass rate on SWE-Bench Pro. The more effective approach, called /prewalk, has the frontier model start the task and swap to the cheap model after the first edit and a todo list are generated, reducing costs to $1.46 (47% cheaper than Opus solo) while maintaining 92% of performance.