Kimi K3 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

Together AI · 2 months ago · 35 ● 33 sources

Kimi K3 and GPT-5.6 Sol were compared on DeepSWE, a software engineering benchmark. GPT-5.6 Sol wins on single-shot accuracy at 72.7% pass@1 versus Kimi K3's 68.5%, but Kimi K3 achieves better multi-attempt performance at 89.4% pass@4 for $4.65 per rollout compared to Sol's $8.37, delivering 2.8 times more solved tasks per dollar. Routing between the two models—running Kimi K3 first and escalating to Sol on test failures—reaches 85.6% accuracy while beating either model alone and reducing cost per solved task.