DeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

Together AI · 1 week ago · 19 ● 2 sources

Researchers compared DeepSeek V4 Pro 0813 and GPT-5.6 Sol on DeepSWE, a software engineering benchmark with 113 tasks across multiple languages and domains. Pro costs $0.24 per rollout while Sol costs $8.37—a 35x difference—with Sol achieving 72.7% pass@1 versus Pro's 62.8%, but Pro reaching 88.5% pass@4 versus Sol's 85.8%. A cascading approach that runs Pro first and escalates to Sol only on failures solves 83.0% of tasks for $3.35 each, outperforming either model alone and beating a perfect oracle router.