DeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing
Together AI ● Covered by 2 sources
Researchers compared DeepSeek V4 Pro 0813 and GPT-5.6 Sol on DeepSWE, a software engineering benchmark with 113 tasks across multiple languages and domains. Pro costs $0.24 per rollout while Sol costs $8.37—a 35x difference—with Sol achieving 72.7% pass@1 versus Pro's 62.8%, but Pro reaching 88.5% pass@4 versus Sol's 85.8%. A cascading approach that runs Pro first and escalates to Sol only on failures solves 83.0% of tasks for $3.35 each, outperforming either model alone and beating a perfect oracle router.
Why it matters
We ran 904 DeepSWE rollouts on DeepSeek V4 Pro 0813 and GPT-5.6 Sol. Sol leads pass@1 by 10 points at 35x the cost; Pro wins pass@4, and a Pro-first cascade hits 83.0%.