GLM-5.3 vs. GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing
Together AI ● Covered by 3 sources
GLM-5.3 is cheaper and better on retries; GPT-5.6 Sol only wins first-shot coding. Run GLM first, then escalate, and the pair beats Sol alone on DeepSWE.
Based on reporting by Together AI — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
GPT-5.6 Sol still takes the single-shot crown on DeepSWE, but GLM-5.3 closes the gap fast once retries are allowed. On one attempt, Sol scores 72.7% to GLM-5.3’s 69.0%. By two attempts they’re basically tied, and by four tries GLM-5.3 moves ahead at 87.6% versus 85.8%.
The bigger surprise is the bill. GLM-5.3 costs $3.99 per rollout, about 2.1 times less than Sol’s $8.37. That works out to 17 solved tasks per $100 for GLM-5.3, versus 9 for Sol. Sol is the faster machine, though: 19 minutes and 61 steps per rollout, compared with GLM-5.3’s 35 minutes and 124 steps.
The two models also fail differently. Sol breaks already-passing tests in 20% of its failures, while GLM-5.3 does that in 11%. GLM-5.3 is also the model that reaches a wider set of tasks overall, with 87.6% coverage against Sol’s 85.8%, even though Sol has the stronger four-for-four reliability at 84.5% versus 78.8%.
That difference makes routing the real story. Run GLM-5.3 first and only send the problem to Sol if the test suite rejects the answer, and the cascade solves 85.9% of tasks at $6.61 each. That beats Sol alone on both accuracy and cost, because the open model clears most of the queue cheaply and Sol handles the hard leftovers.
The split is also clean by language. GLM-5.3 stands out on JavaScript and Rust, while Sol does better on Python, Go, and TypeScript. The two models disagree enough to matter, with a per-task correlation of 0.43, which is why the best setup here is not choosing a winner. It’s using both in order.
My take — AI-written commentary, not fact-checked reporting
This is the kind of result that should annoy anyone still worshipping a single “best model.” The boring answer wins: use the cheaper model first, then pay for the fancy one only when the tests complain. Open models don’t need to be mythical; they just need to be good enough to make the closed model earn its keep.
Read more about this at: Together AI