DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding

Together AI ran a head-to-head on DeepSWE, its benchmark of 113 real, long-horizon feature requests pulled from live open-source repos, four trials per task, graded pass/fail by a hidden test suite. That's 900 rollouts total. On its own, GPT-5.6 Luna is simply the better engineer: 67.2% pass@1 against DeepSeek-V4 Flash 0731's 53.3%, a 14-point gap that holds at every equal attempt count. If quality alone decided things, this piece would be one sentence long.

But price changes the calculus fast. DeepSeek-V4 Flash costs about $0.10 per rollout versus Luna's $0.61, which works out to 532 solves per $100 compared with Luna's 110. The catch is time, not accuracy: DeepSeek is the slower of the two, taking a median 23 minutes and 148 steps against Luna's 16 minutes and 92, and it burns through more output tokens along the way. It grinds toward answers rather than sprinting to them.

Luna's edge shows up hardest where reasoning matters most. It wins seven of eight task domains and sweeps all five programming languages tested. DeepSeek's one real win is query-and-config work, the SQL builders and parsers, where it actually edges ahead. Its JavaScript, by contrast, collapses to 35% against Luna's 60%, the weakest cell in the whole matchup. DeepSeek does fail more gracefully, though: when it misses, it breaks the repo's existing test suite in only 9% of failures versus Luna's 15%.

The interesting result is what happens when you chain them. Run DeepSeek first and escalate to Luna only when the test suite rejects the answer, and you solve 78.9% of tasks at $0.385 each — more accurate than Luna alone and roughly 37% cheaper. That even beats a theoretical perfect one-shot router, which tops out at 74.3%, because two attempts beat one lucky guess. The cheap model clears roughly half the queue for a dime apiece, leaving Luna to spend its budget only on the genuinely hard remainder.

The two models don't overlap all that much either: per-task correlation sits at 0.50, and their combined coverage reaches 106 of 113 tasks. Luna alone lands four tasks DeepSeek never touches; there's no equivalent case running the other way. Seven tasks defeat both models entirely, which suggests the ceiling here isn't a routing problem, it's a capability one. As a standalone tool, DeepSeek-V4 Flash 0731 is mid-pack with a glaring JavaScript hole. As the first stage of a cascade in front of a pricier flagship, it earns its keep.