I trust Claude for everything. This test made me rethink that.

xAI dropped Grok 4.5 on July 8, and buried in the training data was something unusual: real Cursor session logs. The pitch that followed was blunt. xAI claims Grok 4.5 performs roughly on par with Claude Opus 4.8 on coding work, but does it using about 4.2 times fewer output tokens. Pricing backs up the pitch, too — Grok runs $2 per million input tokens and $6 per million output, versus Opus at $5 and $25. That's less than half the cost on both ends, before anyone even opens an editor.

Benchmarks mostly support the story. On Terminal-Bench 2.1, which measures how a model handles real command-line work, Grok posted 83.3% against Opus's 78.9%. SWE-Bench Pro, a tougher test built around fixing actual bugs from open-source projects, still favors Opus. So the honest version of xAI's claim isn't "better." It's "just as good, for a fraction of the spend."

One developer decided to check that claim directly, running both models inside Cursor's Agent mode on fd, sharkdp's Rust file-finder, chosen specifically because it has a long, documented bug history to draw from. Three tasks ran in the same repo: a bug fix pulled from a real 2021 issue, a refactor of a bloated config function, and a new feature flag. Everything stayed identical between runs except the model.

The bug fix went to Opus — it finished in about 30 seconds on 174.1K tokens versus Grok's 46 seconds and 210.2K tokens across two requests, though both produced the exact same one-line fix and passed all 70 tests. But the other two tasks flipped hard. On the refactor, Grok wrapped up in 83 seconds using 197.1K tokens and $0.27, while Opus took roughly five and a half minutes and burned 953.7K tokens for $1.67 — nearly five times the tokens for an equivalent result. The feature build was even more lopsided: Grok finished in 97 seconds for $0.54, Opus took about the same five-and-a-half-minute stretch but spent 3.2 million tokens and $3.25, touching one extra file and adding a man page entry that Grok skipped.

Add it all up and Opus used 4.3 times the tokens Grok did across the three jobs — almost exactly the 4.2x gap xAI advertised, and slightly wider. Total cost came to $1.00 for Grok against $5.14 for Opus, total time about 3 minutes 46 seconds versus 11 minutes 40 seconds. Worth noting: Grok ran on Cursor's "fast" tier while Opus used its "thinking" tier, and Grok's pricing included a 50 percent promo — even doubled, its three jobs would land around $2, still less than half of Opus's tab.