Apple's New Speech API vs Whisper: The First Real Benchmark
Inscribe
Apple's new SpeechAnalyzer engine beats every Whisper model in a fresh benchmark on the same hardware. It's also faster, meaning on-device English transcription just got a new best-in-class option.
Based on reporting by Inscribe — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Somebody finally ran the numbers, and the result cuts against what a lot of developers have assumed for a while. Apple quietly swapped SFSpeechRecognizer for a new pair of APIs, SpeechAnalyzer and SpeechTranscriber, in iOS 26 and macOS 26, but published no accuracy figures of its own. So the team behind Inscribe, a private on-device transcription app that ships both Apple's engines and three sizes of Whisper, decided to test all five side by side, on the same M2 Pro machine, through the exact production code their users actually run.
The headline number is stark. On LibriSpeech's clean test set, SpeechAnalyzer posts a 2.12% word error rate, comfortably ahead of Whisper Small's 3.74%, and it does that while running roughly three times faster per second of audio. On the noisier test-other split the gap holds: 4.56% versus 7.95%. Meanwhile the API Apple just retired, SFSpeechRecognizer, actually finishes last on clean speech, worse even than Whisper Tiny, a 40-megabyte model. That's not a subtle regression argument for migrating away from the legacy API — it's a 3.5-to-4x cut in error rate on identical audio, with no accuracy trade-off to weigh against it.
What makes this worth trusting rather than shrugging at is the methodology, which the team built specifically to survive suspicion. Because they publish transcription software that includes Whisper, they validated their own harness against OpenAI's own published Whisper numbers on LibriSpeech, and landed within a small, consistently positive offset on all six measurements — the kind of tight, one-directional gap that signals a stricter normalizer and quantization, not sloppy measurement. They also released the raw transcripts for both Apple engines, all 5,559 utterances each, so anyone skeptical of their scoring can rescore it themselves.
Whisper isn't obsolete, and the piece is careful not to pretend otherwise. It still covers far more languages than SpeechTranscriber's roughly 30 locales, and it runs on any platform, not just current Apple hardware running OS 26. But for English transcription on a modern iPhone or Mac, the argument that Whisper is automatically the more accurate choice no longer holds up. The team even changed their own product's defaults on the back of this data, now preferring SpeechAnalyzer wherever it supports the language.
There's a smaller, almost funnier subplot buried in the writeup: building the benchmark surfaced a real bug in Inscribe itself, where a missing finalization call made Apple-engine file imports hang indefinitely, invisible until someone actually forced the app to lean on that engine. It's a decent reminder that the value of rigorous testing isn't just the headline chart — it's what breaks when you finally look closely.
My take — AI-written commentary, not fact-checked reporting
Credit where it's due: publishing raw transcripts and reproducing a rival's own published numbers before claiming victory is how a benchmark earns trust, and most vendor comparisons don't bother. The real story here isn't Apple beating Whisper Small on English — it's that on-device speech recognition quietly got good enough that the
Read more about this at: Inscribe
