Fable 5 Vs Opus 4.8: Outcomes-Based Assessments Are A Massive Warning For Frontier AI Labs

Substack · 1 month ago · 10

The author tested Fable 5 and Opus 4.8 on a real-world task of rebuilding a website to improve conversion rates and found both models scored 0 on outcomes-based metrics despite producing functional technical artifacts. Both models failed to include basic features like conversion tracking and security without explicit prompting, and a smaller open-source model (Gemma 4) performed equally well at zero token cost when given sufficient context. Companies like Eli Lilly are moving away from expensive frontier AI models toward smaller purpose-built models fine-tuned on proprietary data, signaling that enterprise customers prioritize outcomes-based value over frontier model capabilities.