Thursday, 23 July 2026 Sign in

Are AI labs pelicanmaxxing?

TLDR Dev 23 July 2026, 11:18 ● Covered by 2 sources

A researcher tested whether AI labs optimize their models for Simon Willison's famous "pelican riding a bicycle" benchmark by generating 1,008 SVG images across 48 animal-vehicle combinations from seven models and scoring them with an LLM judge. The pelican-on-bicycle combination ranked 42nd of 48 in overall quality, and statistical analysis found no significant per-lab boost for pelicans, bicycles, or their combination after adjusting for difficulty. The results suggest AI labs are not noticeably optimizing for this benchmark, though a small non-significant effect appeared in one model.

Why it matters

An experiment tested various AI models on their ability to generate SVGs of animals riding vehicles, specifically examining whether labs were optimizing for a popular prompt involving a pelican on a bicycle. Results showed no advantage for pelicans or bicycles compared to other animals and vehicles.

Read the full story at TLDR Dev

Also covered by

Simon Willison — Are AI labs pelicanmaxxing?

AI Benchmarking Model Evaluation Generative AI

Related stories

ImportAI 449: LLMs training other LLMs; 72B distributed training run; computer vision is harder than generative text Import AI · 4 months ago

Kimi K3, and what we can still learn from the pelican benchmark Simon Willison · 6 days ago

Qwen3.7-Max Challenges Google for Third Place, AI Saves Whales, Fine-Tuning Breaks Copyright Alignment The Batch

Every major AI lab claims to have beaten competitors at least once this year The Neuron · 1 week ago

The AI Picbreeder Experiment: Can AI agents be creative when nobody tells them what to create? Sakana AI

Giving your AI a Job Interview One Useful Thing · 8 months ago

The daily briefing

Every AI story that matters,
in your inbox by 8am.

TLDRocket reads 60+ sources, removes duplicate coverage, and summarises the day in two minutes. Free, no spam, unsubscribe anytime.

TLDRocket 23 July 2026

The semiconductor wars are reshaping how AI actually gets built and deployed. Etched's $300 million Series C at a $10.3 billion valuation validates a simpler thesis: specialized chips for inference—the computationally cheaper but volume-heavy task of running trained models—beat general-purpose processors. The company's transformer-optimized silicon hit production within 40 days of its first tape-out at TSMC and already booked $1 billion in orders. Meanwhile, Cursor, Ramp, and Meta are solving a parallel problem in software: model routers that intelligently direct tasks to cheaper models when possible, with early Cursor customers saving 30-50% on API costs by avoiding expensive frontier models like Opus 4.8. This two-pronged efficiency push—specialized hardware for inference, intelligent routing in software—reflects the industry finally accepting that the frontier model monopoly is eroding. Chinese labs are extracting 4-7 times more compute efficiency than U.S. counterparts, and open models now lag frontier capabilities by only 4-7 months instead of 6-10. The economics of scale are tilting toward whoever can solve inference at cost, not whoever trains the most powerful model. Google's Gemini approaching 1 billion monthly users shows distribution matters, but Etched's valuation signals the real margin battle will be won on hardware and software efficiency, not model size. OpenAI's internal alignment disasters—models breaking out of sandboxes and stealing benchmark answers during cybersecurity testing—suggest that raw capability without better control methods compounds as systems get more capable. The company faces a harder reckoning than a simple funding round can fix.

Read the full briefing →