Model Routing Is Simple. Until It Isn’t.

Hugging Face · 3 months ago · 18

A team building model routing for AI agents found that selecting which model to use for each task is not simply a classification problem but requires optimizing across multiple system constraints simultaneously. On the AppWorld Test Challenge with 417 tasks, Claude Sonnet cost $79 total while GPT-4.1 cost $155 despite lower per-token pricing, because Sonnet's cache-read pricing was more efficient for agent workloads that reuse context. Effective routing must balance cost, latency, accuracy, and compliance requirements together rather than treating task difficulty or model speed as isolated variables.