Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks

MarkTechPost · 2 months ago · 27

Supabase open sourced Evals, a benchmark framework that tests AI coding agents like Claude Code and GPT models on real Supabase development tasks such as schema building and policy debugging. The benchmark uses three dimensions (products, topics, stages) with separate benchmark and regression test suites, combining deterministic checks with LLM-based scoring. The results show top models like Opus 5 and Kimi K3 achieve 100% on build tasks unaided, while skills improve smaller models' performance and reveal agents under-utilize declarative schemas and documentation.