Patterns for Building Cybersecurity Evals

Eugene Yan · 3 weeks ago

Researchers have developed patterns and benchmarks for evaluating whether AI models can find and exploit security vulnerabilities, including Cybench (40 capture-the-flag tasks with difficulty ranging from 2 minutes to 25 hours), CVE-Bench (40 critical real-world vulnerabilities), and CyberGym (1,507 memory-safety bug instances). The best-performing models achieved only 17.5% success on unguided Cybench tasks and 12.5% on CVE-Bench zero-day scenarios, with all models failing on challenges above 11 minutes difficulty. These evaluation frameworks enable security teams to measure when AI agents become useful for defense versus when they risk assisting attackers, using sandboxed environments, grading mechanisms, and partial-credit scoring along attack chains.