Hugging Face Blog
·
2 weeks ago
Researchers released ScarfBench, an open benchmark for evaluating AI agents on Java framework migration tasks across Spring, Jakarta EE, and Quarkus. The benchmark contains 34 applications, 204 migration tasks, and approximately 151,000 lines of code, with success measured by whether applications build, deploy, and preserve behavior. Current frontier AI agents achieve less than 10% behavioral success on the benchmark, revealing that dependency management across configuration, infrastructure, and runtime environments—rather than code translation itself—is the primary migration challenge.
Together AI
·
2 weeks ago
Together AI published nine research papers at ICML 2026 spanning AI agent development, model reasoning techniques, and inference optimizations across the full ML stack. Key results include DSGym evaluating data-science agents across 1,000+ tasks, Aurora achieving 1.25× speedup through adaptive speculative decoding in production, and ParallelKernelBench establishing the first multi-GPU kernel generation benchmark with 87 workloads. These advances enable faster inference, better reasoning without verifiers, and more efficient use of GPU resources for frontier AI systems.
Hugging Face Blog
·
2 weeks ago
Every Eval Ever and Hugging Face Community Evals have integrated their evaluation result systems to enable cross-posting and linking of benchmark scores across platforms. The combined datastore now contains approximately 229,000 evaluation results across 22,000 models and 2,200 benchmarks, drawn from 31 different reporting formats. Users can now submit evaluation results to both platforms simultaneously using a converter tool, with results appearing on model pages and linking back to full standardized records for reproducibility and interpretation.
OpenAI Blog
·
2 weeks ago
GeneBench-Pro is a new benchmark for testing AI systems on genomics and biology tasks using real-world datasets. The benchmark evaluates performance across complex scientific research problems in these domains. Organizations can now measure how well AI models handle actual genomic data rather than simplified test cases.
