Hugging Face Blog
·
3 weeks ago
Researchers released ScarfBench, an open benchmark for evaluating AI agents on Java framework migration tasks across Spring, Jakarta EE, and Quarkus. The benchmark contains 34 applications, 204 migration tasks, and approximately 151,000 lines of code, with success measured by whether applications build, deploy, and preserve behavior. Current frontier AI agents achieve less than 10% behavioral success on the benchmark, revealing that dependency management across configuration, infrastructure, and runtime environments—rather than code translation itself—is the primary migration challenge.
NVIDIA
·
3 weeks ago
● 3 sources
NVIDIA released the BioNeMo Agent Toolkit, which integrates with Anthropic's Claude Science to let life sciences researchers run accelerated computational workflows through natural language commands. The toolkit includes accelerated tools like RAPIDS-singlecell that compress a 1.3-million-cell workflow from 52 minutes to 25 seconds, and nvMolKit that accelerates cheminformatics operations by up to 3,000x. Scientists can now access NVIDIA's accelerated models and libraries directly within Claude Science's conversational environment, with 18 of the top 20 pharmaceutical companies already using BioNeMo.
Microsoft Research
·
3 weeks ago
SkillOpt treats agent skill files as trainable parameters that can be optimized through a controlled training loop rather than manual editing, using bounded text edits and validation gating to improve performance without modifying model weights. Across six benchmarks, seven models, and three execution modes (52 evaluation cells total), SkillOpt achieved best or tied-best results, with GPT-5.5 improving from 58.8 to 82.3 on average across benchmarks. The optimized skills remain compact (median 920 tokens with one to four accepted edits), transfer across model scales and execution environments, and enable smaller models to match larger baselines without additional inference costs.
NVIDIA
·
3 weeks ago
Jaiveer Singh leads NVIDIA's Isaac ROS team, which develops software infrastructure for robotics developers to build autonomous robots faster by providing modular, CUDA-accelerated packages built on open source ROS 2. Isaac ROS offers developers pre-built components for perception, object detection, mapping, and motion planning that run on NVIDIA's Jetson edge systems and can be customized like modular building blocks. By releasing robotics software as open source, NVIDIA enables developers to inspect, modify and trust the platform over multi-year development cycles, allowing more robotics startups and builders to accelerate their progress.
NVIDIA
·
3 weeks ago
NVIDIA is providing reusable workflows and blueprints for building vision AI agents that analyze video data at the edge using synthetic data generation and model fine-tuning. A benchmark with Corning showed that a model trained on eight real defect images plus synthetically generated defects achieved 95% average precision, compressing a multi-quarter project into days. Organizations can now deploy vision AI agents faster by using pre-built skills for defect generation, video augmentation, and model fine-tuning instead of rebuilding workflows from scratch.
TLDR
·
3 weeks ago
● 2 sources
A product designer argues that when AI systems are hard to evaluate, it reflects poor product design rather than an evaluation problem, and demonstrates how to restructure products to make verification easier by showing working, sources, and incremental changes rather than opaque final outputs. The examples span data agents showing 50% more detail through notebooks, PE lesson planners anchored to vetted templates with diffs, and medical report generators that surface contradictions and source citations before the final document. Redesigning products for verifiability reduces user friction, enables better automated testing, and creates feedback loops that improve quality over time.
Hugging Face Blog
·
3 weeks ago
Every Eval Ever and Hugging Face Community Evals have integrated their evaluation result systems to enable cross-posting and linking of benchmark scores across platforms. The combined datastore now contains approximately 229,000 evaluation results across 22,000 models and 2,200 benchmarks, drawn from 31 different reporting formats. Users can now submit evaluation results to both platforms simultaneously using a converter tool, with results appearing on model pages and linking back to full standardized records for reproducibility and interpretation.
