AI Snake Oil
·
3 months ago
Researchers introduced open-world evaluations, a new method for testing AI capabilities in complex real-world tasks beyond standard benchmarks, and launched CRUX, a collaboration of 17 researchers that successfully tasked an AI agent with building and publishing an iOS app to the App Store. In CRUX's first experiment, the agent completed the task after two errors, one requiring manual intervention, with the entire process costing approximately $1,000. This approach aims to provide early warnings about emerging AI capabilities and identify blind spots in existing benchmarks before such abilities become widespread.
Deep Learning Weekly
·
3 months ago
This week's deep learning newsletter covers new AI model releases including Stanford's 2026 AI Index Report, Google's Gemini Robotics ER 1.6, Meta's Muse Spark multimodal model, and various tools for evaluation and agent development. Key concrete details include Claude Opus 4.6 autonomously reimplementing a 16,000-line bioinformatics toolkit estimated to require 2-17 weeks of human work, and I-DLM matching autoregressive model quality while achieving 3x higher throughput than prior diffusion language models. These releases enable faster model inference, improved robotics reasoning, new coding capabilities, and better tools for debugging and evaluating AI agents in production systems.
Google Research
·
3 months ago
Google researchers introduced Simula, a framework for generating synthetic datasets using mechanism design and reasoning-first principles rather than manual prompts or evolutionary algorithms. The system decomposes dataset generation into controllable axes including global diversification, local diversification, complexification, and quality checks, and has been deployed across Google's products including Gemma models, Gemini safety classifiers, and user protection features like scam detection. The work demonstrates that synthetic data generation can be treated as a controllable science, enabling specialized AI models to be trained on high-fidelity datasets in domains where real-world data is scarce or inaccessible.
Google Research
·
3 months ago
Google Research developed MoGen, an AI model that generates synthetic neuronal shapes to improve the accuracy of brain mapping reconstruction algorithms. Training the PATHFINDER neuron reconstruction model with 10% synthetic data from MoGen reduced reconstruction errors by 4.4%, which translates to 157 person-years of manual proofreading saved for a complete mouse brain. The approach enables faster and more scalable brain mapping by reducing the manual verification work required from human experts.
OpenAI Blog
·
3 months ago
Anthropic updated its Codex app to include computer use capabilities, web browsing, image generation, memory features, and plugin support on macOS and Windows. The app now enables developers to automate tasks across their entire system rather than working within a single application. This integration of multiple tools into one interface is intended to streamline development processes and reduce context-switching between separate applications.
OpenAI Blog
·
3 months ago
OpenAI released GPT-Rosalind, a reasoning model designed to assist with drug discovery, genomics analysis, and protein research tasks. The model applies advanced reasoning capabilities to scientific workflows that typically require extensive computational analysis and expert interpretation. Researchers in life sciences can now use GPT-Rosalind to process complex biological data more efficiently than previous general-purpose AI systems.
Hugging Face Blog
·
3 months ago
The Sentence Transformers library now supports training and finetuning multimodal embedding models that handle text, images, audio, and video across tasks like visual document retrieval and semantic search. A finetuned Qwen3-VL-Embedding-2B model achieved an NDCG@10 score of 0.947 on visual document retrieval, compared to the base model's 0.888 and outperforming all tested competitors including models four times larger. Users can now train domain-specific multimodal models using the same SentenceTransformerTrainer component used for text-only models, with automatic handling of image preprocessing and support for multiple loss functions including CachedMultipleNegativesRankingLoss.
Hugging Face Blog
·
3 months ago
The MLX team created a Skill and test harness to help contributors port language models from the transformers library to mlx-lm using AI agents, while maintaining code quality standards. The Skill automates scaffolding tasks like downloading model variants and running per-layer comparisons, but produces PRs that disclose they were agent-assisted and include generation examples and numerical comparisons for reviewer confidence. This approach aims to make new transformers models available in mlx-lm shortly after their release, while ensuring maintainers can review agent-generated code without being overwhelmed by volume.
Hugging Face Blog
·
3 months ago
Researchers extended a reinforcement-learning framework to train e-commerce conversational agents across eight realistic shopping tasks (product discovery, cart building, returns, order tracking, and others) using procedurally generated problems and algorithmically verifiable rewards. The system uses a 12-axis difficulty curriculum that scales task complexity across dimensions like constraint count, distractor products, and mid-conversation stock changes, with training conducted over 300 steps on a Qwen 3 8B model. This approach eliminates the need for human annotation or LLM judges by making all outcomes—product correctness, variant selection, hallucination detection—directly verifiable through code.
OpenAI Blog
·
3 months ago
● 2 sources
OpenAI launched Trusted Access for Cyber, a program that provides security firms and enterprises with GPT-5.4-Cyber and $10 million in API grants to improve cyber defense capabilities. The initiative offers $10 million in total API credits to participating organizations. Security companies and enterprises can now access specialized AI models designed for cybersecurity tasks, potentially reducing response times to threats.
