Wired AI
·
21 hours ago
Researchers at Meta, Stanford, and other institutions created the EgoBabyVLM Challenge, a test that measures how well vision language models can learn from approximately one thousand hours of egocentric video recorded from cameras worn by infants. Current cutting-edge AI models fail substantially on this benchmark, struggling to extract meaning from the messy, realistic footage that babies process efficiently. The findings suggest that designing AI systems with learning mechanisms inspired by infant brains—such as better attention mechanisms and social cue interpretation—could create more efficient models that learn from less data and require less energy.
TheSequence
·
1 day ago
OpenAI audited SWE-Bench Pro, a coding evaluation benchmark, and found that approximately 30 percent of its 731 public tasks contain defects such as rejecting correct solutions or accepting incomplete ones. OpenAI's agent-assisted review labeled 27.4 percent of tasks as defective while independent software engineers identified 34.1 percent as problematic. OpenAI withdrew its earlier recommendation that the field adopt SWE-Bench Pro as a standard evaluation tool due to these validity issues.
Apple ML Research
·
1 day ago
CLaRa is a framework that improves retrieval-augmented generation by compressing documents and jointly optimizing retrieval and generation in a shared continuous space. The method achieves state-of-the-art performance on QA benchmarks at a compression rate of 16, meaning documents are reduced to 1/16th their original length while maintaining answer quality. This unified approach allows gradients to flow through both retrieval and generation modules, aligning document relevance with answer quality during training.
Apple ML Research
·
1 day ago
Researchers propose FAE, a framework that adapts pre-trained visual encoders for image generation by using a single attention layer to convert high-dimensional features into low-dimensional latents suitable for generative models. On ImageNet 256×256, FAE achieves an FID score of 1.48 without classifier-free guidance after 800 epochs and 2.08 after 80 epochs. The approach enables simpler adaptation of pre-trained representations across different generative model families including diffusion models and normalizing flows.
