Amazon Science
·
1 week ago
Anthropic released Turnstile, a proxy tool written in Rust that captures exact token-level data during reinforcement learning training of language models on multi-step tasks. Turnstile records token IDs, log probabilities, and loss masks at the moment of generation without modifying existing agent harnesses, solving the problem that transcript-based data loses critical information needed for effective RL training. The system enables RL training runs with existing agent harnesses as black boxes while handling complexities like mixture-of-experts routing and multimodal inputs from vision-language models.
TheSequence
·
1 week ago
The article argues that verifiability alone is insufficient for determining whether a domain is suitable for AI development, proposing instead a multi-dimensional framework where domains like mathematics and chess excel because they score highly across multiple properties including grindability. The author contrasts high-performing domains such as code and board games with struggling domains like robotics and open-ended knowledge work, suggesting the latter fail on several unstated axes despite partial strength in others. This framework explains why AI systems have made faster progress in formal domains and why some reinforcement learning environment startups may ultimately disappoint investors despite their high valuations.
Apple ML Research
·
1 week ago
Researchers developed Temporal Global Policy Optimization (TGPO), a reinforcement learning algorithm that improves multimodal large language models' understanding of temporal sequences in egocentric videos by contrasting outputs from correctly ordered versus shuffled frames. TGPO was evaluated on five egocentric video benchmarks and consistently improved temporal grounding and causal coherence compared to prior RL-based approaches. The method enables MLLMs to better understand event ordering and narrative progression in first-person video understanding tasks.
