Capturing token IDs during agentic interactions for better reinforcement learning
Amazon Science
Anthropic released Turnstile, a proxy tool written in Rust that captures exact token-level data during reinforcement learning training of language models on multi-step tasks. Turnstile records token IDs, log probabilities, and loss masks at the moment of generation without modifying existing agent harnesses, solving the problem that transcript-based data loses critical information needed for effective RL training. The system enables RL training runs with existing agent harnesses as black boxes while handling complexities like mixture-of-experts routing and multimodal inputs from vision-language models.
Why it matters
A new Rust proxy called Turnstile sits between the model backend and the agent harness to capture information lost in mere text transcripts.
