Incentivizing Temporal-Awareness in Egocentric Video Understanding Models
Apple ML Research
Researchers developed Temporal Global Policy Optimization (TGPO), a reinforcement learning algorithm that improves multimodal large language models' understanding of temporal sequences in egocentric videos by contrasting outputs from correctly ordered versus shuffled frames. TGPO was evaluated on five egocentric video benchmarks and consistently improved temporal grounding and causal coherence compared to prior RL-based approaches. The method enables MLLMs to better understand event ordering and narrative progression in first-person video understanding tasks.
Why it matters
Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness, particularly in egocentric settings where reasoning depends on the correct ordering and evolution of events. This deficiency stems in part from training objectives that fail to explicitly reward temporal reasoning and instead rely on frame-level spatial shortcuts. To address this limitation, we propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm designed to incentivize temporal…
