Incentivizing Temporal-Awareness in Egocentric Video Understanding Models

Apple · 3 months ago · 39

Researchers developed Temporal Global Policy Optimization (TGPO), a reinforcement learning algorithm that improves multimodal large language models' understanding of temporal sequences in egocentric videos by contrasting outputs from correctly ordered versus shuffled frames. TGPO was evaluated on five egocentric video benchmarks and consistently improved temporal grounding and causal coherence compared to prior RL-based approaches. The method enables MLLMs to better understand event ordering and narrative progression in first-person video understanding tasks.