MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning

arXiv cs.CL · 6 hours ago

Researchers developed MEMORA, a system that builds embodied action memory from egocentric videos to enable robots to plan long-horizon tasks by maintaining typed stores of environmental, entity, activity, and inferred knowledge. The system was evaluated on 45 hours of EPIC-KITCHENS-100 videos from 18 participants, achieving up to 20.5 point improvement in memory-assessment accuracy over baselines and 16.6% relative improvement on out-of-distribution planning tasks. The approach demonstrates that robots can use consolidated experience memory to ground language-based plans for downstream control.