MolmoMotion: Language-guided 3D motion forecasting

Allen Institute (AI2) · 4 weeks ago

Researchers released MolmoMotion, a model that predicts how 3D points on objects will move in the future based on video frames and language instructions describing actions. The model achieved 0.109 meters average displacement error on the PointMotionBench benchmark, outperforming existing forecasting methods. The model and accompanying datasets enable applications in robot manipulation planning and controllable video generation.