Gradient-based Planning for World Models at Longer Horizons

BAIR · 2 months ago

Researchers propose GRASP, a gradient-based planning method for world models that addresses long-horizon planning challenges by optimizing over both actions and states in parallel, adding stochasticity for exploration, and reshaping gradients to avoid brittle state-input optimization through vision models. The method lifts the dynamics constraint to a soft penalty, enabling parallel computation across time steps and avoiding exponential Jacobian conditioning from sequential rollouts that causes exploding/vanishing gradients. By treating states as optimization variables rather than computed through repeated model applications, GRASP mitigates adversarial robustness vulnerabilities in deep learning-based dynamics models that make naive state optimization unstable.