A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism

arXiv cs.CL · 18 hours ago

Researchers tested Group Relative Policy Optimization (GRPO) on 4B to 8B parameter language and vision-language models for web agent tasks and found no configuration improved success rates on tasks the supervised baseline had largely mastered. Across 18 controlled runs varying learning rate, KL weight, and other hyperparameters, moderate to high learning rates credibly degraded performance, though GRPO did improve success by 22 percentage points on tasks where the sampled policy outperformed the greedy baseline. The failure mechanism differs by learning rate regime: middle rates degrade specific attention and MLP blocks while high rates cause broader collapse, and this pattern is specific to smaller models as the relationship between effective rank and capability diverges at 8B scale.