Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards

arXiv cs.AI · 6 hours ago

Researchers established the first non-vacuous generalization bounds for parameter-efficient reinforcement learning with verifiable rewards when fine-tuning billion-parameter language models. The Progressive RLVR framework achieved 84-97% of standard LoRA fine-tuning performance while producing models 14,796x more compressible, with generalization bounds within 6-11% of fine-tuned model accuracy across four domains. This enables theoretical guarantees about how well RLVR-trained models will generalize to unseen data, addressing a gap in understanding these widely-used reasoning improvement techniques.