Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

Large language models have a bad habit: they don't know when to stop. Ask one for a short answer and it might ramble for 500 tokens anyway, burning compute and money along the way. Most fixes so far have worked at the sequence level, basically nudging the whole response to be shorter or longer, which is a blunt instrument for a problem that's really about individual decoding steps.

A team from UC Santa Barbara, Carnegie Mellon, LMSYS, and University of Wisconsin-Madison, working with Apple, took a different angle. Their system, called LenVM, treats remaining generation length as a value-estimation problem, the same math used in reinforcement learning. Every generated token gets a small negative reward, and LenVM learns to predict a bounded, discounted return that acts as a stand-in for how many tokens are left before the model should wrap up. No human labeling required, no extra annotation pipeline. The signal comes for free from the generation process itself.

The numbers are the interesting part. On LIFEBench's exact length matching benchmark, slapping LenVM onto a 7B model pushed the length score from 30.9 up to 64.8, beating frontier closed-source models that are presumably far larger and far more expensive to run. On GSM8K, when the team capped the token budget at 200, a baseline model managed just 6 percent accuracy, basically falling apart under the constraint. With LenVM guiding generation, accuracy held at 63 percent. That's not a marginal tweak, that's the difference between a model that's useless under a tight budget and one that still does its job.

Beyond squeezing efficiency out of a fixed budget, LenVM can predict total output length just from looking at the prompt, and it gives researchers a token-by-token view into how specific words or phrases push a model toward longer or shorter reasoning chains. That interpretability angle matters as much as the efficiency gains, since it turns length control from a black-box knob into something you can actually inspect and reason about. The authors frame this as a first step toward using length-aware value signals inside future reinforcement learning training, not just as an inference-time trick.