HybridAgent Thinks in Latent Space and Acts in Text
StackSweep
Stanford, UIUC and Together AI built an agent that thinks in hidden states, not text. It kept accuracy up while cutting tokens, time and cost.
Based on reporting by StackSweep — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Stanford, UIUC and Together AI are betting that an agent does not need to spell out every thought. HybridAgent keeps most of the reasoning inside the model’s hidden states, then switches back to ordinary text only when it needs to call Python, search, or answer a person.
The result is a single generation stream with two modes. Text starts the task and handles the visible bits. Latent steps carry the reasoning in the background, with a small adapter called HybridLink turning hidden states back into the input embedding space so the model can keep going without decoding every step into words.
On Qwen3 backbones at 4B, 8B, and 14B, that setup beats matched text-only agents on the main benchmarks the paper highlights. At 14B, HybridAgent scores 52.4 on GAIA versus 43.7 for the ARPO text-RL baseline, and 49.5 on WebWalker versus 40.5. It also uses 35.5% to 54.7% fewer tokens and runs 1.8x to 2.6x faster, with estimated decoding cost per million questions dropping from $470 to $176 at 14B and from $359 to $98 at 4B.
The paper makes a point of showing that the latent steps are doing real work, not just burning compute in secret. Swapping in another problem’s latents drops MATH500 from 88.2 to 48.6. Forcing the model to stay in text-only mode also hurts, taking it down to 78.8 on MATH500, below the plain GRPO text agent. And a nearest-token readout of the latent states is close enough to produce a pseudo-text trace you can inspect.
There are limits. The gains are measured on Qwen3-Base at 4B to 14B, not some giant frontier system. Current inference engines do not support this multi-turn hybrid setup out of the box, so the team built custom training and inference support. Still, the basic message is hard to ignore: a lot of agent text is just the model thinking out loud for nobody in particular.
My take — AI-written commentary, not fact-checked reporting
This is the rare AI paper that attacks a real waste: models narrating their own internal monologue like a corporate meeting transcript. The catch is obvious too — hidden reasoning is cheaper and faster, but it also gives providers more control and everyone else less visibility. That tradeoff is becoming the whole business model, whether the demos admit it or not.
Read more about this at: StackSweep