CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning
Apple
Apple researchers built CLaRa, a system that compresses documents before an AI model reads them. It shrinks text 16x and still beats models fine-tuned the normal way.
Based on reporting by Apple — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Retrieval-augmented generation sounds simple on paper: pull relevant documents, hand them to a language model, get a better answer. In practice it's messy. Stuffing long documents into a model's context window slows everything down, and the two halves of the system — the retriever that finds documents and the generator that writes the answer — are usually trained separately, optimizing for different goals that don't always line up.
A team from Apple and the University of Edinburgh, led by Jie He and colleagues, built something called CLaRa, short for Continuous Latent Reasoning, to close that gap. Instead of treating retrieval and generation as two disconnected steps, CLaRa compresses documents into dense vector representations and does its reasoning in that shared continuous space rather than in raw text. The goal is to feed the generator something much shorter without losing the substance of the original document.
To make those compressed vectors actually useful, the team introduced a data synthesis method called SCP, which is built around question-answering and paraphrase supervision and designed to keep key information intact even after compression. From there, CLaRa trains the reranker — the part that decides which documents matter — and the generator together, using a single language modeling loss. Gradients flow through both components at once, made possible by a differentiable top-k estimator that lets the ranking step stay part of the same optimization loop instead of sitting outside it.
That end-to-end setup is the real bet here: align what the retriever considers relevant with what actually makes the generator's answer better, rather than training each piece to its own separate metric and hoping they mesh. According to the paper, it works. Across several QA benchmarks, CLaRa hit state-of-the-art compression and reranking results, and it kept that edge even when compressing text down by a factor of 16 — beating baselines that were fine-tuned directly on raw text rather than compressed representations.
My take — AI-written commentary, not fact-checked reporting
Nobody outside a research lab will touch CLaRa this year, but the instinct behind it is the one worth watching: stop bolting a retriever and a generator together and start training them as one system with one loss function, because that's where the wasted accuracy has been hiding all along. It's also a small window into how research talent moves — a chunk of this work happened while contributors were at Apple, not necessarily where they are now, which says something about how fluid AI labs' rosters really are underneath the polished paper credits.
Read more about this at: Apple
