CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning
Apple ML Research
CLaRa is a framework that improves retrieval-augmented generation by compressing documents and jointly optimizing retrieval and generation in a shared continuous space. The method achieves state-of-the-art performance on QA benchmarks at a compression rate of 16, meaning documents are reduced to 1/16th their original length while maintaining answer quality. This unified approach allows gradients to flow through both retrieval and generation modules, aligning document relevance with answer quality during training.
Why it matters
Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but still suffers from long contexts and disjoint retrieval–generation optimization. In this work, we propose CLaRa (Continuous Latent Reasoning), a unified framework that performs embedding-based compression and joint optimization in a shared continuous space. To obtain semantically rich and retrievable compressed vectors, thereby reducing the document length fed into the generator, we introduce SCP, a key-preserving data synthesis framework based on question-answering and paraphrase…
