Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train

Google Research · 1 month ago · 17 ● 2 sources

Retrieve-for-Train introduces a training-time RL method that compiles efficient, property-aligned query fan-outs so set-valued retrieval can run without test-time LLM “thinking” and latency overhead. It trains a 53.9M-parameter diffusion retriever that delivers a 12 to 20 speedup versus autoregressive approaches. The result is single-pass, sub-second-to-few-seconds inference that still optimizes set-level properties like diversity and groundedness instead of relying on expensive, token-heavy decomposition at query time.