A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth
MarkTechPost Asif Razzaq
A Princeton researcher proposed RLT, a decoder that carries its state from one token to the next. It gives every token 96 blocks of work and claims unbounded temporal depth, but no results yet.
Based on reporting by MarkTechPost, Asif Razzaq — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
A Princeton researcher, Yifan Zhang, has laid out a new transformer design that refuses to wipe the slate clean between tokens. The proposal, called Recurrent Looped Transformer, carries the decoder’s final hidden state and its sliding-window attention cache from one token to the next, across both the prompt and the reply, with no reset at the boundary.
That makes it very different from the usual decoder-only setup, where the last layer of token t does not directly feed token t+1. In RLT, the encoder still processes tokens in parallel, but the decoder keeps the loop alive. The state passed forward includes the final decoder output and the retained cache for every decoder layer. The window includes the current token, so only W-1 older entries remain per layer.
The reference configuration is blunt about the compute: 48 encoder layers tied to 48 decoder layers, which means 96 logical blocks are executed per token. Zhang describes that as parameter reuse rather than activation copying. He also makes a point of not overselling it. This is a design specification, not a benchmark report, and it includes no measured efficiency, reasoning quality, or scaling results.
The pitch is really three ideas stacked together. First, the path through the model grows longer with each token, giving the architecture what Zhang calls latent reasoning with unbounded temporal depth. Second, the design is meant to fit hardware better, with parallel encoder work and batching across independent sequences, while explicitly avoiding claims of an exact parallel scan or a reduced-prefill speedup. Third, the same state transition is meant to serve pretraining, supervised fine-tuning, sampling, and RL replay.
That RL piece is unusually strict. The trainer rebuilds the encoder memory, the recurrent output, and every sliding-window cache from the start of the sequence under current parameters before scoring actions, so old rollout states are never reused. The report also warns that partial detaching is dangerous because gradients can still flow through decoder key-value state even if the final output is detached. For serving, the snapshot has to include the full decoder state, encoder cache and memory, position metadata, the window convention, and the model version.
My take — AI-written commentary, not fact-checked reporting
This is the rare AI paper that sounds more honest than triumphant, which is already refreshing. The interesting move here is not “longer context” marketing fluff; it’s the refusal to pretend recurrence is free, or that state can be half-detached and hand-waved away like a broken toaster. More of the field could use that mood and less keynote confetti.
Read more about this at: MarkTechPost
