Researchers introduced Dory1.0-4B, a 4-billion-parameter looped (latent-recurrent) language model built on a normalized Transformer (nGPT) backbone, with a shared recurrent block that can run a variable number of times at inference. A single checkpoint thus serves different compute budgets, and with two or more loops it outperforms larger models like Qwen3.5-9B and Gemma 4 12B across 12 benchmarks. The checkpoints and multiple quantizations are open-sourced.
A YouTube video by Sam Witteveen walks through NVIDIA's Dory, a 4B looped transformer released as a research model. The video covers looped-architecture mechanics, KV-cache handling, benchmarks against Gemma 4 26B, and deployment.
A YouTube video by Sam Witteveen walks through NVIDIA's Dory, a 4B looped transformer released as a research model. The video covers looped-architecture mechanics, KV-cache handling, benchmarks against Gemma 4 26B, and deployment.
A new arXiv paper introduces ARLoop, a looped transformer architecture that aggregates representations across recurrent depth via attention over depth instead of standard residual connections. The authors release fully open pre-trained looped LLMs: a 1.2B model trained on only 0.5T tokens with three recurrences that outperforms Huginn-3.5B across all evaluated tasks. The result suggests parameter-efficient scaling via recurrence can be competitive with larger non-looped models like Qwen3.5-2B.
A new arXiv paper introduces ARLoop, a looped transformer architecture that aggregates representations across recurrent depth via attention over depth instead of standard residual connections. The authors release fully open pre-trained looped LLMs: a 1.2B model trained on only 0.5T tokens with three recurrences that outperforms Huginn-3.5B across all evaluated tasks. The result suggests parameter-efficient scaling via recurrence can be competitive with larger non-looped models like Qwen3.5-2B.
An arXiv paper introduces LoopCD, a training-free contrastive decoding method for looped Transformers that contrasts the final prediction with an earlier recurrent pass to guide token selection. Across four looped Transformer families it raises benchmark scores at full depth and enables halving recurrent loops while matching or exceeding full-depth baselines, cutting forward FLOPs by 22.5% to 48.2%.
An arXiv paper introduces LoopCD, a training-free contrastive decoding method for looped Transformers that contrasts the final prediction with an earlier recurrent pass to guide token selection. Across four looped Transformer families it raises benchmark scores at full depth and enables halving recurrent loops while matching or exceeding full-depth baselines, cutting forward FLOPs by 22.5% to 48.2%.
A new paper introduces Continuous Depth Batching, a scheduling method that makes depth-adaptive inference practical for looped language models by allowing tokens to exit the batch at different loop counts within a single forward pass. This overcomes the limitation of standard token-level schedulers like vLLM, which cannot handle variable loop counts. The approach promises to reduce compute for easy tokens while preserving quality for hard ones.
A new paper introduces Continuous Depth Batching, a scheduling method that makes depth-adaptive inference practical for looped language models by allowing tokens to exit the batch at different loop counts within a single forward pass. This overcomes the limitation of standard token-level schedulers like vLLM, which cannot handle variable loop counts. The approach promises to reduce compute for easy tokens while preserving quality for hard ones.
An arXiv paper introduces RecurTrace, a method improving latent recurrent reasoning in language models. It addresses two limits of prior recurrence approaches: loop iterations cannot access earlier computations, and fixed loop counts misallocate depth. This matters to practitioners because it increases effective inference depth without added parameters or extra tokens.
An arXiv paper introduces RecurTrace, a method improving latent recurrent reasoning in language models. It addresses two limits of prior recurrence approaches: loop iterations cannot access earlier computations, and fixed loop counts misallocate depth. This matters to practitioners because it increases effective inference depth without added parameters or extra tokens.