In plain words: Loopie reuses the same small set of layers over and over instead of adding more of them, so a model can spend extra training compute by looping rather than growing. It beat plain transformers trained on the same compute budget.
Abstract · Loop the Loopies!
We present the Loopie series, consisting of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6B-parameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N times increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. With a novel post-training method, Loopie develops strong reasoning abilities and achieves frontier-level reasoning performance.
Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang, Ran Tao, Joey Zhou, Bryan Dai
arXiv:2607.16051 · cs.CL, cs.AI · submitted Jul 17, 2026 · updated Jul 20, 2026
abstract · pdf · html