about
Scaling Reinforcement Learning for Trillion-Scale Thinking Model (arxiv.org)
3 points by mountainview 344 days ago | hide | past | pdf | discuss on HN

In plain words: Built a 1-trillion-parameter open reasoning model that uses only about 50 billion of them per piece of text, plus fixes that keep its learn-by-trial-and-error training stable and fast. It reached silver-medal level on the 2025 IMO math olympiad, the best open model yet.

Abstract · Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model

We present Ring-1T, the first open-source, state-of-the-art thinking model with a trillion-scale parameter. It features 1 trillion total parameters and activates approximately 50 billion per token. Training such models at a trillion-parameter scale introduces unprecedented challenges, including train-inference misalignment, inefficiencies in rollout processing, and bottlenecks in the RL system. To address these, we pioneer three interconnected innovations: (1) IcePop stabilizes RL training via token-level discrepancy masking and clipping, resolving instability from training-inference mismatches; (2) C3PO++ improves resource utilization for long rollouts under a token budget by dynamically partitioning them, thereby obtaining high time efficiency; and (3) ASystem, a high-performance RL framework designed to overcome the systemic bottlenecks that impede trillion-parameter model training. Ring-1T delivers breakthrough results across critical benchmarks: 93.4 on AIME-2025, 86.72 on HMMT-2025, 2088 on CodeForces, and 55.94 on ARC-AGI-1. Notably, it attains a silver medal-level result on the IMO-2025, underscoring its exceptional reasoning capabilities. By releasing the complete 1T parameter MoE model to the community, we provide the research community with direct access to cutting-edge reasoning capabilities. This contribution marks a significant milestone in democratizing large-scale reasoning intelligence and establishes a new baseline for open-source model performance.

Ling Team, Anqi Shen, Baihui Li, Bin Hu, Bin Jing, Cai Chen, Chao Huang, Chao Zhang, Chaokun Yang, Cheng Lin, Chengyao Wen, Congqi Li, et al.
arXiv:2510.18855 · cs.CL, cs.AI · submitted Oct 21, 2025 · updated Oct 25, 2025
abstract · pdf · html · Technical Report

add comment on HN
Also discussed: Oct 2025 (4 points, 0 comments)