about
Training Foundation Models with a Single Algorithm (arxiv.org)
2 points by simonpure 15 days ago | hide | past | pdf | discuss on HN

In plain words: Instead of switching algorithms between training stages, this fits the model to its generated targets in one probability-fitting step, so training becomes choosing which data to mix and when. It reached better accuracy for the same compute and covered more tasks without losing skills.

Abstract

Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms. We show that this algorithmic separation is not fundamental. By deriving suitable RL objectives as maximum-likelihood fitting to model-generated target distributions, we place all of these stages under a single likelihood-based training algorithm, unified training. This perspective turns the foundation-model training lifecycle into a problem of choosing which data to train on, in what proportions, and when, with conventional staged training as a special case. We instantiate this framework in a unified training system that mixes static data and RL generations within a single trainer. Across controlled and real-world experiments, unified training improves the accuracy-compute tradeoff and task coverage while preserving broader capabilities, and provides a stronger starting point for continual learning. This work lays the groundwork for a unified science of foundation-model training, connecting mathematical principles, training infrastructure, and the development of model capabilities.

Yuda Song, Alvin Zhang, Fahim Tajwar, Daman Arora, Jeff Schneider, Daniel Khashabi, Ruslan Salakhutdinov, J. Andrew Bagnell, Aarti Singh, Andrea Zanette
arXiv:2610.40564 · cs.LG · submitted Oct 29, 2026
abstract · pdf · html · 67 pages, 31 figures, 22 tables

add comment on HN