In plain words: Instead of copying expert examples, the model answers with the example in view, then learns from its own answers so old skills stay intact. This kept new-task accuracy higher and forgetting far lower, letting one model add skills over time without losing them.
Abstract · Self-Distillation Enables Continual Learning
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inherently off-policy. We introduce Self-Distillation Fine-Tuning (SDFT), a simple method that enables on-policy learning directly from demonstrations. SDFT leverages in-context learning by using a demonstration-conditioned model as its own teacher, generating on-policy training signals that preserve prior capabilities while acquiring new skills. Across skill learning and knowledge acquisition tasks, SDFT consistently outperforms SFT, achieving higher new-task accuracy while substantially reducing catastrophic forgetting. In sequential learning experiments, SDFT enables a single model to accumulate multiple skills over time without performance regression, establishing on-policy distillation as a practical path to continual learning from demonstrations.
Idan Shenfeld, Mehul Damani, Jonas Hübotter, Pulkit Agrawal
arXiv:2601.19897 · cs.LG · submitted Jan 27, 2026 · updated Aug 7, 2026
abstract · pdf · html
The paper is about a way to do SFT will less chance of catastrophic forgetting and performance regressions.
The idea is that SFT on new data that was NOT generated by the model (aka "off policy" data) is likely to cause problems due to the statistical mismatch between the new data and what the model has already learnt. As I understand it, their solution is to statistically align the new data with the old by feeding it to the old model, which will hopefully grok it via in-context learning, then have it regenerate it in its own words such that "off policy" data now becomes "on policy". The model can then be SFT trained on this regenerated data (i.e self-distillation).
To me SFT and "continual learning" are two distinct things.
Human/animal continual learning is always-on learning that removes the need for, and distinction between, training and inference, and it initiated by prediction failure. It's as much about skill acquisition as it is about knowledge acquisition. Continual learning can happen in any context from trying to do something (or just observing something passively) and being wrong about the outcome of your own actions, or what some external entity does next, to curiosity/boredom driven exploration and play which is more along the spectrum of pure learning with less expectation of outcomes.
Continual learning is what, one day, will let the AGI intern pick up new skills on the job by trying to do things and failing/learning/practicing until they get better. This is not the same as sending the intern home with a textbook to read, or a transcript of the conversations you had with it today, and having it take these onboard overnight, which is basically what SFT is designed to do - intermittent addition of new declarative data.