about
The world model remembers, the actor forgets: dissecting AI forgetting on 1 GPU (arxiv.org)
2 points by gurpnijjer 71 days ago | hide | past | pdf | discuss on HN

In plain words: When an agent trains on task after task while keeping past experience, its world model remembers old tasks but the actor forgets. Having the actor copy its own best imagined rollouts kept 3 of 3 four-task chains, versus 0 of 3 with plain replay.

Abstract · The World Model Remembers, the Actor Forgets: Dream Rehearsal for Continual Model-Based RL

Model-based reinforcement-learning agents of the DreamerV3 family forget catastrophically when trained on task sequences, even when an unbounded replay buffer preserves every earlier experience. We ask a question the continual-RL literature has assumed an answer to but never measured: which component forgets? Under never-clear replay, pre-registered component-level probes (n=3 seeds throughout) show that the world model retains essentially everything measurable about old tasks -- reward discrimination (retention ratio ~1.0), value estimates, and termination structure -- while the actor's behavior collapses. Forgetting in this regime is a channel problem, not a memory problem. We demonstrate this by intervention: with the world model frozen and identical imagined rollouts, reinforcement learning in imagination fails to recover a lost skill (0/3 seeds), while supervised self-imitation on the world model's own graded dreams recovers it on 3/3 seeds with zero environment interaction. Interleaved during training, this graded dream rehearsal yields a task-label-free, parameter-constant continual learner: 3/3 four-task chains retained where plain replay passes 0/3, 3/3 eight-task chains, and consistent gains over matched real-episode cloning (paired difference +0.13, bootstrap 95% CI [0.07, 0.24], complete seed separation). The dream-grading step is load-bearing: we characterize two scoring failure modes, provide an offline selection gauge that caught both before they contaminated results, and give a realized-first grading rule that closes them. All experiments were pre-registered with committed protocols; every refuted hypothesis is reported.

Gurp Nijjer
arXiv:2607.19749 · cs.LG, cs.AI · submitted Jul 22, 2026
abstract · pdf · html · 11 pages, 2 figures. Code, pre-registration trail, and run data: https://github.com/gurpnijjer/dream-rehearsal

add comment on HN