虽然“睡眠”、“做梦”和“自我改进”等术语借用于神经科学，但我们的框架完全在固定的模型架构和人类定义的训练规则内运行。在离线睡眠阶段，模型解锁预分配的混合专家（MoE）内存块，并使用自生成的练习示例更新其数值权重。这种自动化的后训练周期有助于已部署的模型吸收新事实并提高推理准确性，同时保持其核心代码库和设计不变。
对于记忆巩固，分析表明，知识播种可以被理解为一个结构化的低秩压缩问题。模型并不被要求将其短期痕迹的每个细节复制到其参数中。相反，它识别出一组紧凑的方向，最好地保留了模型自身的策略内行为和学习目标所奖励的语义质量。
这一观点解释了两个信号的互补作用：策略内蒸馏将巩固与模型在推理时间实际产生的轨迹对齐，减少了仅使用教师生成分布进行训练时可能出现的失配；与此同时，语义奖励改变了值得保留的内容，优先考虑保留意义和任务相关行为的方向，而不仅仅是匹配 token 概率。参数扩展随后提供了写入这些选定记忆所需的能力，而不会覆盖之前巩固的知识。
对于做梦，理论将随机专家激活和基于奖励的样本选择描述为改善学习几何结构的机制。注入不同的 MoE 扩大了合成生成过程中探索的梯度方向集合，使模型接触到比仅由其普通清醒时间数据引起的更丰富的更新集合。基于梯度的过滤然后强调携带最有用的学习信号的合成经验，加强信息丰富的方向，同时保留底层梯度空间的结构。
总之，这些步骤产生了一个更平衡的优化景观：学习不再由少数几个方向主导，梯度条件更好，模型在完善其能力时可以取得更稳定的进展。从这个角度来看，做梦不仅仅是自生成的数据增强；它是一个刻意过程，用于多样化、选择和巩固模型自我改进的更新。
我们的睡眠范式为实现自主、持续学习的 AI 系统提供了一条路径。通过超越静态预训练和脆弱的上下文窗口，在主动学习和离线休息之间交替，赋予 LLMs 保留短期记忆、将新技能抽象为长期参数结构，并持续自我改进而不发生灾难性遗忘的能力。
查看原文
Modern LLMs achieve impressive performance via pre-training and in-context learning (ICL). However, once deployed, they remain largely static, operating with a fixed knowledge cutoff date that leaves them unable to acquire long-term capabilities or adapt to new information.
While updating models with new data seems like the obvious solution, continual parameter updates often trigger catastrophic forgetting, a fundamental issue where learning new tasks severely degrades performance on previously learned ones. Alternatively, relying strictly on ICL limits knowledge retention to the immediate context window: once the context session ends, the acquired short-term information is forgotten.
Inspired by how the human brain processes and stabilizes memory during rest, we introduce a sleep framework in continual learning: a learning paradigm that equips large language models (LLMs) with a continuous periodic lifecycle, enabling them to continually learn, consolidate fragile short-term context into stable long-term parameters. With this framework, we demonstrate that alternating between active interaction and offline memory consolidation allows LLMs to continuously acquire new facts, retain long-context knowledge up to 10M tokens, and improve complex reasoning.
Rethinking machine learning: Active (wake) vs. sleep phases
Traditional machine learning relies on a strict separation between training and inference. In a continual learning framework, this distinction becomes artificial; an intelligent agent must constantly learn, adapt, and process information.
Under the sleep paradigm, an LLM operates across two primary states:
- Active (wake) phase: The model actively receives, processes, and interacts with inputs, storing temporary information in fast, short-term memory components.
- Sleep phase: The model pauses processing new inputs and redirects its internal computations toward memory consolidation and self-improvement.
While terms like "sleep," "dreaming," and "self-improvement" borrow from neuroscience, our framework operates entirely within a fixed model architecture and human-defined training rules. During the offline sleep phase, the model unlocks pre-allocated Mixture-of-Experts (MoE) memory blocks and updates its numerical weights using self-generated practice examples. This automated post-training cycle helps a deployed model absorb new facts and sharpen its reasoning accuracy while keeping its core codebase and design unchanged.
Key components of the sleep process
The sleep paradigm addresses catastrophic forgetting and continuous adaptation through two stages: memory consolidation and dreaming.
1. Memory consolidation
In the human brain, non-rapid eye movement (NREM) slow-wave sleep consolidates fragile memories into long-term structures via abstraction and distillation. Our proposed sleep framework mirrors this natural phenomenon through two key technical mechanisms:
- Parameter expansion & continuum memory hierarchy: To prevent catastrophic forgetting, the model dynamically scales its capacity during sleep. Using a Mixture of Experts (MoE) and Hope architecture , newly unlocked, low-rank experts are integrated into stable, lower-frequency memory blocks to store incoming knowledge without disrupting older parameters.
- Knowledge seeding (upward distillation): Fragile short-term memories stored in high-frequency layers (e.g., in-context memory) are transferred into the newly expanded long-term parameters. This is achieved through Generalized Knowledge Distillation (GKD) combined with reinforcement learning (RL)-based imitation learning:
- On-policy distillation: The expanded student model learns token-specific feedback from teacher distributions generated on self-sampled sequences.
- Learning to imitate (LTI): The student model is rewarded based on semantic and absolute similarity (Levenshtein distance) when completing prefixes of teacher-generated text.
- Synaptic pruning: Once the memory consolidation into lower-frequency modules is complete, the high-frequency temporary parameters are reset, freeing up capacity for future active sessions.
2. Dreaming for self-improvement
During REM sleep, the human brain actively simulates scenarios, consolidates emotional/semantic connections, and explores novel neural pathways. Mirroring this, during the “dreaming stage”, the sleep framework implements an autonomous self-improvement loop:
- Random expert selection: During synthetic data generation, the MoE router randomly activates non-standard experts. This forces the model to mix disparate domains of knowledge and uncover latent patterns.
- Curated rehearsal: The model generates synthetic training samples ("dreams") and filters them using gradient-based data selection to prioritize samples with the highest potential to boost overall model capability.
- RL refinement: The model performs supervised fine-tuning on top-rated dream samples and evaluates whether the generation improves performance on downstream tasks.
Theoretical foundations
Beyond the empirical results, our sleep paradigm is grounded in a theory that clarifies why its two stages (memory consolidation and dreaming) can support continual learning without simply trading new knowledge for old capabilities.
For memory consolidation, the analysis shows that knowledge seeding can be understood as a structured, low-rank compression problem. The model is not asked to copy every detail of its short-term traces into its parameters. Instead, it identifies a compact set of directions that best preserve both the model’s own on-policy behavior and the semantic qualities rewarded by the learning objective.
This perspective explains the complementary roles of the two signals: on-policy distillation aligns consolidation with the trajectories the model will actually produce at inference time, reducing the mismatch that can arise when training only on teacher-generated distributions; meanwhile, the semantic reward changes what is worth retaining, prioritizing directions that preserve meaning and task-relevant behavior rather than merely matching token probabilities. Parameter expansion then provides the capacity needed to write these selected memories without overwriting previously consolidated knowledge.
For dreaming, the theory characterizes random expert activation and reward-based sample selection as mechanisms for improving the geometry of learning. Injecting different MoEs broadens the set of gradient directions explored during synthetic generation, exposing the model to a richer collection of updates than those induced by its ordinary wake-time data alone. Gradient-based filtering then emphasizes the synthetic experiences that carry the most useful learning signal, strengthening informative directions while preserving the structure of the underlying gradient space.
Together, these steps yield a more balanced optimization landscape: learning is less dominated by a small number of directions, gradients are better conditioned, and the model can make more stable progress when refining its capabilities. In this view, dreaming is not simply self-generated data augmentation; it is a deliberate process for diversifying, selecting, and consolidating the updates through which the model improves itself.
For continual translation, in unseen languages, such as Manchu and Kalamang, the sleep framework maintained a near-zero performance drop in multi-stage evaluations, whereas standard baselines like ICL, Cartridges, and supervised fine-tuning (SFT) suffered severe catastrophic forgetting during sequential learning.
In long-context understanding on the BABILong benchmark, our sleep method demonstrated near-perfect scaling up to 10M tokens, outperforming strong baselines like GPT-4, and Titans by using sleep-time consolidation to compress long-context information directly into parametric memory.
For factual knowledge incorporation evaluated on SQuAD and continued pre-training (CPT), our sleep method achieved 48.9% on single-task and 46.2% on CPT evaluations, outperforming baseline methods including the base model, SFT, and Self-Adapting Language Models (SEAL) by successfully integrating new factual information without degrading existing model representations.
Finally, on mathematical reasoning benchmarks including AIME 24/25 and HMMT 25, sleep-enhanced base-model achieved accuracy scores of 79.2%, 69.0%, and 46.1%, respectively. This outperformed baselines like SFT, group relative policy optimization (GRPO), and on-policy self-distillation (OPSD), showing that pairing memory consolidation with imitation learning drives superior reasoning performance.
Conclusion
Our sleep paradigm provides a path toward autonomous, continually learning AI systems. By moving beyond static pre-training and fragile context windows, alternating between active learning and offline rest empowers LLMs to retain short-term memories, abstract new skills into long-term parameter structures, and continuously self-improve without catastrophic forgetting.
Acknowledgements
This research was conducted by Ali Behrouz (Google and Cornell University), Farnoosh Hashemi (Cornell University), Adel Javanmard (Google and University of Southern California), and Vahab Mirrokni. We thank Yuan Deng and Corinna Cortes for reviewing the work and their valuable suggestions. We also thank Mark Simborg, Kimberly Schwede, and Tim Herrmann for their help in crafting this blog post.
Studio