Get Started
Home
Topics
Search
Library
Research questionHow can on-policy distillation remain stable when policy updates change future training states?On-policy distillation continually changes the policy that generates its own training states. This feedback can make later updates operate on increasingly different trajectories, causing entropy growth or degraded performance.
AI
Code Generation & Program Synthesis
LLM Pretraining & Post-training
Machine Learning
Reasoning
Reinforcement Learning
Latest papersRecent research connected to this question, newest first.WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-TrainingApplies to language-model on-policy distillation using student-generated rollouts and a frozen teacher. Evidence comes from recorded Qwen3 experiments at 1.7B and 4B scales in mathematical reasoning and code generation; because some comparisons vary in curriculum or initialization, the results support a stabilization hypothesis rather than a universal causal claim.research paper · Sep 3, 2026
Related questions
How can on-policy distillation prevent early student errors from corrupting rollouts in autoregressive vision-language models?How can autoregressive logit distillation preserve local next-token preferences when teacher and student rankings disagree?How can model distillation block hidden teacher-trait transfer through clean data without degrading the target task?How should on-policy distillation select examples for data-efficient reasoning post-training?