Get Started
Home
Topics
Search
Library
Research questionHow should on-policy distillation select examples for data-efficient reasoning post-training?On-policy distillation can use many training examples, but it remains unclear which examples provide the most useful learning signal. Example difficulty, reasoning-trace length, and token-level uncertainty may not contribute equally to improving reasoning ability.
AI
LLM Pretraining & Post-training
Machine Learning
Natural Language Processing
Reasoning
Latest papersRecent research connected to this question, newest first.What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data SelectionThe evidence comes from experiments on four models ranging from 1.5B to 7B. The study examines one-example training, relates gains more strongly to longer chain-of-thought paths than to token entropy, and reports that eight selected hard examples matched a 17K-example baseline; examples beyond the teacher’s capability were also usable in these experiments.research paper · Sep 4, 2026Rethinking On-Policy Distillation of Large Language Models II: One Training ExampleThe evidence comes from language-model on-policy distillation experiments across task domains and model families. One-query training continues improving for hundreds of steps; 16 semantically distinct queries reach 98.9% state coverage and match full-data training, with similar results reported for multi-teacher distillation. Content-light templates and off-domain WildChat queries also approach the real-query baseline. These results do not establish a universal query count or optimization budget beyond the studied settings.research paper · Sep 3, 2026
Related questions
Should reasoning LLMs combine on-policy distillation and verifiable-reward RL jointly or sequentially during post-training?How can in-context learning select demonstrations by flexible reasoning logic rather than surface similarity?How can on-policy distillation prevent early student errors from corrupting rollouts in autoregressive vision-language models?How can contrastive distillation efficiently transfer representations to smaller students without memory banks or fixed temperatures?