Slides
-
Paper walkthrough · 21 slides
Self-Distillation Enables Continual Learning (SDFT)
How a model can teach itself from demonstrations on-policy, why it forgets less than SFT, and where the theory and the training loss part ways.
Paper walkthrough · 21 slides
How a model can teach itself from demonstrations on-policy, why it forgets less than SFT, and where the theory and the training loss part ways.