Self-Distillation Enables Continual Learning
IntermediateIdan Shenfeld, Mehul Damani et al.Jan 27arXiv
This paper shows a simple way for AI models to keep learning new things without forgetting what they already know.
#Self-Distillation Fine-Tuning#On-Policy Distillation#Continual Learning