How I Study AI - Learn AI Papers & Lectures the Easy Way

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting

Intermediate

Muxi Diao, Lele Yang et al.Jan 5arXiv

Supervised fine-tuning (SFT) often makes a model great at a new task but worse at its old skills; this paper explains a key reason why and how to fix it.

#Entropy-Adaptive Fine-Tuning#confident conflicts#token-level entropy

Papers1

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting