An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift
IntermediateConstantinos Karouzos, Xingwei Tan et al.Jan 9arXiv
Preference tuning teaches language models to act the way people like, but those habits can fall apart when the topic or style changes (domain shift).
#preference tuning#domain shift#supervised fine-tuning