Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
BeginnerYu-Ang Lee, Ching-Yun Ko et al.Feb 4arXiv
When you tune the learning rate carefully, plain old LoRA fine-tuning works about as well as fancy new versions.
#LoRA#parameter-efficient fine-tuning#learning rate tuning