How I Study AI - Learn AI Papers & Lectures the Easy Way

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning

Beginner

Yu-Ang Lee, Ching-Yun Ko et al.Feb 4arXiv

When you tune the learning rate carefully, plain old LoRA fine-tuning works about as well as fancy new versions.

#LoRA#parameter-efficient fine-tuning#learning rate tuning

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection

Beginner

Tao Liu, Taiqiang Wu et al.Jan 14arXiv

Traditional supervised fine-tuning (SFT) makes a model copy one answer too exactly, which can cause overfitting to the exact wording instead of the real idea.

#ProFit#Supervised Fine-Tuning#Token Probability

C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling

Beginner

Jin Qin, Zihan Liao et al.Dec 24arXiv

C2LLM is a new family of code embedding models that helps computers find the right code faster and more accurately.

#code retrieval#embedding model#cross-attention pooling

Papers3

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection

C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling