💬LLM & GenAI
🎛️
Fine-tuning LLMs
Learn to customize LLMs for specific tasks through fine-tuning and adaptation
Prerequisites
🌱
Beginner
BeginnerIntroduction to LLM fine-tuning
What to Learn
- •When to fine-tune vs prompt
- •Dataset preparation and formatting
- •Supervised fine-tuning basics
- •Using Hugging Face Transformers
- •Evaluation of fine-tuned models
Resources
- 📚Hugging Face fine-tuning tutorials
- 📚OpenAI fine-tuning guide
- 📚Alpaca and Vicuna case studies
🌿
Intermediate
IntermediateEfficient fine-tuning methods
What to Learn
- •LoRA (Low-Rank Adaptation)
- •QLoRA for consumer hardware
- •Prefix tuning and prompt tuning
- •Training data quality and curation
- •Distributed fine-tuning setup
Resources
- 📚LoRA paper
- 📚QLoRA paper and tutorials
- 📚PEFT library documentation
🌳
Advanced
AdvancedAdvanced adaptation techniques
What to Learn
- •RLHF (Reinforcement Learning from Human Feedback)
- •DPO (Direct Preference Optimization)
- •Constitutional AI and RLAIF
- •Continual learning for LLMs
- •Merging and model soup techniques
Resources
- 📚InstructGPT and RLHF papers
- 📚DPO paper
- 📚Model merging research