🎓How I Study AIHISA

📖Read

📄Papers 📰Blogs 🎬Courses

💡Learn

🛤️Paths 📚Topics 💡Concepts 🎴Shorts

🎯Practice

🧩Problems 🎯Prompts 🧠Review

How I Study AI - Learn AI Papers & Lectures the Easy Way

Papers2

All Beginner Intermediate Advanced

All Sources arXiv

#tensor cores

DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

Ionut-Vlad Modoranu, Philip Zmushko et al.Feb 2arXiv

Shampoo is a smart optimizer that can train models better than AdamW, but it used to be slow because it must compute tricky inverse matrix roots.

#Shampoo optimizer#second-order optimization#inverse matrix roots

Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation

Andrei Panferov, Erik Schultheis et al.Jan 30arXiv

This paper shows how to train big language models faster and cheaper by using 4-bit numbers (NVFP4) without losing much accuracy.

#NVFP4#FP4 training#quantization-aware training