Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
AdvancedAndrei Panferov, Erik Schultheis et al.Jan 30arXiv
This paper shows how to train big language models faster and cheaper by using 4-bit numbers (NVFP4) without losing much accuracy.
#NVFP4#FP4 training#quantization-aware training