ECO: Quantized Training without Full-Precision Master Weights
IntermediateMahdi Nikdan, Amir Zandieh et al.Jan 29arXiv
Training big AI models uses lots of memory because most methods still keep a secret full-precision copy of the weights called master weights.
#quantized training#master weights#error feedback