COMPOT: Calibration-Optimized Matrix Procrustes Orthogonalization for Transformers Compression
IntermediateDenis Makhov, Dmitriy Shopkhoev et al.Feb 16arXiv
COMPOT is a training-free way to shrink Transformer models while keeping their smarts.
#Transformer compression#orthogonal dictionary learning#orthogonal Procrustes