Groups
Low-rank approximation replaces a big matrix with one that has far fewer degrees of freedom while preserving most of its action.
Eigendecomposition expresses a matrix as a change of basis times a diagonal scaling, revealing its natural stretching directions.