Spectral distributions of random matrices and their surprising connections to neural network behavior.
5 concepts
The Wigner Semicircle Law says that the histogram of eigenvalues of large random symmetric matrices converges to a semicircle-shaped curve.
The Marchenko–Pastur (MP) distribution describes the limiting eigenvalue distribution of sample covariance matrices S = (1/n) XX^{\top} when both the dimension p and the sample size n grow with p/n \to \gamma.
Free probability studies "random variables" that do not commute, where independence is replaced by freeness and noncrossing combinatorics replaces classical partitions.
Spectral analysis studies the distribution of eigenvalues and singular values of neural network weight matrices during training.
Random Matrix Theory (RMT) explains how eigenvalues of large random matrices behave when the dimension p is comparable to the sample size n.