Concepts174
Category
Singular Value Decomposition (SVD)
Singular Value Decomposition (SVD) factors any mΓn matrix A into A = UΞ£V^{T}, where U and V are orthogonal and Ξ£ is diagonal with nonnegative entries.
Convex Optimization
Convex optimization studies minimizing convex functions over convex sets, where every local minimum is guaranteed to be a global minimum.
Eigenvalue Decomposition
Eigenvalue decomposition rewrites a square matrix as a change of basis that reveals how it stretches and rotates space.
Optimization Theory
Optimization theory studies how to choose variables to minimize or maximize an objective while respecting constraints.
Linear Algebra Theory
Linear algebra studies vectors, linear combinations, and transformations that preserve addition and scalar multiplication.
Gradient Descent Convergence Theory
Gradient descent updates parameters by stepping opposite the gradient: x_{t+1} = x_t - \eta \nabla f(x_t).
Central Limit Theorem
The Central Limit Theorem (CLT) says that the sum or average of many independent, identically distributed variables with finite variance becomes approximately normal (Gaussian).
Probability Distributions
Probability distributions describe how random outcomes are spread across possible values and let us compute probabilities, expectations, and uncertainties.
Probability Theory
Probability theory formalizes uncertainty using a sample space, events, and a probability measure that obeys clear axioms.
Mutual Information
Mutual Information (MI) measures how much knowing one random variable reduces uncertainty about another.
KL Divergence (Kullback-Leibler Divergence)
KullbackβLeibler (KL) divergence measures how one probability distribution P devotes probability mass differently from a reference distribution Q.
Shannon Entropy
Shannon entropy quantifies the average uncertainty or information content of a random variable in bits when using base-2 logarithms.