Concepts5
Diffusion Models Theory
Diffusion models learn to reverse a simple noising process by estimating the score (the gradient of the log density) of data at different noise levels.
Variational Inference Theory
Variational Inference (VI) replaces an intractable posterior with a simpler distribution and optimizes it by minimizing KL divergence, which is equivalent to maximizing the ELBO.
ELBO (Evidence Lower Bound)
The Evidence Lower Bound (ELBO) is a tractable lower bound on the log evidence log p(x) that enables learning and inference in latent variable models like VAEs.
KL Divergence (Kullback-Leibler Divergence)
Kullback–Leibler (KL) divergence measures how one probability distribution P devotes probability mass differently from a reference distribution Q.
Information Theory
Information theory quantifies uncertainty and information using measures like entropy, cross-entropy, KL divergence, and mutual information.