Variational inference, evidence lower bound, and optimization-based approaches to approximate intractable distributions.
8 concepts
Variational Inference (VI) turns Bayesian inference into an optimization problem by choosing a simple family q(z) to approximate an intractable posterior p(z|x).
Mean field variational family assumes the joint posterior over latent variables factorizes into independent pieces q(z) = ∏ q_i(z_i).
Expectation Maximization (EM) is an iterative algorithm to estimate parameters when some variables are hidden or unobserved.
The Evidence Lower Bound (ELBO) is a tractable lower bound on the log evidence log p(x) used to perform approximate Bayesian inference.
Stochastic Variational Inference (SVI) scales variational inference to large datasets by taking noisy but unbiased gradient steps using minibatches.