Mathematical foundations of generative models: VAEs, GANs, normalizing flows, diffusion models, and autoregressive methods.
11 concepts
Maximum Likelihood Estimation (MLE) picks parameters that make the observed data most probable under a chosen probabilistic model.
Wasserstein distance (Earth Mover’s Distance) measures how much “work” is needed to transform one probability distribution into another by moving mass with minimal total cost.
Autoregressive (AR) models represent a joint distribution by multiplying conditional probabilities in a fixed order, using the chain rule of probability.
Flow matching learns a time-dependent vector field v_t(x, c) whose ODE transports simple noise to complex data, enabling fast, deterministic sampling.
Classifier-Free Guidance (CFG) steers diffusion sampling toward a condition (like a text prompt) without needing a separate classifier.
A Variational Autoencoder (VAE) is a probabilistic autoencoder that learns to generate data by inferring hidden causes (latent variables) and decoding them back to observations.
GANs frame learning as a two-player game where a generator tries to fool a discriminator, and the discriminator tries to detect fakes.