Concepts27
Category
PAC-Bayes Theory
PAC-Bayes provides high-probability generalization bounds for randomized predictors by comparing a data-dependent posterior Q to a fixed, data-independent prior P through KL(Q||P).
MCMC Theory
MCMC simulates a Markov chain whose long-run behavior matches a target distribution, letting us sample from complex posteriors without knowing the normalization constant.
Graph Neural Network Theory
Graph Neural Networks (GNNs) learn on graphs by repeatedly letting each node aggregate messages from its neighbors and update its representation.
Differential Privacy Theory
Differential privacy (DP) guarantees that the output of a randomized algorithm does not change much when one person’s data is added or removed.
Information-Theoretic Lower Bounds
Information-theoretic lower bounds tell you the best possible performance any learning algorithm can achieve, regardless of cleverness or compute.
Quantum Computing Theory
Quantum computing uses qubits that can be in superpositions, enabling interference-based computation beyond classical bits.
Streaming Algorithm Theory
Streaming algorithms process massive data one pass at a time using sublinear—often polylogarithmic—memory.
Distributed Algorithm Theory
Distributed algorithm theory studies how many independent computers cooperate correctly and efficiently despite delays and failures.
Algorithmic Information Theory
Algorithmic Information Theory studies information content via the shortest programs that generate data, rather than via average-case probabilities.
Optimal Transport Theory
Optimal Transport (OT) formalizes the cheapest way to move one probability distribution into another given a cost to move mass.
Diffusion Models Theory
Diffusion models learn to reverse a simple noising process by estimating the score (the gradient of the log density) of data at different noise levels.
Variational Inference Theory
Variational Inference (VI) replaces an intractable posterior with a simpler distribution and optimizes it by minimizing KL divergence, which is equivalent to maximizing the ELBO.