Concepts105
Category
P vs NP Problem
P vs NP asks whether every problem whose solutions can be verified quickly can also be solved quickly.
GAN Theory
Generative Adversarial Networks (GANs) set up a two-player game where a generator tries to make fake samples that look real while a discriminator tries to tell real from fake.
Representation Learning Theory
Representation learning aims to automatically discover features that make downstream tasks easy, often without human-provided labels.
Information Bottleneck Theory
Information Bottleneck (IB) studies how to compress an input X into a representation Z that still preserves what is needed to predict Y.
Policy Gradient Theorem
The policy gradient theorem tells us how to push a stochastic policy’s parameters to increase expected return by following the gradient of expected rewards.
Transformer Theory
Transformers map sequences to sequences using layers of self-attention and feed-forward networks wrapped with residual connections and LayerNorm.
Reinforcement Learning Theory
Reinforcement Learning (RL) studies how an agent learns to act in an environment to maximize long-term cumulative reward.
Neural Tangent Kernel (NTK) Theory
The Neural Tangent Kernel (NTK) connects very wide neural networks to classical kernel methods, letting us study training as if it were kernel regression.
Calculus of Variations
Calculus of variations optimizes functionals—numbers produced by whole functions—rather than ordinary functions of numbers.
Deep Learning Generalization Theory
Deep learning generalization theory tries to explain why overparameterized networks can fit (interpolate) training data yet still perform well on new data.
Neural Network Expressivity
Neural network expressivity studies what kinds of functions different network architectures can represent and how efficiently they can do so.
Statistical Learning Theory
Statistical learning theory explains why a model that fits training data can still predict well on unseen data by relating true risk to empirical risk plus a complexity term.