Concepts280
Category
P vs NP Problem
P vs NP asks whether every problem whose solutions can be verified quickly can also be solved quickly.
GAN Theory
Generative Adversarial Networks (GANs) set up a two-player game where a generator tries to make fake samples that look real while a discriminator tries to tell real from fake.
Complexity Theory
Complexity theory classifies problems by the resources required to solve or verify them, such as time and memory.
Representation Learning Theory
Representation learning aims to automatically discover features that make downstream tasks easy, often without human-provided labels.
Information Bottleneck Theory
Information Bottleneck (IB) studies how to compress an input X into a representation Z that still preserves what is needed to predict Y.
Contrastive Learning Theory
Contrastive learning learns representations by pulling together positive pairs and pushing apart negatives using a softmax-based objective.
Policy Gradient Theorem
The policy gradient theorem tells us how to push a stochastic policy’s parameters to increase expected return by following the gradient of expected rewards.
Bellman Equations
Bellman equations express how the value of a state or action equals immediate reward plus discounted value of what follows.
Transformer Theory
Transformers map sequences to sequences using layers of self-attention and feed-forward networks wrapped with residual connections and LayerNorm.
Reinforcement Learning Theory
Reinforcement Learning (RL) studies how an agent learns to act in an environment to maximize long-term cumulative reward.
Neural Tangent Kernel (NTK) Theory
The Neural Tangent Kernel (NTK) connects very wide neural networks to classical kernel methods, letting us study training as if it were kernel regression.
Attention Mechanism Theory
Attention computes a weighted sum of values V where the weights come from how similar queries Q are to keys K.