Concepts3
📚TheoryAdvanced
Representation Learning Theory
Representation learning aims to automatically discover features that make downstream tasks easy, often without human-provided labels.
#representation learning#contrastive learning#infonce+12
📚TheoryIntermediate
Contrastive Learning Theory
Contrastive learning learns representations by pulling together positive pairs and pushing apart negatives using a softmax-based objective.
#contrastive learning#infonce#nt-xent+12
📚TheoryIntermediate
Mutual Information
Mutual Information (MI) measures how much knowing one random variable reduces uncertainty about another.
#mutual information#entropy#kl divergence+12