Concepts2
📚TheoryIntermediate
Contrastive Learning Theory
Contrastive learning learns representations by pulling together positive pairs and pushing apart negatives using a softmax-based objective.
#contrastive learning#infonce#nt-xent+12
📚TheoryIntermediate
Attention Mechanism Theory
Attention computes a weighted sum of values V where the weights come from how similar queries Q are to keys K.
#attention#self-attention#multi-head attention+12