InfoNCE Induces Gaussian Distribution
IntermediateRoy Betser, Eyal Gofer et al.Feb 27arXiv
The paper shows that when we train with the popular InfoNCE contrastive loss, the learned features start to behave like they come from a Gaussian (bell-shaped) distribution.
#InfoNCE#contrastive learning#Gaussian embeddings