Concepts2
📚TheoryAdvanced
Information Bottleneck Theory
Information Bottleneck (IB) studies how to compress an input X into a representation Z that still preserves what is needed to predict Y.
#information bottleneck#mutual information#variational information bottleneck+12
📚TheoryAdvanced
Deep Learning Generalization Theory
Deep learning generalization theory tries to explain why overparameterized networks can fit (interpolate) training data yet still perform well on new data.
#generalization#implicit regularization#minimum norm+12