Concepts3
📚TheoryAdvanced
Statistical Learning Theory
Statistical learning theory explains why a model that fits training data can still predict well on unseen data by relating true risk to empirical risk plus a complexity term.
#statistical learning theory#empirical risk minimization#structural risk minimization+11
📚TheoryIntermediate
Universal Approximation Theorem
The Universal Approximation Theorem (UAT) says a feedforward neural network with one hidden layer and a non-polynomial activation (like sigmoid or ReLU) can approximate any continuous function on a compact set as closely as we want.
#universal approximation theorem#cybenko#hornik+12
📚TheoryIntermediate
Bias-Variance Tradeoff
The bias–variance tradeoff explains how prediction error splits into bias squared, variance, and irreducible noise for squared loss.
#bias variance tradeoff#mse decomposition#polynomial regression+12