Concepts3
📚TheoryAdvanced
PAC-Bayes Theory
PAC-Bayes provides high-probability generalization bounds for randomized predictors by comparing a data-dependent posterior Q to a fixed, data-independent prior P through KL(Q||P).
#pac-bayes#generalization bound#kl divergence+12
📚TheoryAdvanced
Statistical Learning Theory
Statistical learning theory explains why a model that fits training data can still predict well on unseen data by relating true risk to empirical risk plus a complexity term.
#statistical learning theory#empirical risk minimization#structural risk minimization+11
📚TheoryAdvanced
Rademacher Complexity
Rademacher complexity is a data-dependent measure of how well a function class can fit random noise on a given sample.
#rademacher complexity#empirical rademacher#generalization bounds+12