Concepts4
📚TheoryAdvanced
PAC-Bayes Theory
PAC-Bayes provides high-probability generalization bounds for randomized predictors by comparing a data-dependent posterior Q to a fixed, data-independent prior P through KL(Q||P).
#pac-bayes#generalization bound#kl divergence+12
📚TheoryAdvanced
Statistical Learning Theory
Statistical learning theory explains why a model that fits training data can still predict well on unseen data by relating true risk to empirical risk plus a complexity term.
#statistical learning theory#empirical risk minimization#structural risk minimization+11
📚TheoryIntermediate
PAC Learning
PAC learning formalizes when a learner can probably (with probability at least 1−δ) and approximately (error at most ε) succeed using a polynomial number of samples.
#pac learning#agnostic learning#vc dimension+12
📚TheoryAdvanced
Rademacher Complexity
Rademacher complexity is a data-dependent measure of how well a function class can fit random noise on a given sample.
#rademacher complexity#empirical rademacher#generalization bounds+12