📚TheoryAdvanced
Feature Learning vs Kernel Regime
The kernel (lazy) regime keeps neural network parameters close to their initialization, making training equivalent to kernel regression with a fixed kernel such as the Neural Tangent Kernel (NTK).
#neural tangent kernel#kernel ridge regression#lazy training+12