Groups
Category
Numerical stability measures how much rounding and tiny input changes can distort an algorithm’s output on real computers using floating-point arithmetic.
Momentum methods add an exponentially weighted memory of past gradients to make descent steps smoother and faster, especially in ravines and ill-conditioned problems.