Groups
Category
Momentum methods add an exponentially weighted memory of past gradients to make descent steps smoother and faster, especially in ravines and ill-conditioned problems.
Optimization theory studies how to choose variables to minimize or maximize an objective while respecting constraints.