Smooth manifolds, Riemannian metrics, and natural gradients for geometry-aware deep learning.
6 concepts
A smooth manifold is a space that looks like ordinary Euclidean space when you zoom in, glued together using charts that transition smoothly.
Geodesics are the “straightest possible” paths on curved spaces (manifolds) and locally minimize distance.
Curvature measures how a geometric object bends, and it comes in several flavors: Gaussian, sectional, and Ricci curvature.
Lie groups model continuous symmetries like rotations and rigid-body motions, combining algebra (group law) and calculus (smooth manifolds).
Natural gradient scales the ordinary gradient by the inverse Fisher information matrix to account for the geometry of probability distributions.