Groups
Category
Group theory gives a precise language for symmetries, and neural networks can exploit these symmetries to learn faster and generalize better.
The Convolution Theorem says that convolving two signals in time (or space) equals multiplying their spectra in the frequency domain.