How I Study AI - Learn AI Papers & Lectures the Easy Way

On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking

Intermediate

Jianliang He, Leda Wang et al.Feb 18arXiv

This paper explains, in detail, how a simple two-layer neural network learns to add numbers on a clock (modular addition) by building and combining wave-like patterns called Fourier features.

#modular addition#Fourier features#discrete Fourier transform

Papers1

On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking