Derivatives, gradients, and the chain rule — the engine behind backpropagation and neural network training.
10 concepts
A limit describes what value a function approaches as the input gets close to some point, even if the function is not defined there.
Derivatives measure how fast a function changes, and rules like the product, quotient, and chain rule let us differentiate complex expressions efficiently.
Partial derivatives measure how a multivariable function changes when you wiggle just one input while keeping the others fixed.
The gradient \(\nabla f\) points in the direction of steepest increase of a scalar field and its length equals the maximum rate of increase.
The multivariable chain rule explains how rates of change pass through a pipeline of functions by multiplying the right derivatives (Jacobians) in the right order.
The Jacobian matrix collects all first-order partial derivatives of a vector-valued function, describing how small input changes linearly affect each output component.
The Hessian matrix collects all second-order partial derivatives of a scalar function and measures local curvature.
Taylor series approximate a complicated function near a point by a simple polynomial built from its derivatives.
Automatic differentiation (AD) computes exact derivatives by systematically applying the chain rule to your program, not by symbolic algebra or numerical differences.
Implicit differentiation lets you find slopes and higher derivatives even when y is given indirectly by an equation F(x,y)=0.