šŸŽ“How I Study AIHISA
šŸ“–Read
šŸ“„PapersšŸ“°BlogsšŸŽ¬Courses
šŸ’”Learn
šŸ›¤ļøPathsšŸ“šTopicsšŸ’”ConceptsšŸŽ“Shorts
šŸŽÆPractice
šŸ“Daily LogšŸŽÆPromptsšŸ§ Review
SearchSettings
How I Study AI - Learn AI Papers & Lectures the Easy Way

Concepts9

Groups

šŸ“Linear Algebra15šŸ“ˆCalculus & Differentiation10šŸŽÆOptimization14šŸŽ²Probability Theory12šŸ“ŠStatistics for ML9šŸ“”Information Theory10šŸ”ŗConvex Optimization7šŸ”¢Numerical Methods6šŸ•øGraph Theory for Deep Learning6šŸ”µTopology for ML5šŸŒDifferential Geometry6āˆžMeasure Theory & Functional Analysis6šŸŽ°Random Matrix Theory5šŸŒŠFourier Analysis & Signal Processing9šŸŽ°Sampling & Monte Carlo Methods10šŸ§ Deep Learning Theory12šŸ›”ļøRegularization Theory11šŸ‘ļøAttention & Transformer Theory10šŸŽØGenerative Model Theory11šŸ”®Representation Learning10šŸŽ®Reinforcement Learning Mathematics9šŸ”„Variational Methods8šŸ“‰Loss Functions & Objectives10ā±ļøSequence & Temporal Models8šŸ’ŽGeometric Deep Learning8

Category

šŸ”·Allāˆ‘Mathāš™ļøAlgošŸ—‚ļøDSšŸ“šTheory

Level

AllBeginnerIntermediate
āš™ļøAlgorithmIntermediate

Lion Optimizer

Lion (Evolved Sign Momentum) is a first-order, sign-based optimizer discovered through automated program search.

#lion optimizer#sign-based optimization#momentum+12
šŸ“šTheoryIntermediate

LSTM & Gating Mechanisms

Long Short-Term Memory (LSTM) networks use gates (forget, input, and output) to control what information to erase, write, and reveal at each time step.

#lstm
Advanced
Filtering by:
#c++ implementation
#forget gate
#input gate
+11
āš™ļøAlgorithmIntermediate

Efficient Attention Mechanisms

Standard softmax attention costs O(nĀ²) in sequence length because every token compares with every other token.

#linear attention#efficient attention#kernel trick+12
šŸ“šTheoryAdvanced

Neural Tangent Kernel (NTK)

Neural Tangent Kernel (NTK) describes how wide neural networks train like kernel machines, turning gradient descent into kernel regression in the infinite-width limit.

#neural tangent kernel#ntk#nngp+12
āš™ļøAlgorithmIntermediate

Importance Sampling

Importance sampling rewrites an expectation under a hard-to-sample distribution p as an expectation under an easier distribution q, multiplied by a weight w = p/q.

#importance sampling#proposal distribution#self-normalized+12
āš™ļøAlgorithmIntermediate

Numerical Differentiation & Finite Differences

Numerical differentiation uses finite differences to estimate derivatives when an analytical derivative is hard or impossible to obtain.

#numerical differentiation#finite differences#forward difference+12
šŸ“šTheoryIntermediate

Online Algorithm Theory

Online algorithms make decisions step by step without seeing the future and are judged against an all-knowing offline optimum.

#online algorithms#competitive analysis#ski rental+12
šŸ“šTheoryAdvanced

Neural Network Expressivity

Neural network expressivity studies what kinds of functions different network architectures can represent and how efficiently they can do so.

#neural network expressivity#depth separation#relu linear regions+12
šŸ—‚ļøData StructureIntermediate

Trie (Prefix Tree)

A trie (prefix tree) stores strings or bit-sequences so that common prefixes share nodes, making operations depend on the key length L rather than the set size.

#trie#prefix tree#autocomplete+12