Concepts3

Groups

In-Context Learning Theory

In-context learning (ICL) means a model learns from examples provided in the input itself, without updating its parameters.

#in-context learning#transformer#attention+12

📚TheoryIntermediate

Implicit Bias of Gradient Descent

In underdetermined linear systems (more variables than equations), gradient descent started at zero converges to the minimum Euclidean norm solution without any explicit regularizer.

#implicit bias

Concepts3

In-Context Learning Theory

Implicit Bias of Gradient Descent

Double Descent Phenomenon