Groups
Category
Level
Softmax turns arbitrary real-valued scores (logits) into probabilities that sum to one.
Sinusoidal positional encoding represents each token’s position using pairs of sine and cosine waves at exponentially spaced frequencies.