How I Study AI - Learn AI Papers & Lectures the Easy Way

Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

Intermediate

Ailin Huang, Ang Li et al.Feb 11arXiv

Step 3.5 Flash is a huge but efficient AI that keeps 196 billion total parameters but only wakes up about 11 billion per token, so it thinks smart and fast.

#Sparse Mixture-of-Experts#Sliding-Window Attention#Head-wise Gated Attention

Papers1

Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters