How I Study AI - Learn AI Papers & Lectures the Easy Way

SLA2: Sparse-Linear Attention with Learnable Routing and QAT

Intermediate

Jintao Zhang, Haoxu Wang et al.Feb 13arXiv

SLA2 is a new way for AI to pay attention faster by smartly splitting work between two helpers: a precise one (sparse attention) and a speedy one (linear attention).

#Sparse Attention#Linear Attention#SLA2

Implicit Neural Representation Facilitates Unified Universal Vision Encoding

Intermediate

Matthew Gwilliam, Xiao Wang et al.Jan 20arXiv

This paper introduces HUVR, a single vision model that can both recognize what’s in an image and reconstruct or generate images from tiny codes.

#Implicit Neural Representation#Hyper-Networks#Vision Transformer

Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers

Beginner

Yifan Zhou, Zeqi Xiao et al.Dec 18arXiv

This paper introduces Log-linear Sparse Attention (LLSA), a new way for Diffusion Transformers to focus only on the most useful information using a smart, layered search.

#Log-linear Sparse Attention#Hierarchical Top-K#Hierarchical KV Enrichment

Papers3

SLA2: Sparse-Linear Attention with Learnable Routing and QAT

Implicit Neural Representation Facilitates Unified Universal Vision Encoding

Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers