Papers6

#Sparse Attention

Hepato-LLaVA: An Expert MLLM with Sparse Topo-Pack Attention for Hepatocellular Pathology Analysis on Whole Slide Images

Yuxuan Yang, Zhonghao Yan et al.Feb 23arXiv

Hepato-LLaVA is a special AI that reads giant microscope pictures of the liver and answers medical questions about cancer.

#Hepato-LLaVA#Hepatocellular Carcinoma#Whole Slide Images

SLA2: Sparse-Linear Attention with Learnable Routing and QAT

Intermediate

Jintao Zhang, Haoxu Wang et al.Feb 13arXiv

SLA2 is a new way for AI to pay attention faster by smartly splitting work between two helpers: a precise one (sparse attention) and a speedy one (linear attention).

#Sparse Attention#Linear Attention#SLA2

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing

Intermediate

Yizhao Gao, Jianyu Wei et al.Feb 3arXiv

HySparse is a new way for AI models to pay attention that mixes a few full attention layers with many fast, memory‑saving sparse layers.

#Hybrid Sparse Attention#Oracle Token Selection#KV Cache Sharing

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection

Intermediate

Dongwon Jo, Beomseok Kang et al.Feb 3arXiv

This paper speeds up how AI models read very long texts by carefully choosing which words (tokens) to focus on at each step.

#Token Sparse Attention#Dynamic Token Coverage#Representation Drift

SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices

Intermediate

Dongting Hu, Aarush Gupta et al.Jan 13arXiv

This paper shows how to make powerful image‑generating Transformers run fast on phones without needing the cloud.

#Diffusion Transformer#Sparse Attention#Adaptive Sparse Self-Attention

SWAA: Sliding Window Attention Adaptation for Efficient Long-Context LLMs Without Pretraining

Intermediate

Yijiong Yu, Jiale Liu et al.Dec 11arXiv

Long texts make standard attention in large language models very slow because it checks every word against every other word.

#Sliding Window Attention#SWAA#FA Decode