Papers2

#sliding-window attention

SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer

Tongcheng Fang, Hanling Zhang et al.Jan 23arXiv

Videos are made of very long lists of tokens, and regular attention looks at every pair of tokens, which is slow and expensive.

#SALAD#sparse attention#linear attention

VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory

Beginner

Yifei Yu, Xiaoshan Wu et al.Dec 4arXiv

VideoSSM is a new way to make long, stable, and lively videos by giving the model two kinds of memory: a short-term window and a long-term state-space memory.

#autoregressive video diffusion#state-space model#hybrid memory