How I Study AI - Learn AI Papers & Lectures the Easy Way

mHC: Manifold-Constrained Hyper-Connections

Intermediate

Zhenda Xie, Yixuan Wei et al.Dec 31arXiv

The paper fixes a stability problem in Hyper-Connections (HC) by gently steering the network’s mixing matrix onto a safe shape (a manifold) where signals don’t blow up or vanish.

#Residual Connections#Hyper-Connections#Manifold Projection

Papers1

mHC: Manifold-Constrained Hyper-Connections