Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in $\{\pm 1, \pm i\}$
IntermediateFeiyu Wang, Xinyu Tan et al.Dec 2arXiv
Fairy2i turns any pre-trained real-valued Transformer layer into an exactly equivalent complex form, so nothing changes before quantization.
#LLM quantization#complex-valued neural networks#widely-linear transformation