CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
BeginnerHaoran Li, Sucheng Ren et al.Feb 5arXiv
The paper introduces CoPE, a simple change to how models track word positions that makes long documents much easier for them to understand.
#CoPE#RoPE#Rotary Positional Embedding