Reinforced Fast Weights with Next-Sequence Prediction
IntermediateHee Seung Hwang, Xindi Wu et al.Feb 18arXiv
Fast weight models remember context with a tiny, fixed memory, but standard next-token training teaches them to think only one word ahead.
#fast weight models#next-sequence prediction#reinforcement learning for LMs