Fast-weight Product Key Memory
IntermediateTianyu Zhao, Llion JonesJan 2arXiv
The paper introduces Fast-weight Product Key Memory (FwPKM), a memory layer that can quickly learn from the current text it reads, not just from past training.
#Fast-weight memory#Product Key Memory#Sparse retrieval