MeKi: Memory-based Expert Knowledge Injection for Efficient LLM Scaling
IntermediateNing Ding, Fangcheng Liu et al.Feb 3arXiv
MeKi is a new way to grow a language model’s knowledge by using storage (ROM) instead of extra heavy calculations (FLOPs).
#MeKi#memory-based scaling#token-level experts