K-EXAONE Technical Report
IntermediateEunbi Choi, Kibong Choi et al.Jan 5arXiv
K-EXAONE is a super-sized language model that speaks six languages and can read very long documents (up to 256,000 tokens) without forgetting important details.
#Mixture-of-Experts#Hybrid Attention#Sliding Window Attention