Mechanistic Interpretability of Large-Scale Counting in LLMs through a System-2 Strategy
IntermediateHosein Hasani, Mohammadali Banayeeanzade et al.Jan 6arXiv
Large language models (LLMs) are good at many math problems but often mess up simple counting when the list gets long.
#mechanistic interpretability#counting in LLMs#System-2 prompting