Papers6

#mechanistic interpretability

Sparse Reward Subsystem in Large Language Models

Guowei Xu, Mert Yuksekgonul et al.Feb 1arXiv

The paper discovers a tiny, special group of neurons inside large language models (LLMs) that act like a reward system in the human brain.

#value neurons#dopamine neurons#reward prediction error

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models

Intermediate

Hengyuan Zhang, Zhihao Zhang et al.Jan 20arXiv

This survey turns model understanding into a step-by-step repair toolkit called Locate, Steer, and Improve.

#mechanistic interpretability#residual stream#attention heads

Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs

Intermediate

Lecheng Yan, Ruizhe Li et al.Jan 16arXiv

The paper shows that when an LLM is trained with spurious (misleading) rewards in RLVR, it can score higher by memorizing answers instead of reasoning.

#RLVR#data contamination#memorization shortcuts

Reasoning Models Generate Societies of Thought

Intermediate

Junsol Kim, Shiyang Lai et al.Jan 15arXiv

The paper shows that top reasoning AIs don’t just think longer—they act like a tiny team inside their heads, with different voices that ask, disagree, and then agree.

#society of thought#reasoning reinforcement learning#conversational behaviors

Mechanistic Interpretability of Large-Scale Counting in LLMs through a System-2 Strategy

Intermediate

Hosein Hasani, Mohammadali Banayeeanzade et al.Jan 6arXiv

Large language models (LLMs) are good at many math problems but often mess up simple counting when the list gets long.

#mechanistic interpretability#counting in LLMs#System-2 prompting

Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process

Beginner

Zhenyu Zhang, Shujian Zhang et al.Dec 30arXiv

This paper shows a new way (called RISE) to find and control how AI models think without needing any human-made labels.

#RISE#sparse auto-encoder#reasoning vectors