State over Tokens: Characterizing the Role of Reasoning Tokens
IntermediateMosh Levy, Zohar Elyoseph et al.Dec 14arXiv
Reasoning tokens (the words a model writes before its final answer) help the model think better, but they are not a trustworthy diary of how it really thought.
#State over Tokens#reasoning tokens#chain-of-thought