Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models
IntermediateHyunjong Ok, Jaeho LeeJan 20arXiv
Putting the reading passage before the question and answer choices (CQO) makes language models much more accurate than putting it after (QOC), by about 15 percentage points on average.
#causal attention#prompt order sensitivity#multiple-choice question answering