Papers2

#LLM reliability

Spilled Energy in Large Language Models

Adrian Robert Minut, Hazem Dewidar et al.Feb 21arXiv

The paper treats the last layer of a Large Language Model (the softmax over tokens) as an Energy-Based Model, which lets us measure a new signal called spilled energy.

#spilled energy#energy-based models#marginal energy

Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models

Intermediate

Hyunjong Ok, Jaeho LeeJan 20arXiv

Putting the reading passage before the question and answer choices (CQO) makes language models much more accurate than putting it after (QOC), by about 15 percentage points on average.

#causal attention#prompt order sensitivity#multiple-choice question answering