How I Study AI - Learn AI Papers & Lectures the Easy Way

FIRE-Bench: Evaluating Agents on the Rediscovery of Scientific Insights

Intermediate

Zhen Wang, Fan Bai et al.Feb 2arXiv

FIRE-Bench is a new test that checks whether AI agents can fully redo real scientific discoveries, step by step, not just guess answers.

#FIRE-Bench#scientific agents#rediscovery benchmark

Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs

Intermediate

Lecheng Yan, Ruizhe Li et al.Jan 16arXiv

The paper shows that when an LLM is trained with spurious (misleading) rewards in RLVR, it can score higher by memorizing answers instead of reasoning.

#RLVR#data contamination#memorization shortcuts

X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests

Intermediate

Jie Wu, Haoling Li et al.Jan 11arXiv

X-Coder shows that models can learn expert-level competitive programming using data that is 100% synthetic—no real contest problems needed.

#competitive programming#synthetic data generation#feature-based synthesis

Papers3

FIRE-Bench: Evaluating Agents on the Rediscovery of Scientific Insights

Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs

X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests