How I Study AI - Learn AI Papers & Lectures the Easy Way

On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models

Intermediate

Shumin Wang, Yuexiang Xie et al.Feb 3arXiv

The paper builds a simple, math-light rule to predict whether training makes a language model more open-minded (higher entropy) or more sure of itself (lower entropy).

#reinforcement fine-tuning#entropy dynamics#GRPO

EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

Intermediate

Taofeng Xue, Chong Peng et al.Jan 22arXiv

Before this work, computer-using AIs mostly copied old examples and struggled with long step-by-step tasks on real computers.

#computer use agent#verifiable synthesis#validator

Diversity or Precision? A Deep Dive into Next Token Prediction

Intermediate

Haoyuan Wu, Hai Wang et al.Dec 28arXiv

The paper shows that teaching a language model with a special “reward-shaped” next-token objective can make later reinforcement learning (RL) work much better.

#next-token prediction#cross-entropy as policy gradient#reward shaping

Papers3

On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models

EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

Diversity or Precision? A Deep Dive into Next Token Prediction