How I Study AI - Learn AI Papers & Lectures the Easy Way

Approximation of Log-Partition Function in Policy Mirror Descent Induces Implicit Regularization for LLM Post-Training

Intermediate

Zhenghao Xu, Qin Lu et al.Feb 5arXiv

The paper studies a simple way to train giant language models with reinforcement learning by replacing a hard-to-compute term (the log-partition function) with something easy: the mean reward.

#Policy Mirror Descent#KL regularization#chi-squared regularization

EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

Intermediate

Taofeng Xue, Chong Peng et al.Jan 22arXiv

Before this work, computer-using AIs mostly copied old examples and struggled with long step-by-step tasks on real computers.

#computer use agent#verifiable synthesis#validator

WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks

Intermediate

Hao Bai, Alexey Taymanov et al.Jan 5arXiv

WebGym is a giant practice world (almost 300,000 tasks) that lets AI web agents learn on real, ever-changing websites instead of tiny, fake ones.

#WebGym#visual web agents#vision-language models

Papers3

Approximation of Log-Partition Function in Policy Mirror Descent Induces Implicit Regularization for LLM Post-Training

EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks