Papers2

#rate–distortion

An Information Theoretic Perspective on Agentic System Design

Shizhe He, Avanika Narayan et al.Dec 25arXiv

The paper shows that many AI systems work best when a small 'compressor' model first shrinks long text into a short, info-packed summary and a bigger 'predictor' model then reasons over that summary.

#agentic systems#compressor-predictor#mutual information

Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning

Intermediate

Seijin Kobayashi, Yanick Schimpf et al.Dec 23arXiv

The paper shows that big sequence models (like transformers) quietly learn longer goals inside their hidden activations, even though they are trained one step at a time.

#hierarchical reinforcement learning#temporal abstractions#autoregressive models