Papers9

#world modeling

Beyond Language Modeling: An Exploration of Multimodal Pretraining

Shengbang Tong, David Fan et al.Mar 3arXiv

The paper trains one model from scratch to both read text and see images/videos, instead of starting from a language-only model.

#multimodal pretraining#representation autoencoder#RAE

World Guidance: World Modeling in Condition Space for Action Generation

Intermediate

Yue Su, Sijin Chen et al.Feb 25arXiv

WoG (World Guidance) teaches a robot to imagine just the right bits of the near future and use those bits to pick better actions.

#Vision-Language-Action#world modeling#condition space

VLANeXt: Recipes for Building Strong VLA Models

Intermediate

Xiao-Ming Wu, Bin Fan et al.Feb 20arXiv

This paper studies Vision–Language–Action (VLA) robots under one fair setup to find which design choices truly matter.

#Vision-Language-Action#robot manipulation#flow matching

FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment

Intermediate

Han Zhao, Jingbo Wang et al.Feb 19arXiv

Robots learn better when they predict short, meaningful summaries of future images instead of drawing every pixel of the future scene.

#world modeling#vision-language-action (VLA)#diffusion policy

Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents

Intermediate

Haiyang Xu, Xi Zhang et al.Feb 15arXiv

This paper builds GUI-Owl-1.5, an AI that can use phones, computers, and web browsers like a careful human helper.

#GUI agent#visual grounding#reinforcement learning

Reinforcement World Model Learning for LLM-based Agents

Intermediate

Xiao Yu, Baolin Peng et al.Feb 5arXiv

Large language models are great at words, but they struggle to predict what will happen after they act in a changing world.

#Reinforcement World Model Learning#world modeling#LLM agents

Action100M: A Large-scale Video Action Dataset

Intermediate

Delong Chen, Tejaswi Kasarla et al.Jan 15arXiv

Action100M is a gigantic video dataset with about 100 million labeled action moments built automatically from 1.2 million instructional videos.

#Action100M#open-vocabulary action recognition#hierarchical temporal segmentation

Web World Models

Intermediate

Jichen Feng, Yifan Zhang et al.Dec 29arXiv

This paper introduces Web World Models (WWMs), a way to build huge, explorable worlds by putting strict rules in code and letting AI write the fun details.

#Web World Model#typed interfaces#deterministic hashing

MMGR: Multi-Modal Generative Reasoning

Intermediate

Zefan Cai, Haoyi Qiu et al.Dec 16arXiv

MMGR is a new benchmark that checks whether AI image and video generators follow real-world rules, not just whether their outputs look pretty.

#multi-modal generative reasoning#video generation evaluation#physical commonsense