about
WorldVLA: Towards Autoregressive Action World Model (arxiv.org)
25 points by chrsw on Jun 29, 2025 | hide | past | pdf | 2 comments on HN

In plain words: One system guesses the next image from what it sees and does, and picks an action from the image, so each task teaches the other. It beat action-only and image-prediction-only systems, and hiding earlier actions when writing each new one stopped errors piling up.

Abstract

We present WorldVLA, an autoregressive action world model that unifies action and image understanding and generation. Our WorldVLA intergrates Vision-Language-Action (VLA) model and world model in one single framework. The world model predicts future images by leveraging both action and image understanding, with the purpose of learning the underlying physics of the environment to improve action generation. Meanwhile, the action model generates the subsequent actions based on image observations, aiding in visual understanding and in turn helps visual generation of the world model. We demonstrate that WorldVLA outperforms standalone action and world models, highlighting the mutual enhancement between the world model and the action model. In addition, we find that the performance of the action model deteriorates when generating sequences of actions in an autoregressive manner. This phenomenon can be attributed to the model's limited generalization capability for action prediction, leading to the propagation of errors from earlier actions to subsequent ones. To address this issue, we propose an attention mask strategy that selectively masks prior actions during the generation of the current action, which shows significant performance improvement in the action chunk generation task.

Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, Deli Zhao, Hao Chen
arXiv:2506.21539 · cs.RO, cs.AI · submitted Jun 26, 2025
abstract · pdf · html · Code: https://github.com/alibaba-damo-academy/WorldVLA

add comment on HN

More details on the actual arch would be nice. It seems like this autoregressive action world model space is a local max until the JEPA work takes over.
Dont see how we could not hack on jepa instead of any other input encoders on a transformers llm .