In plain words: Physics games where an agent must push or move objects at just the right moment, since each action changes what happens next. Tested AI agents and planning strategies against people, and the best agents still fell far short of human scores.
Abstract · I-PHYRE: Interactive Physical Reasoning
Current evaluation protocols predominantly assess physical reasoning in stationary scenes, creating a gap in evaluating agents' abilities to interact with dynamic events. While contemporary methods allow agents to modify initial scene configurations and observe consequences, they lack the capability to interact with events in real time. To address this, we introduce I-PHYRE, a framework that challenges agents to simultaneously exhibit intuitive physical reasoning, multi-step planning, and in-situ intervention. Here, intuitive physical reasoning refers to a quick, approximate understanding of physics to address complex problems; multi-step denotes the need for extensive sequence planning in I-PHYRE, considering each intervention can significantly alter subsequent choices; and in-situ implies the necessity for timely object manipulation within a scene, where minor timing deviations can result in task failure. We formulate four game splits to scrutinize agents' learning and generalization of essential principles of interactive physical reasoning, fostering learning through interaction with representative scenarios. Our exploration involves three planning strategies and examines several supervised and reinforcement agents' zero-shot generalization proficiency on I-PHYRE. The outcomes highlight a notable gap between existing learning algorithms and human performance, emphasizing the imperative for more research in enhancing agents with interactive physical reasoning capabilities. The environment and baselines will be made publicly available.
Shiqian Li, Kewen Wu, Chi Zhang, Yixin Zhu
arXiv:2312.03009 · cs.AI, cs.CV, cs.LG, cs.RO · submitted Dec 4, 2023 · updated Mar 25, 2024
abstract · pdf · html · 21 pages, ICLR 2024