In plain words: When you can't see the state and must commit to an action sequence, this planner keeps a stack of trial-and-error choosers, growing or shrinking it to fit the plan's length using only a simulator. On four problems it stayed effective and insensitive to settings.
Abstract
We propose Stable Yet Memory Bounded Open-Loop (SYMBOL) planning, a general memory bounded approach to partially observable open-loop planning. SYMBOL maintains an adaptive stack of Thompson Sampling bandits, whose size is bounded by the planning horizon and can be automatically adapted according to the underlying domain without any prior domain knowledge beyond a generative model. We empirically test SYMBOL in four large POMDP benchmark problems to demonstrate its effectiveness and robustness w.r.t. the choice of hyperparameters and evaluate its adaptive memory consumption. We also compare its performance with other open-loop planning algorithms and POMCP.
Thomy Phan, Thomas Gabor, Robert Müller, Christoph Roch, Claudia Linnhoff-Popien
arXiv:1907.05861 · cs.AI · submitted Jul 11, 2019 · updated Dec 28, 2023
abstract · pdf · html · Accepted to IJCAI 2019. arXiv admin note: substantial text overlap with arXiv:1905.04020