about
Generalized Policy Iteration for Optimal Control in Continuous Time (arxiv.org)
1 point by sel1 on Sep 16, 2019 | hide | past | pdf | discuss on HN

In plain words: A two-phase controller first tunes any starting policy until it is safe, then improves it toward a near-best control for nonlinear systems. Unlike usual approaches, it needs no safe start and works on general nonlinear systems; it is proven to converge and tracked vehicle paths.

Abstract · Relaxed Actor-Critic with Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems

This paper presents the Relaxed Continuous-Time Actor-critic (RCTAC) algorithm, a method for finding the nearly optimal policy for nonlinear continuous-time (CT) systems with known dynamics and infinite horizon, such as the path-tracking control of vehicles. RCTAC has several advantages over existing adaptive dynamic programming algorithms for CT systems. It does not require the ``admissibility" of the initialized policy or the input-affine nature of controlled systems for convergence. Instead, given any initial policy, RCTAC can converge to an admissible, and subsequently nearly optimal policy for a general nonlinear system with a saturated controller. RCTAC consists of two phases: a warm-up phase and a generalized policy iteration phase. The warm-up phase minimizes the square of the Hamiltonian to achieve admissibility, while the generalized policy iteration phase relaxes the update termination conditions for faster convergence. The convergence and optimality of the algorithm are proven through Lyapunov analysis, and its effectiveness is demonstrated through simulations and real-world path-tracking tasks.

Jingliang Duan, Jie Li, Qiang Ge, Shengbo Eben Li, Monimoy Bujarbaruah, Fei Ma, Dezhao Zhang
arXiv:1909.05402 · eess.SY, cs.LG · submitted Sep 11, 2019 · updated Mar 30, 2023
abstract · pdf · html

add comment on HN