about
Reward Advancement: Transforming Policy Under Maximum Causal Entropy Principle (arxiv.org)
2 points by sel1 on Jul 13, 2019 | hide | past | pdf | discuss on HN

In plain words: The problem is finding extra rewards that nudge a human-like, imperfectly rational decision maker from its current behavior to a chosen target behavior. It proves infinitely many such rewards work, and gives a way to pick the one that costs least to apply.

Abstract · Reward Advancement: Transforming Policy under Maximum Causal Entropy Principle

Many real-world human behaviors can be characterized as a sequential decision making processes, such as urban travelers choices of transport modes and routes (Wu et al. 2017). Differing from choices controlled by machines, which in general follows perfect rationality to adopt the policy with the highest reward, studies have revealed that human agents make sub-optimal decisions under bounded rationality (Tao, Rohde, and Corcoran 2014). Such behaviors can be modeled using maximum causal entropy (MCE) principle (Ziebart 2010). In this paper, we define and investigate a general reward trans-formation problem (namely, reward advancement): Recovering the range of additional reward functions that transform the agent's policy from original policy to a predefined target policy under MCE principle. We show that given an MDP and a target policy, there are infinite many additional reward functions that can achieve the desired policy transformation. Moreover, we propose an algorithm to further extract the additional rewards with minimum "cost" to implement the policy transformation.

Guojun Wu, Yanhua Li, Zhenming Liu, Jie Bao, Yu Zheng, Jieping Ye, Jun Luo
arXiv:1907.05390 · cs.AI · submitted Jul 11, 2019
abstract · pdf · html

add comment on HN