about
A Tutorial on Thompson Sampling (arxiv.org)
2 points by gwern on Feb 16, 2018 | hide | past | pdf | discuss on HN

In plain words: Thompson sampling handles sequential choices by drawing a random guess from your beliefs about each option and taking the best draw, balancing what works against learning more. The tutorial shows it stays efficient when one action reveals information about others, and when it fails.

Abstract

Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what is known to maximize immediate performance and investing to accumulate new information that may improve future performance. The algorithm addresses a broad range of problems in a computationally efficient manner and is therefore enjoying wide use. This tutorial covers the algorithm and its application, illustrating concepts through a range of examples, including Bernoulli bandit problems, shortest path problems, product recommendation, assortment, active learning with neural networks, and reinforcement learning in Markov decision processes. Most of these problems involve complex information structures, where information revealed by taking an action informs beliefs about other actions. We will also discuss when and why Thompson sampling is or is not effective and relations to alternative algorithms.

Daniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen
arXiv:1707.02038 · cs.LG · submitted Jul 7, 2017 · updated Jul 14, 2020
abstract · pdf · html

add comment on HN
Also discussed: Jun 2024 (1 point, 0 comments) · Jun 2024 (2 points, 0 comments)