Sciweavers

Share
CORR
2011
Springer

Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems

8 years 7 months ago
Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems
The analysis of online least squares estimation is at the heart of many stochastic sequential decision-making problems. We employ tools from the self-normalized processes to provide a simple and self-contained proof of a tail bound of a vector-valued martingale. We use the bound to construct new tighter confidence sets for the least squares estimate. We apply the confidence sets to several online decision problems, such as the multi-armed and the linearly parametrized bandit problems. The confidence sets are potentially applicable to other problems such as sleeping bandits, generalized linear bandits, and other linear control problems. We improve the regret bound of the Upper Confidence Bound (UCB) algorithm of Auer et al. (2002) and show that its regret is with high-probability a problem dependent constant. In the case of linear bandits (Dani et al., 2008), we improve the problem dependent bound in the dimension and number of time steps. Furthermore, as opposed to the previous re...
Yasin Abbasi-Yadkori, Dávid Pál, Csa
Added 13 May 2011
Updated 13 May 2011
Type Journal
Year 2011
Where CORR
Authors Yasin Abbasi-Yadkori, Dávid Pál, Csaba Szepesvári
Comments (0)
books