Sciweavers

Share
FOCS
2007
IEEE

Approximation Algorithms for Partial-Information Based Stochastic Control with Markovian Rewards

10 years 5 months ago
Approximation Algorithms for Partial-Information Based Stochastic Control with Markovian Rewards
We consider a variant of the classic multi-armed bandit problem (MAB), which we call FEEDBACK MAB, where the reward obtained by playing each of n independent arms varies according to an underlying on/off Markov process with known parameters. The evolution of the Markov chain happens irrespective of whether the arm is played, and furthermore, the exact state of the Markov chain is only revealed to the player when the arm is played and the reward observed. At most one arm (or in general, M arms) can be played any time step. The goal is to design a policy for playing the arms in order to maximize the infinite horizon time average expected reward. This problem is an instance of a Partially Observable Markov Decision Process (POMDP), and a special case of the notoriously intractable “restless bandit” problem. Unlike the stochastic MAB problem, the FEEDBACK MAB problem does not admit to greedy index-based optimal policies. The state of the system at any time step encodes the beliefs ab...
Sudipto Guha, Kamesh Munagala
Added 02 Jun 2010
Updated 02 Jun 2010
Type Conference
Year 2007
Where FOCS
Authors Sudipto Guha, Kamesh Munagala
Comments (0)
books