Gaussian Process Bandits for Tree Search

We motivate and analyse a new Tree Search algorithm, based on recent advances in the use of Gaussian Processes for bandit problems. We assume that the function to maximise on the leaves of the tree is drawn from a GP with a prior kernel function defined between the paths from the root to the leaves. We consider these paths as arms of a bandit problem, and we use the GP-UCB algorithm to select those to explore: the posterior mean and variance after observing t samples are used to define confidence intervals for the f values, and, at each time step t + 1, we select the arm with highest upper confidence bound to sample f at. Our contribution is in the application of GP-UCB to tree search: we give a detailed description of the resulting algorithm, and we adapt recent cumulative regret bounds by determining the decay rate of the eigenvalues of the kernel matrix on the whole set of tree paths. We consider two kernels in the feature space of binary vectors indexed by the nodes of the tree: l...

• Type: Journal
• Year: 2010
• Where: CORR
• Authors: Louis Dorard, John Shawe-Taylor
• Keywords: CORR 2010, Education, Tree, Tree Search, Tree Search Algorithm