My research attempts to address on-line action selection in reinforcement learning from a Bayesian perspective. The idea is to develop more effective action selection techniques by exploiting information in a Bayesian posterior, while also selecting actions by growing an adaptive, sparse lookahead tree. I further augment the approach by considering a new value function approximation strategy for the belief-state Markov decision processes induced by Bayesian learning. Bayesian Reinforcement Learning Imagine a mobile vendor robot ("vendorbot") loaded with snacks and bustling around a building, learning where to visit to optimize its profit. The robot must choose wisely between selling snacks somewhere far away from its home or going back to its charger before its battery dies. How could a robot effectively learn to behave from its experience (previous sensations and actions) in such an environment? Reinforcement mechanisms provide a robot an opportunity to improve its decision...