Complexity Analysis of Real-Time Reinforcement Learning

9 years 2 months ago
Complexity Analysis of Real-Time Reinforcement Learning
This paper analyzes the complexity of on-line reinforcement learning algorithms, namely asynchronous realtime versions of Q-learning and value-iteration, applied to the problem of reaching a goal state in deterministic domains. Previous work had concluded that, in many cases, tabula rasa reinforcement learning was exponential for such problems, or was tractable only if the learning algorithm was augmented. We show that, to the contrary, the algorithms are tractable with only a simple change in the task representation or initialization. We provide tight bounds on the worst-case complexity, and show how the complexity is even smaller if the reinforcement learning algorithms have initial knowledge of the topology of the state space or the domain has certain special properties. We also present a novel bidirectional Q-learning algorithm to nd optimal paths from all states to a goal state and show that it is no more complex than the other algorithms.
Sven Koenig, Reid G. Simmons
Added 02 Nov 2010
Updated 02 Nov 2010
Type Conference
Year 1993
Where AAAI
Authors Sven Koenig, Reid G. Simmons
Comments (0)