Sciweavers

JAIR
2002

Efficient Reinforcement Learning Using Recursive Least-Squares Methods

13 years 3 months ago
Efficient Reinforcement Learning Using Recursive Least-Squares Methods
The recursive least-squares (RLS) algorithm is one of the most well-known algorithms used in adaptive filtering, system identification and adaptive control. Its popularity is mainly due to its fast convergence speed, which is considered to be optimal in practice. In this paper, RLS methods are used to solve reinforcement learning problems, where two new reinforcement learning algorithms using linear value function approximators are proposed and analyzed. The two algorithms are called RLS-TD( ) and Fast-AHC (Fast Adaptive Heuristic Critic), respectively. RLS-TD( ) can be viewed as the extension of RLS-TD(0) from =0 to general 0 1, so it is a multi-step temporal-difference (TD) learning algorithm using RLS methods. The convergence with probability one and the limit of convergence of RLS-TD( ) are proved for ergodic Markov chains. Compared to the existing LS-TD( ) algorithm, RLS-TD( ) has advantages in computation and is more suitable for online learning. The effectiveness of RLS-...
Xin Xu, Hangen He, Dewen Hu
Added 22 Dec 2010
Updated 22 Dec 2010
Type Journal
Year 2002
Where JAIR
Authors Xin Xu, Hangen He, Dewen Hu
Comments (0)