Replacing eligibility trace for action-value learning with function approximation

The eligibility trace is one of the most used mechanisms to speed up reinforcement learning. Earlier reported experiments seem to indicate that replacing eligibility traces would perform better than accumulating eligibility traces. However, replacing traces are currently not applicable when using function approximation methods where states are not represented uniquely by binary values. This paper proposes two modifications to replacing traces that overcome this limitation. Experimental results from the Mountain-Car task indicate that the new replacing traces outperform both the accumulating and the ‘ordinary’ replacing traces.

• Type: Conference
• Year: 2007
• Where: ESANN
• Authors: Kary Främling
• Keywords: Eligibility Traces, ESANN 2007, Function Approximation Methods, Neural Networks, Reinforcement Learning