Hierarchical Traces for Reduced NSM Memory Requirements

10 years 2 months ago
Hierarchical Traces for Reduced NSM Memory Requirements
This paper presents work on using hierarchical long term memory to reduce the memory requirements of nearest sequence memory (NSM) learning, a previously published, instance-based reinforcement learning algorithm. A hierarchical memory representation reduces the memory requirements by allowing traces to share common sub-sequences. We present moderated mechanisms for estimating discounted future rewards and for dealing with hidden state using hierarchical memory. We also present an experimental analysis of how the sub-sequence length affects the memory compression achieved and show that the reduced memory requirements do not effect the speed of learning. Finally, we analyse and discuss the persistence of the sub-sequences independent of specific trace instances.
Torbjørn S. Dahl
Added 15 Feb 2011
Updated 15 Feb 2011
Type Journal
Year 2010
Where SGAI
Authors Torbjørn S. Dahl
Comments (0)