Sciweavers

Share
ICML
2002
IEEE

Action Refinement in Reinforcement Learning by Probability Smoothing

12 years 2 months ago
Action Refinement in Reinforcement Learning by Probability Smoothing
In many reinforcement learning applications, the set of possible actions can be partitioned by the programmer into subsets of similar actions. This paper presents a technique for exploiting this form of prior information to speed up model-based reinforcement learning. We call it an action refinement method, because it treats each subset of similar aca single "abstract" action early in the learning process and then later "refines" ract action into individual actions as more experience is gathered. Our method estimates the transition probabilities P(s |s, a) for an action a by combining the results of executions of action a with executions of other actions in the same subset of similar actions. This is a form of "smoothing" of the probability estimates that trades increased bias for reduced variance. The paper derives a formula for optimal smoothing which shows that the degree of smoothing should decrease as the amount of data increases. Experiments show th...
Carles Sierra, Dídac Busquets, Ramon L&oacu
Added 17 Nov 2009
Updated 17 Nov 2009
Type Conference
Year 2002
Where ICML
Authors Carles Sierra, Dídac Busquets, Ramon López de Mántaras, Thomas G. Dietterich
Comments (0)
books