Binary action search for learning continuous-action control policies

Reinforcement Learning methods for controlling stochastic processes typically assume a small and discrete action space. While continuous action spaces are quite common in real-world problems, the most common approach still employed in practice is coarse discretization of the action space. This paper presents a novel method, called Binary Action Search, for realizing continuousaction policies by searching efficiently the entire action range through increment and decrement modifications to the values of the action variables according to an internal binary policy defined over an augmented state space. The proposed approach essentially approximates any continuous action space to arbitrary resolution and can be combined with any discrete-action reinforcement learning algorithm for learning continuousaction policies. Binary Action Search eliminates the restrictive modification steps of Adaptive Action Modification and requires no temporal action locality in the domain. Our approach is coupl...

• Type: Conference
• Year: 2009
• Where: ICML
• Authors: Jason Pazis, Michail G. Lagoudakis
• Keywords: Binary Action Search, Continuous Action Space, Discrete Action Space, ICML 2009, Machine Learning