Stochastic Enforced Hill-Climbing

Enforced hill-climbing is an effective deterministic hillclimbing technique that deals with local optima using breadth-first search (a process called "basin flooding"). We propose and evaluate a stochastic generalization of enforced hill-climbing for online use in goal-oriented probabilistic planning problems. We assume a provided heuristic function estimating expected cost to the goal with flaws such as local optima and plateaus that thwart straightforward greedy action choice. While breadth-first search is effective in exploring basins around local optima in deterministic problems, for stochastic problems we dynamically build and solve a local Markov-decision process model of the basin in order to find a good escape policy exiting the local optimum. We evaluate our proposal in a wide range of recent probabilistic planning-competition benchmark domains. For evaluation, we show that stochastic enforced hill-climbing produces better policies than greedy action choice for valu...

• Type: Conference
• Year: 2008
• Where: AIPS
• Authors: Jia-Hong Wu, Rajesh Kalyanam, Robert Givan
• Keywords: AIPS 2008, Artificial Intelligence, Greedy Action, Greedy Action Choice, Local Optima