Sciweavers

AAAI
2006

Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions

13 years 5 months ago
Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions
Following verbal route instructions requires knowledge of language, space, action and perception. We present MARCO, an agent that follows free-form, natural language route instructions by representing and executing a sequence of compound action specifications that model which actions to take under which conditions. MARCO infers implicit actions from knowledge of both linguistic conditional phrases and from spatial action and local configurations. Thus, MARCO performs explicit actions, implicit actions necessary to achieve the stated conditions, and exploratory actions to learn about the world. We gathered a corpus of 786 route instructions from six people in three large-scale virtual indoor environments. Thirtysix other people followed these instructions and rated them for quality. These human participants finished at the intended destination on 69% of the trials. MARCO followed the same instructions in the same environments, with a success rate of 61%. We measured the efficacy of act...
Matt MacMahon, Brian Stankiewicz, Benjamin Kuipers
Added 30 Oct 2010
Updated 30 Oct 2010
Type Conference
Year 2006
Where AAAI
Authors Matt MacMahon, Brian Stankiewicz, Benjamin Kuipers
Comments (0)