A natural language planner interface for mobile manipulators

Natural language interfaces for robot control aspire to find the best sequence of actions that reflect the behavior intended by the instruction. This is difficult because of the diversity of language, variety of environments, and heterogeneity of tasks. Previous work has demonstrated that probabilistic graphical models constructed from the parse structure of natural language can be used to identify motions that most closely resemble verb phrases. Such approaches however quickly succumb to computational bottlenecks imposed by construction and search the space of possible actions. Planning constraints, which define goal regions and separate the admissible and inadmissible states in an environment model, provide an interesting alternative to represent the meaning of verb phrases. In this paper we present a new model called the Distributed Correspondence Graph (DCG) to infer the most likely set of planning constraints from natural language instructions. A trajectory planner then uses these planning constraints to find a sequence of actions that resemble the instruction. Separating the problem of identifying the action encoded by the language into individual steps of planning constraint inference and motion planning enables us to avoid computational costs associated with generation and evaluation of many trajectories. We present experimental results from comparative experiments that demonstrate improvements in efficiency in natural language understanding without loss of accuracy.

Authors

• Thomas M. Howard
• Stefanie Tellex
• Nicholas Roy

Keywords

• Grounding
• Mathematical model
• Robots
• Planning
• Natural languages
• Trajectory
• Equations
• Natural Language
• Probabilistic Model
• Sequence Of Actions
• Efficiency Improvement
• Graphical Model
• Path Planning
• Trajectory Planning
• Probabilistic Graphical Models
• Verb Phrase
• Natural Language Understanding
• Training Set
• Conditional Independence
• Rigid Body
• Red Box
• Part-of-speech
• Configuration Space
• Blue Box
• Green Box
• Human-robot Interaction
• Pseudotime
• Noun Phrase
• Prepositional Phrase
• Trajectories In Space
• Parse Tree
• Factor Graph
• Robot Base
• State Action Space
• Similar Physical Characteristics
• Probable Trajectory
• Colored Boxes