Skill Generalization with Verbs

It is imperative that robots can understand natural language commands issued by humans. Such commands typically contain verbs that signify what action should be performed on a given object and that are applicable to many objects. We propose a method for generalizing manipulation skills to novel objects using verbs. Our method learns a probabilistic classifier that determines whether a given object trajectory can be described by a specific verb. We show that this classifier accurately generalizes to novel object categories with an average accuracy of 76. 69% across 13 object categories and 14 verbs. We then perform policy search over the object kinematics to find an object trajectory that maximizes classifier prediction for a given verb. Our method allows a robot to generate a trajectory for a novel object based on a verb, which can then be used as input to a motion planner. We show that our model can generate trajectories that are usable for executing five verb commands applied to novel instances of two different object categories on a real robot.

Authors

• Rachel Ma
• Lyndon Lam
• Benjamin A. Spiegel
• Aditya Ganeshan
• Roma Patel
• Ben Abbatematteo
• David Paulius
• Stefanie Tellex
• George Konidaris 0001

Keywords

• Natural languages
• Kinematics
• Search problems
• Probabilistic logic
• Trajectory
• Planning
• Intelligent robots
• Kinematic
• Natural Language
• Path Planning
• Real Robot
• Object Trajectory
• Changes In Conditions
• Optimization Algorithm
• Number Of Steps
• Motor Skills
• Image Object
• Termination Condition
• Manipulation Tasks
• Robotic Arm
• Objective Conditions
• Markov Decision Process
• Trajectory Optimization
• Adverbs
• Number Of Time Steps
• Dishwasher
• Object Instances
• Washing Machine
• Instances Of Categories
• Imitation Learning
• CNN Classifier
• Goal State
• Object Recognition