Functional object-oriented network for manipulation learning

This paper presents a novel structured knowledge representation called the functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects. Using a well-trained FOON, robots can decipher a task goal, seek the correct objects at the desired states on which to operate, and generate a sequence of proper manipulation motions. The paper describes FOON's structure and an approach to form a universal FOON with extracted knowledge from online instructional videos. A graph retrieval approach is presented to generate manipulation motion sequences from the FOON to achieve a desired goal, demonstrating the flexibility of FOON in creating a novel and adaptive means of solving a problem using knowledge gathered from multiple sources. The results are demonstrated in a simulated environment to illustrate the motion sequences generated from the FOON to carry out the desired tasks.

Authors

• David Paulius
• Yongqiang Huang 0001
• Roger Milton
• William D. Buchanan
• Jeanine Sam
• Yu Sun 0004

Keywords

• Videos
• Merging
• Robots
• Mirrors
• Neurons
• Object oriented modeling
• Visualization
• Knowledge Of Structure
• Simulation Environment
• Graphical Model
• Video For Instructions
• Manipulation Tasks
• Objective Conditions
• Motion Sequences
• Use Of Motion
• Interactive
• Motor Function
• Object Recognition
• Functional Unit
• Motor Response
• Action Recognition
• List Of Items
• Sequential Task
• Euler Angles
• Type Of Motion
• Mirror Neurons
• Depth-first
• Object Affordances
• Dynamic Time Warping
• Motion Generation
• Semantic Graph
• Bipartite Network
• Input Object
• 3D Motion Capture System
• Merging Process
• Set Of Degrees
• Motion Trajectory