Automatic weight learning for multiple data sources when learning from demonstration

Traditional approaches to programming robots are generally inaccessible to non-robotics-experts. A promising exception is the learning from demonstration paradigm. Here a policy mapping world observations to action selection is learned, by generalizing from task demonstrations by a teacher. Most learning from demonstration work to date considers data from a single teacher. In this paper, we consider the incorporation of demonstrations from multiple teachers. In particular, we contribute an algorithm that handles multiple data sources, and additionally reasons about reliability differences between them. For example, multiple teachers could be inequally proficient at performing the demonstrated task. We introduce Demonstration Weight Learning (DWL) as a learning from demonstration algorithm that explicitly represents multiple data sources and learns to select between them, based on their observed reliability and according to an adaptive expert learning inspired approach. We present a first implementation of DWL within a simulated robot domain. Data sources are shown to differ in reliability, and weighting is found impact task execution success. Furthermore, DWL is shown to produce appropriate data source weights that improve policy performance.

Authors

• Brenna Argall
• Brett Browning
• Manuela Veloso

Keywords

• Robotics and automation
• Robot programming
• Humans
• Automatic programming
• Mathematical model
• Educational robots
• Computer science
• Application software
• Motion control
• Motion planning
• Data Sources
• Multiple Data Sources
• Inverse Reinforcement Learning
• Task Execution
• Reliable Differences
• Multiple Teachers
• Simulated Robot
• Learning Algorithms
• Policy Development
• Motor Control
• Global Status
• Equal Weight
• Probability Of Selection
• Robotic Applications
• Learned Weights
• Policy Learning
• Range Of Sensors
• Execution Speed
• Weighting Scheme
• Changes In Head
• Dynamic Update
• Reliable Data Source
• Data Source Selection
• Query Point
• General Society
• Policy Execution