Learning equivalent action choices from demonstration

In their interactions with the world robots inevitably face equivalent action choices, situations in which multiple actions are equivalently applicable. In this paper, we address the problem of equivalent action choices in learning from demonstration, a robot learning approach in which a policy is acquired from human demonstrations of the desired behavior. We note that when faced with a choice of equivalent actions, a human teacher often demonstrates an action arbitrarily and does not make the choice consistently over time. The resulting inconsistently labeled training data poses a problem for classification-based demonstration learning algorithms by violating the common assumption that for any world state there exists a single best action. This problem has been overlooked by previous approaches for demonstration learning. In this paper, we present an algorithm that identifies regions of the state space with conflicting demonstrations and enables the choice between multiple actions to be represented explicitly within the robotpsilas policy. An experimental evaluation of the algorithm in a real-world obstacle avoidance domain shows that reasoning about action choices significantly improves the robotpsilas learning performance.

Authors

• Sonia Chernova
• Manuela Veloso

Keywords

• Robots
• Classification algorithms
• Robot sensing systems
• Distance measurement
• Training
• Humans
• Nearest neighbor searches
• Training Data
• Learning Algorithms
• Experimental Evaluation
• Multiple Activities
• Regions Of Space
• Single Activity
• Obstacle Avoidance
• Human Education
• Inverse Reinforcement Learning
• Difference In The Number
• Performance Of Algorithm
• K-nearest Neighbor
• Low Confidence
• Gaussian Mixture Model
• Decision Boundary
• Confidence Threshold
• Action Classes
• Policy Learning
• Sensor Noise
• Nearest Neighbor Distance
• Number Of Demonstrations
• Improve Learning Performance
• Action Labels
• Existence Of Noise
• Underlying Data Distribution
• High-level Tasks