Incremental Task Modification via Corrective Demonstrations

In realistic environments, fully specifying a task model such that a robot can perform a task in all situations is impractical. In this work, we present Incremental Task Modification via Corrective Demonstrations (ITMCD), a novel algorithm that allows a robot to update a learned model by making use of corrective demonstrations from an end-user in its environment. We propose three different types of model updates that make structural changes to a finite state automaton (FSA) representation of the task by first converting the FSA into a state transition auto-regressive hidden Markov model (STARHMM). The STARHMM's probabilistic properties are then used to perform approximate Bayesian model selection to choose the best model update, if any. We evaluate ITMCD Model Selection in a simulated block sorting domain and the full algorithm on a real-world pouring task. The simulation results show our approach can choose new task models that sufficiently incorporate new demonstrations while remaining as simple as possible. The results from the pouring task show that ITMCD performs well when the modeled segments of the corrective demonstrations closely comply with the original task model.

Authors

• Reymundo A. Gutierrez
• Vivian Chu
• Andrea Thomaz
• Scott Niekum

Keywords

• Hidden Markov models
• Task analysis
• Robots
• Computational modeling
• Adaptation models
• Probabilistic logic
• Bayes methods
• Incremental Modifications
• Model Selection
• Transition State
• Bayes Factor
• State Machine
• Updated Model
• Real-world Tasks
• Task Representations
• Probabilistic Properties
• Time Step
• Akaike Information Criterion
• Phase Transition
• Markov Chain Monte Carlo
• Free Parameters
• Lidocaine
• Increase In Parameters
• Directed Graph
• Gaussian Mixture Model
• Termination Condition
• Reward Function
• Type Of Editing
• Akaike Information Criterion Scores
• Pitchers
• Inverse Reinforcement Learning
• Policy Model
• Side Of The Table
• Physical Robot
• Increase In The Likelihood
• Blue Block
• Incremental Change