Online Human Capability Estimation Through Reinforcement Learning and Interaction

Service robots are expected to assist users in a constantly growing range of environments and tasks. People may be unique in many ways, and online adaptation of robots is central to personalized assistance. We focus on collaborative tasks in which the human collaborator may not be fully ablebodied, with the aim for the robot to automatically determine the best level of support. We propose a methodology for online adaptation based on Reinforcement Learning and Bayesian inference. As the Reinforcement Learning process continuously adjusts the robot's behavior, the actions that become part of the improved policy are used by the Bayesian inference module as local evidence of human capability, which can be generalized across the state space. The estimated capabilities are then used as pre-conditions to collaborative actions, so that the robot can quickly disable actions that the person seems unable to perform. We demonstrate and validate our approach on two simulated tasks and one real-world collaborative task across a range of motion and sensing capabilities.

Authors

• Chengke Sun
• Anthony G. Cohn 0001
• Matteo Leonetti

Keywords

• Q-learning
• Service robots
• Estimation
• Collaboration
• Robot sensing systems
• Bayes methods
• Sensors
• Human Capabilities
• Bayesian Inference
• State Space
• Real-world Tasks
• Simulated Task
• Collaborative Tasks
• Reinforcement Learning Process
• Value Function
• Latent Variables
• Statistical Inference
• Experimental Evaluation
• Online Learning
• Optimal Policy
• Target Object
• Chickpea
• Manipulation Tasks
• Reward Function
• Real-world Experiments
• Markov Decision Process
• Reinforcement Learning Agent
• Navigation Task
• Robotic Assistance
• Real Robot
• Random Action