Active Exploration for Robotic Manipulation

Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from scratch with our method, on a challenging ball pushing task on tilted tables, where the target ball position is not known to the agent a-priori. Our real-world robot experiment serves as a fundamental application of active exploration in model-based reinforcement learning of complex robotic manipulation tasks. Project page https://sites.google.com/view/aerm.

Authors

• Tim Schneider
• Boris Belousov
• Georgia Chalvatzaki
• Diego Romeres
• Devesh K. Jha
• Jan Peters 0001

Keywords

• Systematics
• Heuristic algorithms
• Tactile sensors
• Reinforcement learning
• Predictive models
• Probabilistic logic
• Behavioral sciences
• Robot Manipulator
• Dynamic Environment
• Information Gain
• Manipulation Tasks
• Model Predictive Control
• Real-world Experiments
• Advanced Machine Learning
• Robotic Tasks
• Real Robot
• Project Page
• Advanced Robotics
• Model-based Reinforcement Learning
• Neural Network
• Learning Models
• Low Probability
• Weighting Factor
• Kullback-Leibler
• Path Planning
• Hidden State
• Tactile Sensor
• Epistemic Uncertainty
• Extrinsic Rewards
• Intrinsic Rewards
• Markov Decision Process
• Simulated Task
• Active Inference
• Reward Model
• Target Zone
• Ensemble Of Neural Networks
• Real-world Tasks