Learning Stable Normalizing-Flow Control for Robotic Manipulation

Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control stability. Ideally, one would like to achieve stability guarantees while staying within the framework of state-of-the-art deep RL algorithms. Such a solution does not exist in general, especially one that scales to complex manipulation tasks. We contribute towards closing this gap by introducing normalizing-flow control structure, that can be deployed in any latest deep RL algorithms. While stable exploration is not guaranteed, our method is designed to ultimately produce deterministic controllers with provable stability. In addition to demonstrating our method on challenging contact-rich manipulation tasks, we also show that it is possible to achieve considerable exploration efficiency–reduced state space coverage and actuation efforts– without losing learning efficiency.

Authors

• Shahbaz Abdul Khader
• Hang Yin 0001
• Pietro Falco
• Danica Kragic

Keywords

• Automation
• Conferences
• Reinforcement learning
• Aerospace electronics
• Space exploration
• Control theory
• Task analysis
• Stability Control
• Robot Manipulator
• State Space
• Efficient Learning
• Manipulation Tasks
• Deep Reinforcement Learning
• Reinforcement Learning Algorithm
• Deep Reinforcement Learning Algorithm
• Stability Guarantees
• Neural Network
• System Dynamics
• Energy Function
• Trainable Parameters
• Equilibrium Point
• Asymptotically Stable
• Joint Space
• Reward Function
• Real-world Experiments
• Policy Gradient
• Joint Torque
• Proximal Policy Optimization
• Goal Position
• Deep Reinforcement Learning Method
• Policy Search
• Lyapunov Analysis
• Deterministic Control
• Deterministic Policy
• Reinforcement Learning Process
• Efficient Exploration
• Reinforcement Learning Problem