Local Policy Optimization for Trajectory-Centric Reinforcement Learning

The goal of this paper is to present a method for simultaneous trajectory and local stabilizing policy optimization to generate local policies for trajectory-centric model-based reinforcement learning (MBRL). This is motivated by the fact that global policy optimization for non-linear systems could be a very challenging problem both algorithmically and numerically. However, a lot of robotic manipulation tasks are trajectory-centric, and thus do not require a global model or policy. Due to inaccuracies in the learned model estimates, an open-loop trajectory optimization process mostly results in very poor performance when used on the real system. Motivated by these problems, we try to formulate the problem of trajectory optimization and local policy synthesis as a single optimization problem. It is then solved simultaneously as an instance of nonlinear programming. We provide some results for analysis as well as achieved performance of the proposed technique under some simplifying assumptions.

Authors

• Patrik Kolaric
• Devesh K. Jha
• Arvind U. Raghunathan
• Frank L. Lewis
• Mouhacine Benosman
• Diego Romeres
• Daniel Nikovski

Keywords

• Robustness
• Trajectory optimization
• Uncertainty
• Learning (artificial intelligence)
• Robots
• Local Policy
• Optimal Policy
• Optimization Problem
• Nonlinear Systems
• Postural Stability
• Nonlinear Programming
• Global Policy
• Robotic Tasks
• Single Optimization Problem
• Model-based Reinforcement Learning
• Control System
• Absolute Difference
• System Dynamics
• Rest Of The Paper
• Uncertainty Estimation
• Optimization Procedure
• Control Performance
• Stability Control
• Gaussian Process
• Trajectory Optimization Problem
• Robust Constraints
• Dry Friction
• Uncertainty Region
• Systematic Deviations
• Rest Of This Section
• Ball Position
• Differences In Trajectories
• Feedback Gain
• Robust Optimization Problem