Differential dynamic programming with nonlinear constraints

Differential dynamic programming (DDP) is a widely used trajectory optimization technique that addresses nonlinear optimal control problems, and can readily handle nonlinear cost functions. However, it does not handle either state or control constraints. This paper presents a novel formulation of DDP that is able to accommodate arbitrary nonlinear inequality constraints on both state and control. The main insight in standard DDP is that a quadratic approximation of the value function can be derived using a recursive backward pass, however the recursive formulae are only valid for unconstrained problems. The main technical contribution of the presented method is a derivation of the recursive quadratic approximation formula in the presence of nonlinear constraints, after a set of active constraints has been identified at each point in time. This formula is used in a new Constrained-DDP (CDDP) algorithm that iteratively determines these active set and is guaranteed to converge toward a local minimum. CDDP is demonstrated on several underactuated optimal control problems up to 12D with obstacle avoidance and control constraints and is shown to outperform other methods for accommodating constraints.

Authors

• Zhaoming Xie
• C. Karen Liu
• Kris Hauser

Keywords

• Trajectory
• Optimal control
• Optimization
• Dynamic programming
• Approximation algorithms
• Standards
• Iterative methods
• Differentiation Program
• Nonlinear Constraints
• Differential Dynamic Programming
• Optimization Problem
• Value Function
• Taylor Series
• Nonlinear Programming
• Inequality Constraints
• Function Approximation
• Nonlinear Control
• Optimal Control Problem
• Trajectory Optimization
• Obstacle Avoidance
• State Constraints
• Active Constraints
• Recursive Formula
• Backward Pass
• Control Constraints
• Nominal Trajectory
• Forward Pass
• Sequential Quadratic Programming
• Current Trajectory
• Linear Constraints
• Collocation Method
• Time Step
• Body Frame
• Estimation Problem
• Trust Region