Optimizing Trajectories with Closed-Loop Dynamic SQP

Indirect trajectory optimization methods such as Differential Dynamic Programming (DDP) have found considerable success when only planning under dynamic feasibility constraints. Meanwhile, nonlinear programming (NLP) has been the state-of-the-art approach when faced with additional constraints (e. g. , control bounds, obstacle avoidance). However, a naïve implementation of NLP algorithms, e. g. , shooting-based sequential quadratic programming (SQP), may suffer from slow convergence – caused from natural instabilities of the underlying system manifesting as poor numerical stability within the optimization. Re-interpreting the DDP closed-loop rollout policy as a sensitivity-based correction to a second-order search direction, we demonstrate how to compute analogous closedloop policies (i. e. , feedback gains) for constrained problems. Our key theoretical result introduces a novel dynamic programmingbased constraint-set recursion that augments the canonical “cost-to-go” backward pass. On the algorithmic front, we develop a hybrid-SQP algorithm incorporating DDP-style closedloop rollouts, enabled via efficient parallelized computation of the feedback gains. Finally, we validate our theoretical and algorithmic contributions on a set of increasingly challenging benchmarks, demonstrating significant improvements in convergence speed over standard open-loop SQP.

Authors

• Sumeet Singh
• Jean-Jacques E. Slotine
• Vikas Sindhwani

Keywords

• Jacobian matrices
• Heuristic algorithms
• Dynamic programming
• Robots
• Programming
• Perturbation methods
• Trajectory optimization
• Parallelization
• Additional Constraints
• Indirect Method
• Quadratic Programming
• Numerical Stability
• Slow Convergence
• Search Direction
• Obstacle Avoidance
• Sequential Quadratic Programming
• Backward Pass
• Value Function
• Optimal Control
• Morphine
• Equality Constraints
• Path Planning
• Newton Method
• Optimal Sequence
• Pendulum
• Linear Dynamics
• Sensitivity Matrix
• Bellman Equation
• Linear Constraints
• Forward Pass
• Stable Equilibrium
• Interior Point
• Optimization Variables