Tree-structured Policy Planning with Learned Behavior Models

Autonomous vehicles (AVs) need to reason about the multimodal behavior of neighboring agents while planning their own motion. Many existing trajectory planners seek a single trajectory that performs well under all plausible futures simultaneously, ignoring bi-directional interactions and thus leading to overly conservative plans. Policy planning, whereby the ego agent plans a policy that reacts to the environment's multimodal behavior, is a promising direction as it can account for the action-reaction interactions between the AV and the environment. However, most existing policy planners do not scale to the complexity of real autonomous vehicle applications: they are either not compatible with modern deep learning prediction models, not interpretable, or not able to generate high quality trajectories. To fill this gap, we propose Tree Policy Planning (TPP), a policy planner that is compatible with state-of-the-art deep learning prediction models, generates multistage motion plans, and accounts for the influence of ego agent on the environment behavior. The key idea of TPP is to reduce the continuous optimization problem into a tractable discrete Markov Decision Process (MDP) through the construction of two tree structures: an ego trajectory tree for ego trajectory options, and a scenario tree for multi-modal ego-conditioned environment predictions. We demonstrate the efficacy of TPP in closed-loop simulations based on real-world nuScenes dataset and results show that TPP scales to realistic AV scenarios and significantly outperforms non-policy baselines.

Authors

• Yuxiao Chen 0008
• Péter Karkus
• Boris Ivanovic
• Xinshuo Weng
• Marco Pavone 0001

Keywords

• Deep learning
• Runtime
• Simulation
• Predictive models
• Real-time systems
• Trajectory
• Behavioral sciences
• Tree Structure
• Policy Planning
• Prediction Model
• Autonomous Vehicles
• Path Planning
• Environmental Behavior
• Conservation Planning
• Markov Decision Process
• Single Trajectory
• Deep Learning Model For Prediction
• Value Function
• Deep Learning Models
• Latent Space
• Dynamic Programming
• Root Node
• Optimal Policy
• Terminal Point
• Mode Of Distribution
• Child Nodes
• Ego-motion
• Trajectory Prediction
• Vehicle Motion
• Closed-loop Performance
• Lane Change
• Future Motion
• Distribution Of Trajectories
• Contingency Plans