Multi-Agent Imitation Learning for Driving Simulation

Simulation is an appealing option for validating the safety of autonomous vehicles. Generative Adversarial Imitation Learning (GAIL) has recently been shown to learn representative human driver models. These human driver models were learned through training in single-agent environments, but they have difficulty in generalizing to multi-agent driving scenarios. We argue these difficulties arise because observations at training and test time are sampled from different distributions. This difference makes such models unsuitable for the simulation of driving scenes, where multiple agents must interact realistically over long time horizons. We extend GAIL to address these shortcomings through a parameter-sharing approach grounded in curriculum learning. Compared with single-agent GAIL policies, policies generated by our PS-GAIL method prove superior at interacting stably in a multi-agent setting and capturing the emergent behavior of human drivers.

Authors

• Raunak P. Bhattacharyya
• Derek J. Phillips
• Blake Wulfe
• Jeremy Morton
• Alex Kuefler
• Mykel J. Kochenderfer

Keywords

• Vehicles
• Training
• Trajectory
• Optimization
• Biological system modeling
• Testing
• Markov processes
• Imitation Learning
• Horizon
• Human Model
• Autonomous Vehicles
• Emergent Behavior
• Vehicle Safety
• Human Drivers
• Driver Model
• Neural Network
• Root Mean Square Error
• Mean Square Error
• Environmental Policy
• Recurrent Neural Network
• Feed-forward Network
• Optimal Policy
• Number Of Agents
• Transition Model
• Reward Function
• Markov Decision Process
• Policy Learning
• Observation Space
• Covariate Shift
• Multi-agent Reinforcement Learning
• Inverse Reinforcement Learning
• Partial Observation
• Gated Recurrent Unit
• State-action Pair
• Step Size Parameter