Adversarial Differentiable Data Augmentation for Autonomous Systems

Autonomous systems often rely on neural networks to achieve high performance on planning and control problems. Unfortunately, neural networks suffer severely when input images become degraded in ways that are not reflected in the training data. This is particularly problematic for robotic systems like autonomous vehicles (AV) for which reliability is paramount. In this work, we consider robust optimization methods for hardening control systems against image corruptions and other unexpected domain shifts. Recent work on robust optimization for neural nets has been focused largely on combating adversarial attacks. In this work, we borrow ideas from the adversarial training and data augmentation literature to enhance robustness to image corruptions and domain shifts. To this end, we train networks while augmenting image data with a battery of image degradations. Unlike traditional augmentation methods, we choose the parameters for each degradation adversarially so as to maximize system performance. By formulating image degradations in a way that is differentiable with respect to degradation parameters, we enable the use of efficient optimization methods (PGD) for choosing worst-case augmentation parameters. We demonstrate the efficacy of this method on the learning to steer task for AVs. By adversarially training against image corruptions, we produce networks that are highly robust to image corruptions. We show that the proposed differentiable augmentation schemes result in higher levels of robustness and accuracy for a range of settings as compared to baseline and state-of-the-art augmentation methods.

Authors

• Manli Shu
• Yu Shen
• Ming Lin 0003
• Tom Goldstein

Keywords

• Degradation
• Training
• Neural networks
• Optimization methods
• Training data
• Robustness
• Data models
• Autonomic System
• Data Augmentation
• Adversarial Data
• Adversarial Data Augmentation
• Differentiable Data Augmentation
• Neural Network
• Input Image
• Control Problem
• Domain Shift
• Autonomous Vehicles
• Augmentation Methods
• Robust Optimization
• Adversarial Training
• Adversarial Attacks
• Image Degradation
• Projected Gradient Descent
• Convolutional Neural Network
• Image Quality
• Gaussian Noise
• Adversarial Examples
• Augmentation Operations
• Robust System
• RGB Space
• Type Of Augmentation
• Steering Angle
• Test Scenarios
• Gaussian Blur
• Augmented Training
• Good Weather Conditions