Taxonomy Driven Fast Adversarial Training

Adversarial training (AT) is an effective defense method against gradient-based attacks to enhance the robustness of neural networks. Among them, single-step AT has emerged as a hotspot topic due to its simplicity and efficiency, requiring only one gradient propagation in generating adversarial examples. Nonetheless, the problem of catastrophic overfitting (CO) that causes training collapse remains poorly understood, and there exists a gap between the robust accuracy achieved through single- and multi-step AT. In this paper, we present a surprising finding that the taxonomy of adversarial examples reveals the truth of CO. Based on this conclusion, we propose taxonomy driven fast adversarial training (TDAT) which jointly optimizes learning objective, loss function, and initialization method, thereby can be regarded as a new paradigm of single-step AT. Compared with other fast AT methods, TDAT can boost the robustness of neural networks, alleviate the influence of misclassified examples, and prevent CO during the training process while requiring almost no additional computational and memory resources. Our method achieves robust accuracy improvement of 1.59%, 1.62%, 0.71%, and 1.26% on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet-100 datasets, when against projected gradient descent PGD10 attack with perturbation budget 8/255. Furthermore, our proposed method also achieves state-of-the-art robust accuracy against other attacks. Code is available at https://github.com/bookman233/TDAT.

Authors

• Kun Tong Southeast University, Nanjing, China
• Chengze Jiang Southeast University, Nanjing, China
• Jie Gui School of Cyber Science and Engineering, Southeast University; Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education; Purple Mountain Laboratories
• Yuan Cao School of Computer Science and Technology, Ocean University of China

Keywords

• CV: Adversarial Attacks & Robustness
• CV: Scene Analysis & Understanding
• ML: Adversarial Learning & Robustness