RenderBender: A Survey on Adversarial Attacks Using Differentiable Rendering

Differentiable rendering techniques like Gaussian Splatting and Neural Radiance Fields have become powerful tools for generating high-fidelity models of 3D objects and scenes. Their ability to produce both physically plausible and differentiable models of scenes are key ingredient needed to produce physically plausible adversarial attacks on DNNs. However, the adversarial machine learning community has yet to fully explore these capabilities, partly due to differing attack goals (e. g. , misclassification, misdetection) and a wide range of possible scene manipulations used to achieve them (e. g. , alter texture, mesh). This survey contributes a framework that unifies diverse goals and tasks, facilitating easy comparison of existing work, identifying research gaps, and highlighting future directions—ranging from expanding attack goals and tasks to account for new modalities, state-of-the-art models, tools, and pipelines, to underscoring the importance of studying real-world threats in complex scenes.

Authors

• Matthew Hull
• Haoran Wang Shanghai Engineering Research Center of AI & Robotics, Academy for Engineering & Technology, Fudan University
• Matthew Lau
• Alec Helbling Georgia Tech
• Mansi Phute Georgia Tech
• Chao Zhang
• Zsolt Kira
• Willian Lunardi
• Martin Andreoni
• Wenke Lee
• Duen Horng Chau Georgia Institute of Technology

Keywords

• Computer Vision: CV: 3D computer vision
• Computer Vision: CV: Adversarial learning, adversarial attack and defense methods
• Computer Vision: CV: Machine learning for vision
• Machine Learning: ML: Adversarial machine learning