UAV-Sim: NeRF-based Synthetic Data Generation for UAV-based Perception

Tremendous variations coupled with large degrees of freedom in UAV-based imaging conditions lead to a significant lack of data in adequately learning UAV-based perception models. Using various synthetic renderers in conjunction with perception models is prevalent to create synthetic data to augment the learning in the ground-based imaging domain. However, severe challenges in the austere UAV-based domain require distinctive solutions to image synthesis for data augmentation. In this work, we leverage recent advancements in neural rendering to improve static and dynamic novel-view UAV-based image synthesis, especially from high altitudes, capturing salient scene attributes. Finally, we demonstrate a considerable performance boost is achieved when a state-of-the-art detection model is optimized primarily on hybrid sets of real and synthetic data instead of the real or synthetic data separately.

Authors

• Christopher Maxey
• Jaehoon Choi
• Hyungtae Lee
• Dinesh Manocha
• Heesung Kwon

Keywords

• Image synthesis
• Imaging
• Rendering (computer graphics)
• Data augmentation
• Data models
• Robotics and automation
• Synthetic data
• High Altitude
• Detection Model
• Imaging Conditions
• Large Degree Of Freedom
• Training Data
• Object Detection
• Spatial Dimensions
• Bounding Box
• Unmanned Aerial Vehicles
• Object Of Interest
• Viewing Angle
• Action Recognition
• Pose Estimation
• Masked Images
• Camera Pose
• Dynamic Scenes
• Domain Gap
• Object Detection Model
• Static Scenes
• View Synthesis
• Set Of Planes
• Unmanned Aerial Vehicle Altitude
• Scene Details