Visual-Inertial Navigation Algorithm Development Using Photorealistic Camera Simulation in the Loop

The development of fast, agile micro Unmanned Aerial Vehicles (UAVs) has been limited by (i) on-board computing hardware restrictions, (ii) the lack of sophisticated vision-based perception and vision-in-the-loop control algorithms, and (iii) the absence of development environments where such systems and algorithms can be rapidly and easily designed, implemented, and validated. Here, we first present a new micro UAV platform that integrates high-rate cameras, inertial sensors, and an NVIDIA Jetson Tegra X1 system-on-chip compute module that boasts 256 GPU cores. The UAV mechanics and electronics were designed and built in house, and are described in detail. Second, we present a novel “virtual reality” development environment, in which photorealistically-rendered synthetic on-board camera images are generated in real time while the UAV is in flight. This development environment allows us to rapidly prototype computing and sensing hardware as well as perception and control algorithms, using real physics, real interoceptive sensor data (e. g. , from the on-board inertial measurement unit), and synthetic exteroceptive sensor data (e. g. , from synthetic cameras). Third, we demonstrate repeated agile maneuvering with closed-loop vision-based perception and control algorithms, which we have developed using this environment.

Authors

• Thomas Sayre-McCord
• Winter Guerra
• Amado Antonini
• Jasper Arneberg
• Austin Brown
• Guilherme Venturelli Cavalheiro
• Yajun Fang
• Alex A. Gorodetsky
• Dave McCoy
• Sebastian Quilter
• Fabian Riether
• Ezra Tal
• Yunus Terzioglu
• Luca Carlone
• Sertac Karaman

Keywords

• Cameras
• Sensors
• Visualization
• Real-time systems
• Unmanned aerial vehicles
• Navigation
• Solid modeling
• Evolutionary Algorithms
• Navigation Algorithm
• Visual-inertial Navigation
• Development Environment
• Camera Images
• Inertial Measurement Unit
• Closed-loop Control
• Interoceptive
• Onboard Camera
• Unmanned Aerial Vehicle Platform
• Data Visualization
• Simulation System
• Frame Rate
• Simulation Environment
• Motion Capture
• Visual Measurements
• Posterior Mode
• Maximum A Posteriori
• Extended Kalman Filter
• Visual-inertial Odometry
• Feature Tracking
• Sensor Configuration
• Visual Simulation
• Hardware-in-the-loop
• Factor Graph
• Accurate State Estimation
• Machine Vision
• Local Frame
• RGB-D Sensor