Human Motion Capture Using a Drone

Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments. In this work we introduce a drone-based system for 3D human MoCap. The system only needs an autonomously flying drone with an on-board RGB camera and is usable in various indoor and outdoor environments. A reconstruction algorithm is developed to recover full-body motion from the video recorded by the drone. We argue that, besides the capability of tracking a moving subject, a flying drone also provides fast varying viewpoints, which is beneficial for motion reconstruction. We evaluate the accuracy of the proposed system using our new DroCap dataset and also demonstrate its applicability for MoCap in the wild using a consumer drone.

Authors

• Xiaowei Zhou 0001
• Sikang Liu 0002
• Georgios Pavlakos
• Vijay Kumar 0001
• Kostas Daniilidis

Keywords

• Cameras
• Drones
• Two dimensional displays
• Three-dimensional displays
• Image reconstruction
• Tracking
• Joints
• Human Motion
• Human Motion Capture
• Indoor Environments
• Reconstruction Algorithm
• RGB Camera
• Autonomous Drone
• Onboard Camera
• Convolutional Neural Network
• Smooth Function
• Bounding Box
• Pose Estimation
• Fast Motion
• System Calibration
• Structure From Motion
• Nuclear Norm
• Body Joints
• Human Pose Estimation
• Camera Motion
• Limb Length
• 3D Pose
• 2D Pose
• Virtual Camera
• Camera Rotation
• Bundle Adjustment
• Nuclear Norm Minimization
• RGB-D Sensor
• Camera Viewpoint
• Training Data
• Discrimination Method
• Computer Vision