Self Supervised Learning for Multiple Object Tracking in 3D Point Clouds

Multiple object tracking in 3D point clouds has applications in mobile robots and autonomous driving. This is a challenging problem due to the sparse nature of the point clouds and the added difficulty of annotation in 3D for supervised learning. To overcome these challenges, we propose a neural network architecture that learns effective object features and their affinities in a self supervised fashion for multiple object tracking in 3D point clouds captured with LiDAR sensors. For self supervision, we use two approaches. First, we generate two augmented LiDAR frames from a single real frame by applying translation, rotation and cutout to the objects. Second, we synthesize a LiDAR frame using CAD models or primitive geometric shapes and then apply the above three augmentations to them. Hence, the ground truth object locations and associations are known in both frames for self supervision. This removes the need to annotate object associations in real data, and additionally the need for training data collection and annotation for object detection in synthetic data. To the best of our knowledge, this is the first self supervised multiple object tracking method for 3D data. Our model achieves state of the art results.

Authors

• Aakash Kumar
• Jyoti Kini
• Ajmal Mian
• Mubarak Shah

Keywords

• Point cloud compression
• Solid modeling
• Three-dimensional displays
• Laser radar
• Shape
• Annotations
• Neural networks
• Supervised Learning
• Point Cloud
• Object Tracking
• Self-supervised Learning
• 3D Point Cloud
• Multiple Object Tracking
• Training Data
• Object Detection
• Geometric Shapes
• Object Features
• Single Frame
• Computer-aided Design
• Mobile Robot
• LiDAR Sensor
• Ground Truth Object
• Need For Annotation
• Convolutional Layers
• Pedestrian
• Bounding Box
• Number Of Objects
• KITTI Dataset
• Object In Frame
• Affinity Matrix
• Point Cloud Data
• 3D Bounding Box
• Pretext Task
• Bounding Box Annotations
• 3D Point Cloud Data
• 3D Features
• 3D Tracking