Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

The current neuron reconstruction pipeline for electron microscopy (EM) data usually includes automatic image segmentation followed by extensive human expert proofreading. In this work, we aim to reduce human workload by predicting connectivity between over-segmented neuron pieces, taking both microscopy image and 3D morphology features into account, similar to human proofreading workflow. To this end, we first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain, which is three orders of magnitude larger than existing datasets for neuron segment connection. To learn sophisticated biological imaging features from the connectivity annotations, we propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding. The learned embeddings can be easily incorporated with any point or voxel-based morphological representations for automatic neuron tracing. Extensive comparisons of different combination schemes of image and morphological representation in identifying split errors across the whole fly brain demonstrate the superiority of the proposed approach, especially for the locations that contain severe imaging artifacts, such as section missing and misalignment. The dataset and code are available at https://github.com/Levishery/Flywire-Neuron-Tracing.

Authors

• Qihua Chen University of Science and Technology of China Tencent
• Xuejin Chen University of Science and Technology of China Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
• Chenxuan Wang University of Science and Technology of China Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
• Yixiong Liu University of Science and Technology of China Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
• Zhiwei Xiong MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
• Feng Wu

Keywords

• CV: 3D Computer Vision
• CV: Medical and Biological Imaging
• CV: Multi-modal Vision
• CV: Segmentation