Simultaneous Depth Estimation and Localization for Cell Manipulation Based on Deep Learning

Visual localization, which is a key technology to realize the automation of cell manipulation, has been widely studied. Since the depth of field of the microscope is narrow, the planar localization and depth estimation are usually coupled together. At present, most methods adopt the serial working mode of focusing first and then planar localization, but they usually do not have good real-time performance and stability. In this paper, a simultaneous depth estimation and localization network was developed for cell manipulation. The network takes a focused image and a defocus-offset image as inputs, and outputs the defocus in the depth direction and the offset in the plane at the same time after going through defocus-offset information extraction, defocus classification mapping and offset regression mapping. To train and test our network, we also create two datasets: An Adherent Cell dataset and an Injection Micropipette dataset. The experimental results demonstrated that the proposed method achieves the detection of all test samples with a frame rate of more than 40Hz, and the maximum errors of depth estimation and localization are $\boldsymbol{2. 44\mu m}$ and $\boldsymbol{0. 49\mu m}$, respectively. The proposed method has good stability, which is mainly reflected in its strong generalization ability and anti-noise ability.

Authors

• Zengshuo Wang
• Huiying Gong
• Ke Li 0026
• Bin Yang
• Yue Du
• Yaowei Liu
• Xin Zhao 0010
• Mingzhu Sun

Keywords

• Location awareness
• Training
• Visualization
• Microscopy
• Estimation
• Information retrieval
• Real-time systems
• Deep Learning
• Use Of Cells
• Depth Estimation
• Simultaneous Estimation
• Development Program Of China
• Applied Basic Research Foundation
• Cell Adhesion
• Estimation Error
• Good Stability
• Localization Error
• Depth Direction
• Strong Generalization Ability
• Anti-noise Ability
• Training Set
• Microscopy Images
• Convolutional Neural Network
• Gaussian Noise
• Feature Maps
• Output Data
• Task Loss
• Operational Objectives
• Kinds Of Methods
• Visual Feedback
• Focal Plane
• Convolution Operation
• Template Matching
• Images In Set
• Image Pairs