Perceiving Unseen 3D Objects by Poking the Objects

We present a novel approach to interactive 3D object perception for robots. Unlike previous perception algorithms that rely on known object models or a large amount of annotated training data, we propose a poking-based approach that automatically discovers and reconstructs 3D objects. The poking process not only enables the robot to discover unseen 3D objects but also produces multi-view observations for 3D reconstruction of the objects. The reconstructed objects are then memorized by neural networks with regular supervised learning and can be recognized in new test images. The experiments on real-world data show that our approach could unsupervisedly discover and reconstruct unseen 3D objects with high quality, and facilitate real-world applications such as robotic grasping. The code and supplementary materials are available at the project page: https://zju3dv.github.io/poking_perception/.

Authors

• Linghao Chen
• Yunzhou Song
• Hujun Bao
• Xiaowei Zhou 0001

Keywords

• Solid modeling
• Three-dimensional displays
• Neural networks
• Supervised learning
• Training data
• Grasping
• Object recognition
• Unseen Objects
• Neural Network
• 3D Reconstruction
• Real-world Applications
• Object Reconstruction
• Amount Of Annotated Data
• Object Detection
• Point Cloud
• Multilayer Perceptron
• Robotic Arm
• 3D Point
• Pose Estimation
• Joint Optimization
• Objects In The Scene
• Scene Understanding
• Pixel Color
• Human Pose Estimation
• Foreground Objects
• Iterative Closest Point
• Object Pose
• Neural Field
• Object Proposals
• Depth Point
• View Direction
• Signed Distance Function
• Relative Pose
• Object Regions
• Object Motion
• Depth Images