Hand Pose Estimation for Hand-Object Interaction Cases using Augmented Autoencoder

Hand pose estimation with objects is challenging due to object occlusion and the lack of large annotated datasets. To tackle these issues, we propose an Augmented Autoencoder based deep learning method using augmented clean hand data. Our method takes 3D point cloud of a hand with an augmented object as input and encodes the input to latent representation of the hand. From the latent representation, our method decodes 3D hand pose and we propose to use an auxiliary point cloud decoder to assist the formation of the latent space. Through quantitative and qualitative evaluation on both synthetic dataset and real captured data containing objects, we demonstrate state-of-the-art performance for hand pose estimation with objects, even using only a small number of annotated hand-object samples.

Authors

• Shile Li
• Haojie Wang
• Dongheui Lee

Keywords

• Three-dimensional displays
• Pose estimation
• Decoding
• Image reconstruction
• Task analysis
• Shape
• Feature extraction
• Hand Pose
• Hand-object Interaction
• Hand Pose Estimation
• Augmented Autoencoder
• Deep Learning
• Data Augmentation
• Point Cloud
• 3D Point
• Latent Representation
• 3D Point Cloud
• 3D Pose
• Occluded Objects
• Hand Representation
• Clean Hands
• Training Dataset
• Single Image
• Qualitative Results
• Large-scale Datasets
• Latent Vector
• Clean Samples
• Earth Mover’s Distance
• Augmentation Process
• Chamfer Distance
• Input Point
• Human Pose Estimation
• Variational Autoencoder
• Depth Images
• Earth Mover