Measuring fingertip forces from camera images for random finger poses

Robust fingertip force detection from fingernail image is a critical strategy that can be applied in many areas. However, prior research fixed many variables that influence the finger color change. This paper analyzes the effect of the finger joint on the force detection in order to deal with the constrained finger position setting. A force estimator method is designed: a model to predict the fingertip force from finger joints measured from 2D cameras and 3 rectangular markers in cooperation with the fingernail images are trained. Then the error caused by the color changes of the joint bending can be avoided. This strategy is a significant step forward from a finger force estimator that requires tedious finger joint setting. The approach is evaluated experimentally. The result shows that it increases the accuracy over 10% for the force in conditions of the finger joint free movement. The estimator is used to demonstrate lifting and replacing objects with various weights.

Authors

• Nutan Chen
• Sebastian Urban
• Justin Bayer
• Patrick van der Smagt

Keywords

• Force
• Cameras
• Training
• Image color analysis
• Nails
• Kernel
• Estimation
• Fingertip Force
• Color Change
• Force Estimation
• Finger Joints
• Detection In Order
• Training Set
• Training Data
• Convolutional Neural Network
• Hyperparameters
• Test Dataset
• Gaussian Process
• Kriging
• Joint Angles
• Contact Force
• White Areas
• Force Vector
• Covariance Function
• Human-robot Interaction
• Proximal Interphalangeal
• Distal Interphalangeal Joint
• Force Prediction
• Interphalangeal Joints
• Sum Of Squared Differences
• Extensor Tendon
• Channel Images