Object recognition using visuo-affordance maps

One of the major challenges in developing autonomous systems is to make them able to recognize and categorize objects robustly. However, the appearance-based algorithms that are widely employed for robot perception do not explore the functionality of objects, described in terms of their affordances. These affordances (e. g. , manipulation, grasping) are discriminative for object categories and are important cues for reliable robot performance in everyday environments. In this paper, we propose a strategy for object recognition that integrates both visual appearance and grasp affordance features. Following previous work, we hypothesize that additional grasp information improves object recognition, even if we reconstruct the grasp modality from visual features using a mapping function. We considered two different representations for the grasp modality: (1) motor information of the hand posture while grasping and (2) a more general grasp affordance descriptor. Using a multi-modal classifier we show that having real grasp information significantly boost object recognition. This improvement is preserved, although to a lesser extent, if the grasp modality is reconstructed using the mapping function.

Authors

• Arjan Gijsberts
• Tatiana Tommasi
• Giorgio Metta
• Barbara Caputo

Keywords

• Visualization
• Training
• Object recognition
• Image reconstruction
• Kernel
• Grasping
• Robots
• Visual Features
• Hand Position
• Motor Information
• Hyperparameters
• Data Visualization
• Probabilistic Model
• Kernel Function
• Object Classification
• Object Of Interest
• Radial Basis Function Kernel
• Motion Features
• Object Identification
• Reconstruction Quality
• Real Information
• Object Appearance
• Regularized Least Squares
• Reproducing Kernel Hilbert Space
• Real Motion
• Motion In Space