Grasp Manipulation Relationship Detection based on Graph Sample and Aggregation

In multi-object stacking scenarios, exploring the relationships among objects and determining the correct sequence of operations are crucial for robotic manipulation. However, previous algorithms inefficiently combine global and local information, often focusing solely on the local features of objects or the interactions of object features at a global level. This approach leads to imbalanced distribution of features and the generation of redundant or missing relationships in complex scenes, such as multi-object stacking and partial occlusion. To address this issue, we have developed a grasp manipulation relationship detection algorithm called Graph Sampling Aggregation Network for Visual Manipulation Relationship Detection (GSAGED). This algorithm assists robots in detecting targets in complex scenes and determining the appropriate grasping order. Firstly, the Positional Encoding Module in GSAGED enhances object feature information by considering global contexts. Secondly, the Graph Sampling Aggregation method effectively integrates global and local information, relieving imbalanced distribution of features. Finally, we applied the developed algorithm to a physical robot for grasping. Experimental results on the Visual Manipulation Relationship Dataset (VMRD) and the large-scale relational grasp dataset named REGRAD demonstrate that our method significantly improves the accuracy of relationship detection in complex scenes and exhibits robust generalization capabilities in real-world applications.

Authors

• Jiayuan Luo
• Yaxin Liu
• Han Wang
• Mengyuan Ding
• Xuguang Lan

Keywords

• Visualization
• Accuracy
• Stacking
• Focusing
• Grasping
• Encoding
• Detection algorithms
• Local Information
• Global Information
• Object Features
• Visual Detection
• Robot Manipulator
• Positional Encoding
• Visual Relationship
• Call Graph
• Low Accuracy
• Performance Metrics
• Related Features
• Object Detection
• Multilayer Perceptron
• Stochastic Gradient Descent
• Bounding Box
• Average Precision
• Backbone Network
• Graph Convolutional Network
• Graph Neural Networks
• Objects In The Scene
• Scene Features
• Object Pairs
• Scene Information
• Local Graph
• Conditional Random Field
• Global Context Information
• Aggregation Function
• Real-world Scenes
• RGB Images
• Root Node