GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learning

Grasp detection is a fundamental robotic task critical to the success of many industrial applications. However, current language-driven models for this task often struggle with cluttered images, lengthy textual descriptions, or slow inference speed. We introduce GraspMamba, a new language-driven grasp detection method that employs hierarchical feature fusion with Mamba vision to tackle these challenges. By leveraging rich visual features of the Mamba-based backbone alongside textual information, our approach effectively enhances the fusion of multimodal features. GraspMamba represents the first Mamba-based grasp detection model to extract vision and language features at multiple scales, delivering robust performance and rapid inference time. Intensive experiments show that GraspMamba outperforms recent methods by a clear margin. We validate our approach through real-world robotic experiments, highlighting its fast inference speed.

Authors

• Huy Hoang Nguyen
• An Vuong
• Anh Nguyen 0003
• Ian D. Reid 0001
• Minh Nhat Vu

Keywords

• Representation learning
• Visualization
• Codes
• Accuracy
• Service robots
• Focusing
• Grasping
• Benchmark testing
• Feature extraction
• Intelligent robots
• Hierarchical Feature Learning
• Multiple Scales
• Visual Features
• Textual Information
• Feature Fusion
• Textual Descriptions
• Inference Time
• Inference Speed
• Transformer
• Image Features
• Imaging Modalities
• State Space
• Intersection Over Union
• Point Cloud
• 3D Space
• Global Context
• Target Object
• Language Model
• State-space Model
• Robotic Applications
• Textual Features
• Foundation Model
• Multimodal Representation
• Linear Differential Equations
• Text Encoder
• Instructional Text
• Zero-shot
• Real-world Objects
• Fusion Method