Avoiding Object Damage in Robotic Manipulation

The large-scale deployment of robotic manipulation systems in warehouses has highlighted the rare but costly problem of robot-induced object damage. We present a system that uses a classification model to predict whether an object will get damaged during robotic manipulation. The model uses object attributes retrieved from warehouse information systems as well as attributes available at our robotic workcell. We evaluated different classical machine learning models, as well as a large language model (BERT) and a multimodal-transformer for our task. We show that the multi-modal transformer model that is able to leverage text and image data outperforms models that only rely on categorical and numerical data. Furthermore, our comparative analysis equips the selection the optimal model for an application. We validate our system during an experiment in which the output of the damage prediction system is used to avoid picking objects that are likely to get damaged. In over 50k pick-and-place activities, our system reduces damage rate by 64%.

Authors

• Erica Aduh
• Fan Wang
• Dylan Randle
• Kaiwen Wang
• Priyesh Shah
• Chaitanya Mitash
• Manikantan Nambi

Keywords

• Analytical models
• Large language models
• Machine learning
• Predictive models
• Transformers
• Data models
• Numerical models
• Intelligent robots
• Information systems
• Robot Manipulator
• Imaging Data
• Machine Learning Models
• Numerical Data
• Text Data
• Robotic System
• Language Model
• Object Properties
• Transformer Model
• Damage Rate
• Machine Learning Classification Models
• Convolutional Neural Network
• False Positive Rate
• Time Series Data
• Image Object
• Raw Images
• Deconstruction
• Gradient Boosting
• Recall Rate
• ML Models
• CatBoost
• Autoregressive Integrated Moving Average
• Vision Transformer
• Likelihood Of Damage
• Categorical Attributes
• Actual Proportion
• Predictive Maintenance
• BERT Model
• Low False Positive Rate