Object-Oriented Option Framework for Robotics Manipulation in Clutter

Domestic service robots are becoming increasingly popular due to their ability to help people with household tasks. These robots often encounter the challenge of manipulating objects in cluttered environments (MoC), which is difficult due to the complexity of effective planning and control. Previous solutions involved designing specific action primitives and planning paradigms. However, the pre-coded action primitives can limit the agility and task-solving scope of robots. In this paper, we propose a general approach for MoC called the Object-Oriented Option Framework (O3F), which uses the option framework (OF) to learn planning and control. The standard OF discovers options from scratch based on reinforcement learning, which can lead to collapsed options and hurt learning. To address this limitation, O3F introduces the concept of an object-oriented option space for OF, which focuses specifically on object movement and overcomes the challenges associated with collapsed options. Based on this, we train an object-oriented option planner to determine the option to execute and a universal object-oriented option executor to complete the option. Simulation experiments on the Ginger XR1 robot and robot arm show that O3F is generally applicable to various types of robot and manipulation tasks. Furthermore, O3F achieves success rates of 72. 4% and 90% in grasping and object collecting tasks, respectively, significantly outperforming baseline methods.

Authors

• Jing-Cheng Pang
• Si-Hang Yang
• Xiong-Hui Chen
• Xinyu Yang
• Yang Yu 0001
• Mas Ma
• Ziqi Guo
• Howard Yang
• Bill Huang

Keywords

• Service robots
• Reinforcement learning
• Grasping
• Manipulators
• Planning
• Complexity theory
• Task analysis
• Robot Manipulator
• Object-oriented Framework
• Options Framework
• Manipulation Tasks
• Robotic Arm
• Object Motion
• Domestic Service
• Types Of Robots
• Time Step
• Rest Of The Paper
• State Space
• Target Location
• Workbench
• Training Procedure
• Number Of Objects
• Target Object
• Object Position
• Reward Function
• Markov Decision Process
• Reinforcement Learning Algorithm
• Proximal Policy Optimization
• Maximum Step
• Tensor Elements
• Joint State
• Standard Reinforcement Learning
• Conduct Simulation Experiments
• Reinforcement Learning Task
• Terminal Function