Occlusion-Aware Search for Object Retrieval in Clutter

We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop. The hybrid planner explores likely locations of the occluded target object as predicted by a learned distribution from the observation stream. The search is guided by a heuristic trained with reinforcement learning to act on observations with occlusions. We evaluate our approach in different simulation and real-world settings (video available on https://youtu.be/dY7YQ3LUVQg).The results validate that our approach can search and retrieve a target object in near real time in the real world while only being trained in simulation.

Authors

• Wissam Bejjani
• Wisdom C. Agboh
• Mehmet Remzi Dogar
• Matteo Leonetti

Keywords

• Three-dimensional displays
• Reinforcement learning
• Streaming media
• Search problems
• Transformers
• Solids
• Cognition
• Heuristic
• Target Object
• Neural Network
• Convolutional Neural Network
• Value Function
• Sequence Of Actions
• Long Short-term Memory
• Simulation Environment
• Continuous Action
• Object Shape
• Recurrent Unit
• Physical Simulation
• Linear Activation Function
• Object Pose
• Model-free Approach
• Execution Environment
• Top-down View
• Camera Field Of View
• Greedy Policy
• Stochastic Policy
• High-level Planner
• Target Pose
• Current Observations
• Representative Observations
• High Noise Levels
• Number Of Objects
• Reward Function
• Task Parameters
• Horizon
• Planning Time