Learning to Catch Reactive Objects with a Behavior Predictor

Tracking and catching moving objects is an important ability for robots in a dynamic world. Whilst some objects have highly predictable state evolution e. g. , the ballistic trajectory of a tennis ball, reactive targets alter their behavior in response to motion of the manipulator. Reactive applications range from gently capturing living animals such as snakes or fish for biological investigations, to smoothly interacting with and assisting a person. Existing works for dynamic catching usually perform target prediction followed by planning, but seldom account for highly non-linear reactive behaviors. Alternatively, Reinforcement Learning (RL) based methods simply treat the target and its motion as part of the observation of the world-state, but perform poorly due to the weak reward signal. In this work, we blend the approach of an explicit, yet learned, target state predictor with RL. We further show how a tightly coupled predictor which ‘observes’ the state of the robot leads to significantly improved anticipatory action, especially with targets that seek to evade the robot following a simple policy. Experiments show that our method achieves an 86. 4% (open plane area) and a 73. 8% (room) success rate on evasive objects, outperforming monolithic reinforcement learning and other techniques. We also demonstrate the efficacy of our approach across varied targets and trajectories. All code, data, and additional videos are at this GitHub link: https://kl-research.github.io/dyncatch.

Authors

• Kai Lu 0003
• Jia-Xing Zhong
• Bo Yang 0027
• Bing Wang 0013
• Andrew Markham

Keywords

• Target tracking
• Sports equipment
• Dynamics
• Reinforcement learning
• Trajectory
• Planning
• Task analysis
• Predictor Of Behavior
• Behavioral Reactions
• Target State
• Robot State
• Ballistic Trajectory
• Robotic Arm
• Object Position
• Reward Function
• Markov Decision Process
• Learning Prediction
• Policy Learning
• Yaw Angle
• Reinforcement Learning Methods
• Dynamic Objects
• Reinforcement Learning Model
• Room Setting
• Average Success Rate
• Highest Success Rate
• High-level Policy
• Uneven Terrain
• Mobile Manipulator
• Quadruped Robot
• Reinforcement Learning Policy
• Proximal Policy Optimization
• Linear Parameter Varying
• Mobile Robot
• Multilayer Perceptron
• Distance Vector
• State Space
• Low-level Control