Generalizing Object-Centric Task-Axes Controllers using Keypoints

To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. To achieve this it is often infeasible to train monolithic neural network policies across such large variations in object properties. Towards this generalization challenge, we propose to learn modular task policies which compose object-centric task-axes controllers. These task-axes controllers are parameterized by properties associated with underlying objects in the scene. We infer these controller parameters directly from visual input using multi-view dense correspondence learning. Our overall approach provides a simple and yet powerful framework for learning manipulation tasks. We empirically evaluate our approach on 3 different manipulation tasks and show its ability to generalize to large variance in object size, shape and geometry.

Authors

• Mohit Sharma 0001
• Oliver Kroemer

Keywords

• Training
• Geometry
• Visualization
• Automation
• Shape
• Conferences
• Semantics
• Control Parameters
• Visual Input
• Geometric Model
• Object Properties
• Manipulation Tasks
• Objects In The Scene
• Multi-view Learning
• Multiple Ways
• Qualitative Results
• Control Target
• Complex Shapes
• Image Space
• Button Press
• Proportional-integral-derivative
• 3D Position
• Open Door
• Force Control
• Policy Learning
• Learning Control
• Target Parameters
• Door Handle
• Bilevel Optimization
• Proximal Policy Optimization
• Reference Pixels
• High-dimensional Input
• Joint Rotation
• Objective Description
• Challenging Problem
• Reference Image