Compositional Servoing by Recombining Demonstrations

Learning-based manipulation policies from image inputs often show weak task transfer capabilities. In contrast, visual servoing methods allow efficient task transfer in high-precision scenarios while requiring only a few demonstrations. In this work, we present a framework that formulates the visual servoing task as graph traversal. Our method not only extends the robustness of visual servoing, but also enables multitask capability based on a few task-specific demonstrations. We construct demonstration graphs by splitting existing demonstrations and recombining them. In order to traverse the demonstration graph in the inference case, we utilize a similarity function that helps select the best demonstration for a specific task. This enables us to compute the shortest path through the graph. Ultimately, we show that recombining demonstrations leads to higher task-respective success. We present extensive simulation and real-world experimental results that demonstrate the efficacy of our approach.

Authors

• Max Argus
• Abhijeet Nayak
• Martin Büchner
• Silvio Galesso
• Abhinav Valada
• Thomas Brox

Keywords

• Integrated optics
• Visualization
• Estimation
• Optical imaging
• Visual servoing
• Robustness
• Planning
• Recombination
• Shortest Path
• Real-world Experiments
• Graph Traversal
• Deep Learning
• Similarity Score
• 3D Reconstruction
• Optical Flow
• Manipulation Tasks
• Goal State
• Target State
• Self-supervised Learning
• Image Synthesis
• Task Success
• Illumination Changes
• Real Robot
• Foreground Objects
• Frame Alignment
• Navigation Problem
• Number Of Demonstrations
• Few-shot Classification