FlowControl: Optical Flow Based Visual Servoing

One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a specified foreground mask to attend to an object of interest. Using RGB-D observations, FlowControl requires no 3D object models, and is easy to set up. FlowControl inherits great robustness to visual appearance from decades of work in optical flow. We exhibit FlowControl on a range of problems, including ones requiring very precise motions, and ones requiring the ability to generalize.

Authors

• Max Argus
• Lukás Hermann
• Jon Long
• Thomas Brox

Keywords

• Visualization
• Solid modeling
• Three-dimensional displays
• Streaming media
• Visual servoing
• Task analysis
• Optical flow
• Manipulation Tasks
• Video Presentation
• Learning Algorithms
• Environmental Variables
• Optimal Control
• Point Cloud
• Target Image
• Supplementary Video
• Pose Estimation
• End-effector
• Image Feature Extraction
• Variable Geometry
• Wood Blocks
• Camera Pose
• Partial Occlusion
• Flow Algorithm
• Relative Pose
• Target Frame
• Optical Flow Method
• Imitation Learning
• Scene Geometry
• Policy Learning
• Optical Flow Algorithm
• Appearance Variations
• Depth Camera
• Flow Estimation
• Optical Flow Estimation