Fast Explicit-Input Assistance for Teleoperation in Clutter

The performance of prediction-based assistance for robot teleoperation degrades in unseen or goal-rich environments due to incorrect or quickly-changing intent inferences. Poor predictions can confuse operators or cause them to change their control input to implicitly signal their goal. We present a new assistance interface for robotic manipulation where an operator can explicitly communicate a manipulation goal by pointing the end-effector. The pointing target specifies a region for local pose generation and optimization, providing interactive control over grasp and placement pose candidates. We evaluate this explicit pointing interface against an implicit inference-based assistance scheme and an unassisted control condition in a within-subjects user study (N=20), where participants teleoperate a simulated robot to complete a multi-step singulation and stacking task in cluttered environments. We find that operators prefer the explicit interface, experience fewer pick failures and report lower cognitive workload. Our code is available at: github.com/NVlabs/fast-explicit-teleop.

Authors

• Nick Walker 0001
• Xuning Yang
• Animesh Garg
• Maya Cakmak
• Dieter Fox
• Claudia Pérez-D'Arpino

Keywords

• Codes
• Stacking
• Grasping
• End effectors
• Clutter
• Optimization
• Intelligent robots
• Interactive
• Control Input
• Local Optimum
• Poor Predictor
• End-effector
• Low-pass
• Geodesic
• Point Scale
• Robotic Arm
• User Input
• Robot Control
• Assistance Systems
• Collision Detection
• Real Robot
• Reference Vector
• Scene Geometry
• Current Orientation
• Current Pose
• End-effector Pose