End-User Robot Programming Using Mixed Reality

Mixed Reality (MR) is a promising interface for robot programming because it can project an immersive 3D visualization of a robot's intended movement onto the real world. MR can also support hand gestures, which provide an intuitive way for users to construct and modify robot motions. We present a Mixed Reality Head-Mounted Display (MRHMD) interface that enables end-users to easily create and edit robot motions using waypoints. We describe a user study where 20 participants were asked to program a robot arm using 2D and MR interfaces to perform two pick-and-place tasks. In the primitive task, participants created typical pickand-place programs. In the adapted task, participants adapted their primitive programs to address a more complex pickand-place scenario, which included obstacles and conditional reasoning. Compared to the 2D interface, a higher number of users were able to complete both tasks in significantly less time, and reported experiencing lower cognitive workload, higher usability, and higher naturalness with the MR-HMD interface.

Authors

• Samir Yitzhak Gadre
• Eric Rosen
• Gary Chien
• Elizabeth Phillips
• Stefanie Tellex
• George Konidaris 0001

Keywords

• Task analysis
• Visualization
• Two dimensional displays
• Virtual reality
• Programming
• Robot sensing systems
• Mixed Reality
• Robot Programming
• High Use
• Cognitive Load
• User Study
• Application Programming Interface
• Robotic Arm
• Robot Motion
• Hand Gestures
• Low Workload
• Task Completion
• Workspace
• Graphical User Interface
• Welding
• Path Planning
• Physical Demands
• Hologram
• 7-point Likert-type Scale
• Higher Completion Rates
• Virtual Reality Headset
• NASA Task Load Index
• Task Completion Time
• Inverse Reinforcement Learning
• Visual Programming
• System Usability Scale
• Robot Trajectory
• Real Robot
• Motion Primitives