Augmented Reality Visualizations using Imitation Learning for Collaborative Warehouse Robots

Augmented reality (AR) technologies have been applied to humanrobot collaboration (HRC) domains to enable people to visualize the state of the robots. Current AR-based visualization strategies are manually designed. This design process requires a lot of human efforts, and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too much is visualized, they find it difficult to process the visualized information. In this paper, we develop an intelligent AR agent that learns visualization policies (what to visualize, when, and how) from demonstrations. We developed a Unity-based platform for simulating warehouse environments where human-robot teammates work on collaborative delivery tasks. We have collected a dataset that includes 6000 demonstrations of visualizing robots’ current and planned behaviors. Our results from experiments with real human participants show that, compared with competitive baselines from the literature, our learned visualization strategy significantly increases the efficiency of human-robot teams in delivery tasks.

Authors

• Kishan Chandan
• Jack Albertson SUNY Binghamton
• Shiqi Zhang Zhejiang University of Technology

Keywords

• Augmented Reality
• Multi-agent systems
• Imitation Learning