Mixed reality for robotics

Mixed Reality can be a valuable tool for research and development in robotics. In this work, we refine the definition of Mixed Reality to accommodate seamless interaction between physical and virtual objects in any number of physical or virtual environments. In particular, we show that Mixed Reality can reduce the gap between simulation and implementation by enabling the prototyping of algorithms on a combination of physical and virtual objects, including robots, sensors, and humans. Robots can be enhanced with additional virtual capabilities, or can interact with humans without sharing physical space. We demonstrate Mixed Reality with three representative experiments, each of which highlights the advantages of our approach. We also provide a testbed for Mixed Reality with three different virtual robotics environments in combination with the Crazyflie 2. 0 quadcopter.

Authors

• Wolfgang Hönig
• Christina Milanes
• Lisa Scaria
• Thai Phan
• Mark T. Bolas
• Nora Ayanian

Keywords

• Virtual environments
• Robot sensing systems
• Testing
• Safety
• Mixed Reality
• Virtually
• Research And Development
• Physical Body
• Physical Space
• Virtual Objects
• Combined Objective
• Robotics Research
• Robots In Environments
• Virtual Sensors
• Interactive
• Field Of View
• Technical Details
• Virtual World
• Physical Component
• Motion Capture
• Physical World
• Software Engineering
• Multi-agent Systems
• Motion Capture System
• Shared Environment
• Physical Robot
• Virtual Camera
• Unity 3D
• Swarm Robotics
• Simulated Robot
• Debugging
• Spatial Flexibility
• Physical Ones