TPR: A Training Procedure Representation to Augment XR Simulations with LLMs

Extended reality (XR) is well suited to support the situated learning of technical procedures. At the same time, AI-driven intelligent tutoring systems (ITS) can complement XR by providing adaptive pedagogical support. Many domains would benefit from this combination, especially when trainers, equipment, or team members are limited. We present a domain-agnostic XR-based ITS that integrates a training procedure representation (TPR), XR simulation, and an LLM-driven instructor. We demonstrate the tutor's use for tissue sample handling and engine repair, showing how it delivers adaptive feedback, collaborative roleplay, and dynamic scenario management to create realistic and pedagogically meaningful training experiences.

Authors

• Michael Guevarra Illumia Labs
• Christabel Wayllace New Mexico State University
• Srijita Das University of Michigan - Dearborn
• Carrie Demmans Epp University of Alberta
• Alan Tay Illumia Labs

Keywords

No keywords are indexed for this paper.