Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement

Our goal is to develop an AI Partner that can provide support for group problem solving and social dynamics. In multi-party working group environments, multimodal analytics is crucial for identifying non-verbal interactions of group members. In conjunction with their verbal participation, this creates an holistic understanding of collaboration and engagement that provides necessary context for the AI Partner. In this demo, we illustrate our present capabilities at detecting and tracking nonverbal behavior in student task-oriented interactions in the classroom, and the implications for tracking common ground and engagement.

Authors

• Derek Palmer Brandeis University
• Yifan Zhu Beijing University of Posts and Telecommunications
• Kenneth Lai Brandeis University
• Hannah VanderHoeven Colorado State University
• Mariah Bradford Colorado State University
• Ibrahim Khebour Colorado State University
• Carlos Mabrey Colorado State University
• Jack Fitzgerald Colorado State University
• Nikhil Krishnaswamy Colorado State University
• Martha Palmer University of Colorado at Boulder
• James Pustejovsky Brandeis University

Keywords

No keywords are indexed for this paper.