Adaptive Modality Selection Algorithm in Robot-Assisted Cognitive Training

Interaction of socially assistive robots with users is based on social cues coming from different interaction modalities, such as speech or gestures. However, using all modalities at all times may be inefficient as it can overload the user with redundant information and increase the task completion time. Additionally, users may favor certain modalities over the other as a result of their disability or personal preference. In this paper, we propose an Adaptive Modality Selection (AMS) algorithm that chooses modalities depending on the state of the user and the environment, as well as user preferences. The variables that describe the environment and the user state are defined as resources, and we posit that modalities are successful if certain resources possess specific values during their use. Besides the resources, the proposed algorithm takes into account user preferences which it learns while interacting with users. We tested our algorithm in simulations, and we implemented it on a robotic system that provides cognitive training, specifically Sequential memory exercises. Experimental results show that it is possible to use only a subset of available modalities without compromising the interaction. Moreover, we see a trend for users to perform better when interacting with a system with implemented AMS algorithm.

Authors

• Aleksandar Taranovic
• Aleksandar Jevtic
• Carme Torras

Keywords

• Shape
• Training
• Service robots
• Manipulators
• Task analysis
• Market research
• Selection Algorithm
• Cognitive Training
• Adaptive Selection
• Robotic System
• Social Cues
• User Preferences
• Robotic Assistance
• Valuable Resource
• Mild Cognitive Impairment
• Multiple Modalities
• Visual Attention
• Use Of Modalities
• Beta Distribution
• Robotic Arm
• User Profile
• Mode Choice
• Set Of Modes
• Multi-armed Bandit
• Baseline System
• Correct Shape
• High Probability Of Success
• Bandit Problem
• Predefined Time Period
• Kinect Camera
• Daycare Facilities
• User Attention
• Low Probability Of Success
• Microphone
• General System
• Physical Resources