Author name cluster

Kim Baraka

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

AAMAS Conference 2025 Conference Paper

Can you see how I learn? Human Observers' Inferences about Reinforcement Learning Agents' Learning Processes

Bernhard Hilpert
Muhan Hou
Kim Baraka
Joost Broekens

Human-in-the-loop Reinforcement Learning (RL) often suffers from suboptimal human teaching signals. Yet, how humans perceive and interpret RL agent’s learning behavior is largely unknown. In a bottom-up approach with two experiments, this work provides a data-driven understanding of the factors in RL agents’ behavior that influence the understanding of the agent’s learning process for human observers. In two consecutive experiments with two different RL agents (a tabular and function approximation agent in a navigation and a manipulation task), human observations of agent learning behavior was assessed and systematically analyzed. Four common emerging themes were observed: Agent Goals, Knowledge, Decision Making and Learning Mechanisms, each with specific subclusters, offering insights for transparency in RL and HRI.

PDF

ICRA Conference 2025 Conference Paper

Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approach

Muhan Hou
Koen V. Hindriks
A. E. Eiben
Kim Baraka

Transfer Learning (TL) is a powerful tool that enables robots to transfer learned policies across different environments, tasks, or embodiments. To further facilitate this process, efforts have been made to combine it with Learning from Demonstrations (LfD) for more flexible and efficient policy transfer. However, these approaches are almost exclusively limited to offline demonstrations collected before policy transfer starts, which may suffer from the intrinsic issue of covariance shift brought by LfD and harm the performance of policy transfer. Meanwhile, extensive work in the learning-from-scratch setting has shown that online demonstrations can effectively alleviate covariance shift and lead to better policy performance with improved sample efficiency. This work combines these insights to introduce online demonstrations into a policy transfer setting. We present Policy Transfer with Online Demonstrations, an active LfD algorithm for policy transfer that can optimize the timing and content of queries for online episodic expert demonstrations under a limited demonstration budget. We evaluate our method in eight robotic scenarios, involving policy transfer across diverse environment characteristics, task objectives, and robotic embodiments, with the aim to transfer a trained policy from a source task to a related but different target task. The results show that our method significantly outperforms all baselines in terms of average success rate and sample efficiency, compared to two canonical LfD methods with offline demonstrations and one active LfD method with online demonstrations. Additionally, we conduct preliminary sim-to-real tests of the transferred policy on three transfer scenarios in the real-world environment, demonstrating the policy effectiveness on a real robot manipulator.

Details

ICRA Conference 2023 Conference Paper

KRIS: A Novel Device for Kinesthetic Corrective Feedback during Robot Motion

Jorn Verheggen
Kim Baraka

This paper presents a novel device that can be used to perform kinesthetic corrective feedback for robotic systems. KRIS (Kinesthetic Robotic Interaction System) is a device that can be mounted on the end-effector of an articulated robot. From here it can be manipulated by a human to give corrective feedback to the robot system during execution and in an intuitive way. The device can provide feedback in six degrees of freedom while giving passive haptic feedback to the user about both the position, rotation, and movement of the robot. We evaluated KRIS in a user study with respect to a baseline based on keyboard feedback in the areas of usability, intuitiveness, accuracy of corrections, and user task load. KRIS outperformed our baseline on the first three metrics and performed similar on task load. We believe that KRIS can enable a wide variety of robots to be taught interactively by non-expert humans in diverse collaborative settings.

Details

JAAMAS Journal 2020 Journal Article

Optimal action sequence generation for assistive agents in fixed horizon tasks

Kim Baraka
Francisco S. Melo
Manuela Veloso

Abstract Agents providing assistance to humans are faced with the challenge of automatically adjusting the level of assistance to ensure optimal performance. In this work, we argue that identifying the right level of assistance consists in balancing positive assistance outcomes and some (domain-dependent) measure of cost associated with assistive actions. Towards this goal, we contribute a general mathematical framework for structured tasks where an agent playing the role of a ‘provider’—e. g. , therapist, teacher—assists a human ‘receiver’—e. g. , patient, student. We specifically consider tasks where the provider agent needs to plan a sequence of actions over a fixed time horizon, where actions are organized along a hierarchy with increasing success probabilities, and some associated costs. The goal of the provider is to achieve a success with the lowest expected cost possible. We present OAssistMe, an algorithm that generates cost-optimal action sequences given the action parameters, and investigate several extensions of it, motivated by different potential application domains. We provide an analysis of the algorithms, including proofs for a number of properties of optimal solutions that, we show, align with typical human provider strategies. Finally, we instantiate our theoretical framework in the context of robot-assisted therapy tasks for children with Autism Spectrum Disorder (ASD). In this context, we present methods for determining action parameters based on a survey of domain experts and real child-robot interaction data. Our contributions unlock increased levels of flexibility for agents introduced in a variety of assistive contexts.

Details DOI

AAMAS Conference 2019 Conference Paper

An Optimization Approach for Structured Agent-Based Provider/Receiver Tasks

Kim Baraka
Marta Couto
Francisco S. Melo
Manuela Veloso

This work contributes an optimization framework in the context of structured interactions between an agent playing the role of a ‘provider’ and a human ‘receiver’. Examples of provider/receiver interactions of interest include ones between occupational therapist and patient, or teacher and student. We specifically consider tasks where the provider agent needs to plan a sequence of actions with a fixed horizon, where actions are organized along a hierarchy with increasing probabilities of success and associated costs. The goal of the provider is to achieve a success with the lowest expected cost possible. In our application domains, a success may be for instance eliciting a desired behavior or a correct response from the receiver. We present a linear-time optimal planning algorithm that generates cost-optimal sequences for given action parameters. We also provide proofs for a number of properties of optimal solutions that align with typical human provider strategies. Finally, we instantiate our general formulation in the context of robot-assisted therapy tasks for children with Autism Spectrum Disorders (ASD). In this context, we present methods for determining action parameters, namely (1) an online survey with experts for determining action costs, and (2) a probabilistic model of child response based on data collected in a real child-robot interaction scenario. Our contributions may unlock increased levels of adaptivity for agents introduced in a variety of assistive contexts.

PDF