Reuth Mirsky

AAMAS Conference 2026 Conference Paper

General Dynamic Goal Recognition using Goal-Conditioned and Meta Reinforcement Learning

• Osher Elhadad
• Owen Morrissey
• Reuth Mirsky

Understanding an agent’s goal through its behavior is a common AI problem called Goal Recognition (GR). This task becomes particularly challenging in dynamic environments where goals are numerousandever-changing. Weintroducethe GeneralDynamic Goal Recognition (GDGR) problem, a broader definition of GR aimed at real-time adaptation of GR systems. This paper presents two novel approaches to tackle GDGR: (1) GC-AURA, generalizing to new goals using Goal-Conditioned Reinforcement Learning, and (2) Meta-AURA, adapting to novel environments with Meta- Reinforcement Learning. We evaluate these methods across diverse environments, demonstrating their ability to achieve rapid adaptation and high GR accuracy under dynamic and noisy conditions. This work is a significant step forward in enabling GR in dynamic and unpredictable real-world environments.

AAMAS Conference 2026 Conference Paper

GRAIL: Goal Recognition Alignment through Imitation Learning

• Osher Elhadad
• Felipe Meneguzzi
• Reuth Mirsky

Aligning AI systems with human intentions requires understanding an agent’s goals from its behavior. Existing goal recognition methods typically rely on an approximately optimal goal-oriented policy representation, which may differ from the actor’s true behavior and hinder the accurate recognition of their goal. To address this gap, this paper introduces Goal Recognition Alignment through ImitationLearning(GRAIL), whichleveragesimitationlearningand inverse reinforcement learning to learn one goal-directed policy for each candidate goal directly from (potentially suboptimal) demonstration trajectories. By scoring an observed partial trajectory with each learned goal-directed policy in a single forward pass, GRAIL retains the one-shot inference capability of classical goal recognition while leveraging learned policies that can capture suboptimal and systematically biased behavior. Empirical evaluations show that GRAIL outperforms standard reinforcement learning-based GRtechniquesinrecognizingbothsuboptimalandbiasedbehaviors in controlled, closed-set goal environments.

AAAI Conference 2025 Conference Paper

Bad AI, Good AI: Rethinking the Agency of Our Artificial Teammates

• Reuth Mirsky

A prevalent assumption in human-robot and human-AI teaming is that artificial teammates should be compliant and obedient. In this talk, I will question this assumption by presenting the Guide Robot Grand Challenge and discussing the components required to design and build a service robot that can intelligently disobey. This challenge encompasses a variety of research problems, as I will exemplify via three challenges: reasoning about the goals of other agents, choosing when to interrupt, and interacting in a tightly coupled physical environment.

IJCAI Conference 2025 Conference Paper

GRAML: Goal Recognition As Metric Learning

• Matan Shamir
• Reuth Mirsky

Goal Recognition (GR) is the problem of recognizing an agent's objectives based on observed actions. Recent data-driven approaches for GR alleviate the need for costly, manually crafted domain models. However, these approaches can only reason about a pre-defined set of goals, and time-consuming training is needed for new emerging goals. To keep this model-learning automated while enabling quick adaptation to new goals, this paper introduces GRAML: Goal Recognition As Metric Learning. GRAML frames GR as a deep metric learning problem, using a Siamese network composed of recurrent units to learn an embedding space where traces leading to the same goal are close, and those leading to different goals are distant. This metric is particularly effective for adapting to new goals, even when only a single example trace is available per goal. Evaluated on a versatile set of environments, GRAML shows speed, flexibility, and runtime improvements over the state-of-the-art GR while maintaining accurate recognition.

IJCAI Conference 2024 Conference Paper

A Survey on Model-Free Goal Recognition

• Leonardo Amado
• Sveta Paster Shainkopf
• Ramon Fraga Pereira
• Reuth Mirsky
• Felipe Meneguzzi

Goal Recognition is the task of inferring an agent's intentions from a set of observations. Existing recognition approaches have made considerable advances in domains such as human-robot interaction, intelligent tutoring systems, and surveillance. However, most approaches rely on explicit domain knowledge, often defined by a domain expert. Much recent research focus on mitigating the need for a domain expert while maintaining the ability to perform quality recognition, leading researchers to explore Model-Free Goal Recognition approaches. We comprehensively survey Model-Free Goal Recognition, and provide a perspective on the state-of-the-art approaches and their applications, showing recent advances. We categorize different approaches, introducing a taxonomy with a focus on their characteristics, strengths, weaknesses, and suitability for different scenarios. We compare the advances each approach made to the state-of-the-art and provide a direction for future research in Model-Free Goal Recognition.

AAMAS Conference 2024 Conference Paper

Emergent Dominance Hierarchies in Reinforcement Learning Agents

• Ram Rachum
• Yonatan Nakar
• Bill Tomlinson
• Nitay Alon
• Reuth Mirsky

Modern Reinforcement Learning (RL) algorithms are able to outperform humans in a wide variety of tasks. Multi-agent reinforcement learning (MARL) settings present additional challenges around cooperation in mixed-motive groups. Social conventions and norms, often inspired by human institutions, are used as tools for striking the balance between individual and group objectives. We examine a fundamental social convention that underlies cooperation in animal and human societies: dominance hierarchies. We adapt the ethological theory of dominance hierarchies to artificial agents, borrowing established terminology and definitions. We provide an environment we call Chicken Coop, and we demonstrate that populations of RL agents in that environment can invent, learn, enforce, and transmit a dominance hierarchy to new populations. The dominance hierarchies that emerge in it have a similar structure to those studied in chickens, mice, fish, and other species.

IROS Conference 2023 Conference Paper

A Novel Control Law for Multi-Joint Human-Robot Interaction Tasks While Maintaining Postural Coordination

• Keya Ghonasgi
• Reuth Mirsky
• Adrian M. Haith
• Peter Stone 0001
• Ashish D. Deshpande

Exoskeleton robots are capable of safe torque-controlled interactions with a wearer while moving their limbs through predefined trajectories. However, affecting and assisting the wearer's movements while incorporating their inputs (effort and movements) effectively during an interaction re-mains an open problem due to the complex and variable nature of human motion. In this paper, we present a control algorithm that leverages task-specific movement behaviors to control robot torques during unstructured interactions by implementing a force field that imposes a desired joint angle coordination behavior. This control law, built by using principal component analysis (PCA), is implemented and tested with the Harmony exoskeleton. We show that the proposed control law is versatile enough to allow for the imposition of different coordination behaviors with varying levels of impedance stiffness. We also test the feasibility of our method for unstructured human-robot interaction. Specifically, we demonstrate that participants in a human-subject experiment are able to effectively perform reaching tasks while the exoskeleton imposes the desired joint coordination under different movement speeds and interaction modes. Survey results further suggest that the proposed control law may offer a reduction in cognitive or motor effort. This control law opens up the possibility of using the exoskeleton for training the participating in accomplishing complex multi-joint motor tasks while maintaining postural coordination.

EUMAS Conference 2022 Conference Paper

A Survey of Ad Hoc Teamwork Research

• Reuth Mirsky
• Ignacio Carlucho
• Arrasy Rahman
• Elliot Fosong
• William Macke
• Mohan Sridharan
• Peter Stone 0001
• Stefano V. Albrecht

Abstract Ad hoc teamwork is the research problem of designing agents that can collaborate with new teammates without prior coordination. This survey makes a two-fold contribution: First, it provides a structured description of the different facets of the ad hoc teamwork problem. Second, it discusses the progress that has been made in the field so far, and identifies the immediate and long-term open problems that need to be addressed in ad hoc teamwork.

AAAI Conference 2022 Conference Paper

Goal Recognition as Reinforcement Learning

• Leonardo Amado
• Reuth Mirsky
• Felipe Meneguzzi

Most approaches for goal recognition rely on specifications of the possible dynamics of the actor in the environment when pursuing a goal. These specifications suffer from two key issues. First, encoding these dynamics requires careful design by a domain expert, which is often not robust to noise at recognition time. Second, existing approaches often need costly real-time computations to reason about the likelihood of each potential goal. In this paper, we develop a framework that combines model-free reinforcement learning and goal recognition to alleviate the need for careful, manual domain design, and the need for costly online executions. This framework consists of two main stages: offline learning of policies or utility functions for each potential goal, and online inference. We provide a first instance of this framework using tabular Q-learning for the learning stage, as well as three mechanisms for the inference stage. The resulting instantiation achieves state-of-the-art performance against goal recognizers on standard evaluation domains and superior performance in noisy environments.

IROS Conference 2022 Conference Paper

Quantifying Changes in Kinematic Behavior of a Human-Exoskeleton Interactive System

• Keya Ghonasgi
• Reuth Mirsky
• Adrian M. Haith
• Peter Stone 0001
• Ashish D. Deshpande

While human-robot interaction studies are becoming more common, quantification of the effects of repeated interaction with an exoskeleton remains unexplored. We draw upon existing literature in human skill assessment and present extrinsic and intrinsic performance metrics that quantify how the human-exoskeleton system's behavior changes over time. Specifically, in this paper, we present a new performance metric that provides insight into the system's kinematics associated with ‘successful’ movements resulting in a richer characterization of changes in the system's behavior. A human subject study is carried out wherein participants learn to play a challenging and dynamic reaching game over multiple attempts, while donning an upper-body exoskeleton. The results demonstrate that repeated practice results in learning over time as identified through the improvement of extrinsic performance. Changes in the newly developed kinematics-based measure further illumi-nate how the participant's intrinsic behavior is altered over the training period. Thus, we are able to quantify the changes in the human-exoskeleton system's behavior observed in relation with learning.

IROS Conference 2021 Conference Paper

Capturing Skill State in Curriculum Learning for Human Skill Acquisition

• Keya Ghonasgi
• Reuth Mirsky
• Sanmit Narvekar
• Bharath Masetty
• Adrian M. Haith
• Peter Stone 0001
• Ashish D. Deshpande

Humans learn complex motor skills with practice and training. Though the learning process is not fully understood, several theories from motor learning, neuroscience, education, and game design suggest that curriculum-based training may be the key to efficient skill acquisition. However, designing such a curriculum and understanding its effects on learning are challenging problems. In this paper, we define the Human-skill Curriculum Markov Decision Process (H-CMDP) to systematize the design of training protocols. We also identify a vocabulary of performance features to enable the approximation for a human’s skill level across a variety of cognitive and motor tasks. A novel task domain is introduced as a testbed to evaluate the effectiveness of our approach. Human subject experiments show that (1) participants can learn to improve their performance in tasks within this domain, (2) the learning is quantifiable via our performance features, and (3) the domain is flexible enough to create distinct levels of difficulty. The long-term goal of this work is to systematize the process of curriculum-based training toward the design of protocols for robot-mediated rehabilitation.

AAAI Conference 2021 Conference Paper

Expected Value of Communication for Planning in Ad Hoc Teamwork

• William Macke
• Reuth Mirsky
• Peter Stone

A desirable goal for autonomous agents is to be able to coordinate on the fly with previously unknown teammates. Known as “ad hoc teamwork”, enabling such a capability has been receiving increasing attention in the research community. One of the central challenges in ad hoc teamwork is quickly recognizing the current plans of other agents and planning accordingly. In this paper, we focus on the scenario in which teammates can communicate with one another, but only at a cost. Thus, they must carefully balance plan recognition based on observations vs. that based on communication. This paper proposes a new metric for evaluating how similar are two policies that a teammate may be following - the Expected Divergence Point (EDP). We then present a novel planning algorithm for ad hoc teamwork, determining which query to ask and planning accordingly. We demonstrate the effectiveness of this algorithm in a range of increasingly general communication in ad hoc teamwork problems.

AAMAS Conference 2021 Conference Paper

The Seeing-Eye Robot Grand Challenge: Rethinking Automated Care

• Reuth Mirsky
• Peter Stone

Automated care systems are becoming more tangible than ever: recent breakthroughs in robotics and machine learning can be used to address the need for automated care created by the increasing aging population. However, such systems require overcoming several technological, ethical, and social challenges. One inspirational manifestation of these challenges can be observed in the training of seeing-eye dogs for visually impaired people. A seeing-eye dog is not just trained to obey its owner, but also to “intelligently disobey”: if it is given an unsafe command from its handler, it is taught to disobey it or even insist on a different course of action. This paper proposes the challenge of building a seeing-eye robot, as a thought-provoking use-case that helps identify the challenges to be faced when creating behaviors for robot assistants in general. Through this challenge, this paper delineates the prerequisites that an automated care system will need to have in order to perform intelligent disobedience and to serve as a true agent for its handler.

IJCAI Conference 2020 Conference Paper

A Penny for Your Thoughts: The Value of Communication in Ad Hoc Teamwork

• Reuth Mirsky
• William Macke
• Andy Wang
• Harel Yedidsion
• Peter Stone

In ad hoc teamwork, multiple agents need to collaborate without having knowledge about their teammates or their plans a priori. A common assumption in this research area is that the agents cannot communicate. However, just as two random people may speak the same language, autonomous teammates may also happen to share a communication protocol. This paper considers how such a shared protocol can be leveraged, introducing a means to reason about Communication in Ad Hoc Teamwork (CAT). The goal of this work is enabling improved ad hoc teamwork by judiciously leveraging the ability of the team to communicate. We situate our study within a novel CAT scenario, involving tasks with multiple steps, where teammates' plans are unveiled over time. In this context, the paper proposes methods to reason about the timing and value of communication and introduces an algorithm for an ad hoc agent to leverage these methods. Finally, we introduces a new multiagent domain, the tool fetching domain, and we study how varying this domain's properties affects the usefulness of communication. Empirical results show the benefits of explicit reasoning about communication content and timing in ad hoc teamwork.

AIJ Journal 2019 Journal Article

Corrigendum to “Sequential plan recognition: An iterative approach to disambiguating between hypotheses” [Artif. Intell. 260 (2018) 51–73]

• Reuth Mirsky
• Roni Stern
• Kobi Gal
• Meir Kalech

TIST Journal 2019 Journal Article

Goal and Plan Recognition Design for Plan Libraries

• Reuth Mirsky
• Kobi Gal
• Roni Stern
• Meir Kalech

This article provides new techniques for optimizing domain design for goal and plan recognition using plan libraries. We define two new problems: Goal Recognition Design for Plan Libraries (GRD-PL) and Plan Recognition Design (PRD). Solving the GRD-PL helps to infer which goal the agent is trying to achieve, while solving PRD can help to infer how the agent is going to achieve its goal. For each problem, we define a worst-case distinctiveness measure that is an upper bound on the number of observations that are necessary to unambiguously recognize the agent’s goal or plan. This article studies the relationship between these measures, showing that the worst-case distinctiveness of GRD-PL is a lower bound of the worst-case plan distinctiveness of PRD and that they are equal under certain conditions. We provide two complete algorithms for minimizing the worst-case distinctiveness of plan libraries without reducing the agent’s ability to complete its goals: One is a brute-force search over all possible plans and one is a constraint-based search that identifies plans that are most difficult to distinguish in the domain. These algorithms are evaluated in three hierarchical plan recognition settings from the literature. We were able to reduce the worst-case distinctiveness of the domains using our approach, in some cases reaching 100% improvement within a predesignated time window. Our iterative algorithm outperforms the brute-force approach by an order of magnitude in terms of runtime.

AIJ Journal 2018 Journal Article

Sequential plan recognition: An iterative approach to disambiguating between hypotheses

• Reuth Mirsky
• Roni Stern
• Kobi Gal
• Meir Kalech

Plan recognition algorithms output hypotheses about an agent's plans from its observed actions. Due to imperfect knowledge about the agent's behavior and the environment, it is often the case that there are multiple hypotheses about an agent's plans that are consistent with the observations, though only one of these hypotheses is correct. This paper addresses the problem of how to disambiguate between hypotheses during the recognition process, by querying the acting agent about whether a given plan is part of the correct hypothesis. The main contribution is a sound and complete process for reducing the set of possible hypotheses called Sequential Plan Recognition (SPR). SPR iteratively queries the user and revises the set of possible hypotheses according to the outcome of the query. Several policies are provided for choosing which plans to query the agent. These policies address the problem of how to reduce the number of hypotheses during the recognition process using a minimal number of queries. The proposed policies include policies that use maximum likelihood and information gain measures. The paper provides a complexity analysis of the SPR process and the proposed query policies. It demonstrate its efficiency on two known domains from the literature, describing how performance and runtime are affected by features in the domain. Our results can inform the design of future plan recognition systems that interleave the recognition process with intelligent interventions of their users.

TIST Journal 2017 Journal Article

CRADLE

• Reuth Mirsky
• Ya’akov (Kobi) Gal
• Stuart M. Shieber

In exploratory domains, agents’ behaviors include switching between activities, extraneous actions, and mistakes. Such settings are prevalent in real world applications such as interaction with open-ended software, collaborative office assistants, and integrated development environments. Despite the prevalence of such settings in the real world, there is scarce work in formalizing the connection between high-level goals and low-level behavior and inferring the former from the latter in these settings. We present a formal grammar for describing users’ activities in such domains. We describe a new top-down plan recognition algorithm called CRADLE (Cumulative Recognition of Activities and Decreasing Load of Explanations) that uses this grammar to recognize agents’ interactions in exploratory domains. We compare the performance of CRADLE with state-of-the-art plan recognition algorithms in several experimental settings consisting of real and simulated data. Our results show that CRADLE was able to output plans exponentially more quickly than the state-of-the-art without compromising its correctness, as determined by domain experts. Our approach can form the basis of future systems that use plan recognition to provide real-time support to users in a growing class of interesting and challenging domains.

AAAI Conference 2017 Short Paper

Plan Recognition Design

• Reuth Mirsky
• Roni Stern
• Ya'akov Gal
• Meir Kalech

Goal Recognition Design (GRD) is the problem of designing a domain in a way that will allow easy identification of agents’ goals. This work extends the original GRD problem to the Plan Recognition Design (PRD) problem which is the task of designing a domain using plan libraries in order to facilitate fast identification of an agent’s plan. While GRD can help to explain faster which goal the agent is trying to achieve, PRD can help in faster understanding of how the agent is going to achieve its goal. we define a new measure that quantifies the worst-case distinctiveness of a given planning domain, propose a method to reduce it in a given domain and show the reduction of this new measure in three domains from the literature.

AAMAS Conference 2016 Conference Paper

Plan Recognition in Exploratory Domains (Doctoral Consortium)

• Reuth Mirsky

My thesis focuses on recognition and interventions of users’ activities using open-ended and flexible software. In such exploratory settings, users’ behavior is characterized by exploration, mistakes and trial-and-error. Exploratory domains provide a flexible and rich interaction environment for their users, but induce challenges for automatic recognition and support of their activities. My thesis focuses on the following three challenges which are central to understanding users’ interactions in exploratory settings and to use this understanding in order to provide them with effective support and guidance: (1) Representing and inferring users’ interactions in exploratory domains. (2) Disambiguating between possible explanations in order to improve understanding of users’ behavior. (3) Producing machine-generated support that adapts to the needs of the users. My research activities combines computational models, algorithms and empirical methodologies to meet the challenges above. They are conducted in the context of various types of exploratory settings. Specifically, I am developing novel plan recognition algorithms for inferring users’ interactions in exploratory settings and intervention mechanisms for these environments. I am evaluating my approach in the real world using educational software, medical records and cyber security domains. My results so far include (1) design of a new model for plan recognition; (2) an online plan recognition algorithm that is empirically shown to outperform the state-of-the-art methods in the real world; (3) A sequential process that allows informed disambiguation of possible hypotheses describing an agent’s plans. The long term impact of my contribution to computer science will be demonstrated by (1) developing new algorithms for plan recognition, intervention design and adaptation for exploratory settings; (2) showing that these methods generalize to different types of settings that differ widely in they type of interaction that is provided by the users.

IJCAI Conference 2016 Conference Paper

Sequential Plan Recognition

• Reuth Mirsky
• Roni Stern
• Ya'akov (Kobi) Gal
• Meir Kalech

Plan recognition algorithms infer agents' plans from their observed actions. Due to imperfect knowledge about the agent's behavior and the environment, it is often the case that there are multiple hypotheses about an agent's plans that are consistent with the observations, though only one of these hypotheses is correct. This paper addresses the problem of how to disambiguate between hypotheses, by querying the acting agent about whether a candidate plan in one of the hypotheses matches its intentions. This process is performed sequentially and used to update the set of possible hypotheses during the recognition process. The paper defines the sequential plan recognition process (SPRP), which seeks to reduce the number of hypotheses using a minimal number of queries. We propose a number of policies for the SPRP which use maximum likelihood and information gain to choose which plan to query. We show this approach works well in practice on two domains from the literature, significantly reducing the number of hypotheses using fewer queries than a baseline approach. Our results can inform the design of future plan recognition systems that interleave the recognition process with intelligent interventions of their users.

AAMAS Conference 2016 Conference Paper

Sequential Plan Recognition (Extended Abstract)

• Reuth Mirsky
• Ya'akov (Kobi) Gal
• Roni Stern
• Meir Kalech

Plan recognition algorithms need to maintain all candidate hypotheses which are consistent with the observations, even though there is only a single hypothesis that is the correct one. Unfortunately, the number of possible hypotheses can be exponentially large in practice. This paper addresses the problem of how to disambiguate between many possible hypotheses that are all consistent with the actions of the observed agent. One way to reduce the number of hypotheses is to consult a domain expert or the acting agent directly about its intentions. This process can be performed sequentially, updating the set of hypotheses during the recognition process. The paper specifically addresses the problem of how to minimize the number of queries made that are required to find the correct hypothesis. It adapts a number of probing techniques for choosing which plan to query, such as maximal information gain and maximum likelihood. These approaches were evaluated on a domain from the literature using a well known plan recognition algorithm. The results showed that the information gain approach was able to find the correct plan using significantly fewer queries than the maximum likelihood approach as well as a baseline approach choosing random plans. Our technique can inform the design of future plan recognition systems that interleave the recognition process with intelligent interventions of their users.

IJCAI Conference 2016 Conference Paper

SLIM: Semi-Lazy Inference Mechanism for Plan Recognition

• Reuth Mirsky
• Ya'akov (Kobi) Gal

Plan Recognition algorithms require to recognize a complete hierarchy explaining the agent's actions and goals. While the output of such algorithms is informative to the recognizer, the cost of its calculation is high in run-time, space, and completeness. Moreover, performing plan recognition online requires the observing agent to reason about future actions that have not yet been seen and maintain a set of hypotheses to support all possible options. This paper presents a new and efficient algorithm for online plan recognition called SLIM (Semi-Lazy Inference Mechanism). It combines both a bottom-up and top-down parsing processes, which allow it to commit only to the minimum necessary actions in real-time, but still provide complete hypotheses post factum. We show both theoretically and empirically that although the computational cost of this process is still exponential, there is a significant improvement in run-time when compared to a state of the art of plan recognition algorithm.