Author name cluster

Ronal Singh

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

ECAI Conference 2024 Conference Paper

Towards the New XAI: A Hypothesis-Driven Approach to Decision Support Using Evidence

Thao Le 0002
Tim Miller 0001
Liz Sonenberg
Ronal Singh

Prior research on AI-assisted human decision-making has explored several different explainable AI (XAI) approaches. A recent paper has proposed a paradigm shift calling for hypothesis-driven XAI through a conceptual framework called evaluative AI that gives people evidence that supports or refutes hypotheses without necessarily giving a decision-aid recommendation. In this paper, we describe and evaluate an approach for hypothesis-driven XAI based on the Weight of Evidence (WoE) framework, which generates both positive and negative evidence for a given hypothesis. Through human behavioural experiments, we show that our hypothesis-driven approach increases decision accuracy and reduces reliance compared to a recommendation-driven approach and an AI-explanation-only baseline, but with a small increase in under-reliance compared to the recommendation-driven approach. Further, we show that participants used our hypothesis-driven approach in a materially different way to the two baselines.

Details

AAAI Conference 2023 Conference Paper

Explaining Model Confidence Using Counterfactuals

Thao Le
Tim Miller
Ronal Singh
Liz Sonenberg

Displaying confidence scores in human-AI interaction has been shown to help build trust between humans and AI systems. However, most existing research uses only the confidence score as a form of communication. As confidence scores are just another model output, users may want to understand why the algorithm is confident to determine whether to accept the confidence score. In this paper, we show that counterfactual explanations of confidence scores help study participants to better understand and better trust a machine learning model's prediction. We present two methods for understanding model confidence using counterfactual explanation: (1) based on counterfactual examples; and (2) based on visualisation of the counterfactual space. Both increase understanding and trust for study participants over a baseline of no explanation, but qualitative results show that they are used quite differently, leading to recommendations of when to use each one and directions of designing better explanations.

PDF Details DOI

JAIR Journal 2021 Journal Article

Goal Recognition for Deceptive Human Agents through Planning and Gaze

Thao Le
Ronal Singh
Tim Miller

Eye gaze has the potential to provide insight into the minds of individuals, and this idea has been used in prior research to improve human goal recognition by combining human's actions and gaze. However, most existing research assumes that people are rational and honest. In adversarial scenarios, people may deliberately alter their actions and gaze, which presents a challenge to goal recognition systems. In this paper, we present new models for goal recognition under deception using a combination of gaze behaviour and observed movements of the agent. These models aim to detect when a person is deceiving by analysing their gaze patterns and use this information to adjust the goal recognition. We evaluated our models in two human-subject studies: (1) using data collected from 30 individuals playing a navigation game inspired by an existing deception study and (2) using data collected from 40 individuals playing a competitive game (Ticket To Ride). We found that one of our models (Modulated Deception Gaze+Ontic) offers promising results compared to the previous state-of-the-art model in both studies. Our work complements existing adversarial goal recognition systems by equipping these systems with the ability to tackle ambiguous gaze behaviours.

PDF Details DOI

AIJ Journal 2020 Journal Article

Combining gaze and AI planning for online human intention recognition

Ronal Singh
Tim Miller
Joshua Newn
Eduardo Velloso
Frank Vetere
Liz Sonenberg

Intention recognition is the process of using behavioural cues, such as deliberative actions, eye gaze, and gestures, to infer an agent's goals or future behaviour. In artificial intelligence, one approach for intention recognition is to use a model of possible behaviour to rate intentions as more likely if they are a better ‘fit’ to actions observed so far. In this paper, we draw from literature linking gaze and visual attention, and we propose a novel model of online human intention recognition that combines gaze and model-based AI planning to build probability distributions over a set of possible intentions. In human-behavioural experiments ( n = 40 ) involving a multi-player board game, we demonstrate that adding gaze-based priors to model-based intention recognition improved the accuracy of intention recognition by 22% ( p < 0. 05 ), determined those intentions ≈90 seconds earlier ( p < 0. 05 ), and at no additional computational cost. We also demonstrate that, when evaluated in the presence of semi-rational or deceptive gaze behaviours, the proposed model is significantly more accurate (9% improvement) ( p < 0. 05 ) compared to a model-based or gaze only approaches. Our results indicate that the proposed model could be used to design novel human-agent interactions in cases when we are unsure whether a person is honest, deceitful, or semi-rational.

Details DOI

AAMAS Conference 2018 Conference Paper

Combining Planning with Gaze for Online Human Intention Recognition

Ronal Singh
Tim Miller
Joshua Newn
Liz Sonenberg
Eduardo Velloso
Frank Vetere

Intention recognition is the process of using behavioural cues to infer an agent’s goals or future behaviour. People use many behavioural cues to infer others’ intentions, such as deliberative actions, facial expressions, eye gaze, and gestures. In artificial intelligence, two approaches for intention recognition, among others, are gaze-based and model-based intention recognition. Approaches in the former class use gaze to determine which parts of a space a person looks at more often to infer a person’s intention. Approaches in the latter use models of possible future behaviour to rate intentions as more likely if they are a better ‘fit’ to observed actions. In this paper, we propose a novel model of human intention recognition that combines gaze and model-based approaches for online human intention recognition. Gaze data is used to build probability distributions over a set of possible intentions, which are then used as priors in a model-based intention recognition algorithm. In humanbehavioural experiments (n = 20) involving a multi-player board game, we found that adding gaze-based priors to model-based intention recognition more accurately determined intentions (p < 0. 01), determined those intentions earlier (p < 0. 01), and at no additional cost; all compared to a model-based-only approach.

PDF