Arrow Research search

Author name cluster

Elisabeth André

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

17 papers
2 author rows

Possible papers

17

IJCAI Conference 2024 Conference Paper

Relevant Irrelevance: Generating Alterfactual Explanations for Image Classifiers

  • Silvan Mertes
  • Tobias Huber
  • Christina Karle
  • Katharina Weitz
  • Ruben Schlagowski
  • Cristina Conati
  • Elisabeth André

In this paper, we demonstrate the feasibility of alterfactual explanations for black box image classifiers. Traditional explanation mechanisms from the field of Counterfactual Thinking are a widely-used paradigm for Explainable Artificial Intelligence (XAI), as they follow a natural way of reasoning that humans are familiar with. However, most common approaches from this field are based on communicating information about features or characteristics that are especially important for an AI's decision. However, to fully understand a decision, not only knowledge about relevant features is needed, but the awareness of irrelevant information also highly contributes to the creation of a user's mental model of an AI system. To this end, a novel approach for explaining AI systems called alterfactual explanations was recently proposed on a conceptual level. It is based on showing an alternative reality where irrelevant features of an AI's input are altered. By doing so, the user directly sees which input data characteristics can change arbitrarily without influencing the AI's decision. In this paper, we show for the first time that it is possible to apply this idea to black box models based on neural networks. To this end, we present a GAN-based approach to generate these alterfactual explanations for binary image classifiers. Further, we present a user study that gives interesting insights on how alterfactual explanations can complement counterfactual explanations.

PDF Details DOI

AAMAS Conference 2023 Conference Paper

GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanations

  • Tobias Huber
  • Maximilian Demmler
  • Silvan Mertes
  • Matthew L. Olson
  • Elisabeth André

Counterfactual explanations are a common tool to explain artificial intelligence models. For Reinforcement Learning (RL) agents, they answer "Why not? " or "What if? " questions by illustrating what minimal change to a state is needed such that an agent chooses a different action. Generating counterfactual explanations for RL agents with visual input is especially challenging because of their large state spaces and because their decisions are part of an overarching policy, which includes long-term decision-making. However, research focusing on counterfactual explanations, specifically for RL agents with visual input, is scarce and does not go beyond identifying defective agents. It is unclear whether counterfactual explanations are still helpful for more complex tasks like analyzing the learned strategies of different agents or choosing a fitting agent for a specific task. We propose a novel but simple method to generate counterfactual explanations for RL agents by formulating the problem as a domain transfer problem which allows the use of adversarial learning techniques like StarGAN. Our method is fully model-agnostic and we demonstrate that it outperforms the only previous method in several computational metrics. Furthermore, we show in a user study that our method performs best when analyzing which strategies different agents pursue.

PDF

IJCAI Conference 2022 Conference Paper

Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps (Extended Abstract)

  • Tobias Huber
  • Katharina Weitz
  • Elisabeth André
  • Ofra Amir

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as they act in large state spaces, and their decision-making can be affected by delayed rewards. In this paper, we explore a combination of explanations that attempt to convey the global behavior of the agent and local explanations which provide information regarding the agent's decision-making in a particular state. Specifically, we augment strategy summaries that demonstrate the agent's actions in a range of states with saliency maps highlighting the information it attends to. Our user study shows that intelligently choosing what states to include in the summary (global information) results in an improved analysis of the agents. We find mixed results with respect to augmenting summaries with saliency maps (local information).

PDF Details DOI

AIJ Journal 2021 Journal Article

Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps

  • Tobias Huber
  • Katharina Weitz
  • Elisabeth André
  • Ofra Amir

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the global behavior of the agent, describing the actions it takes in different states. Other approaches devised local explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summaries that extract important trajectories of states from simulations of the agent with saliency maps which show what information the agent attends to. Our results show that the choice of what states to include in the summary (global information) strongly affects people's understanding of agents: participants shown summaries that included important states significantly outperformed participants who were presented with agent behavior in a set of world-states that are likely to appear during gameplay. We find mixed results with respect to augmenting demonstrations with saliency maps (local information), as the addition of saliency maps, in the form of raw heat maps, did not significantly improve performance in most cases. However, we do find some evidence that saliency maps can help users better understand what information the agent relies on during its decision-making, suggesting avenues for future work that can further improve explanations of RL agents.

Details DOI

AAMAS Conference 2019 Conference Paper

Irony Man: Augmenting a Social Robot with the Ability to Use Irony in Multimodal Communication with Humans

  • Hannes Ritschel
  • Ilhan Aslan
  • David Sedlbauer
  • Elisabeth André

Interpersonal communication is often full of irony and irony related humor, which can shape the quality of a conversation and how conversation partners perceive each other. If social robots were able to integrate irony in their communication style, their human conversation partners might perceive them as more natural, credible, and ultimately more attractive and acceptable. In order to explore this assumption, we first describe an approach to transform non-ironic inputs on-the-fly into multimodal ironic utterances. Irony markers are used to adapt language, prosody and facial expression. We argue that doing this allows to dynamically enrich a robot’s spoken language with an expression of socially intelligent behavior. We then demonstrate the feasibility of our approach by reporting on a user study, which compares an ironic version of a robot with a non-ironic version of the same robot in a small talk dialog scenario. Results show that participants are indeed able to correctly identify a robot’s use of irony and that a better user experience is associated with an ironic robot version. This is an important step for dynamically shaping a robot’s personality and humor, and to increase perceived social intelligence.

PDF

AAMAS Conference 2019 Conference Paper

What If I Speak Now? A Decision-Theoretic Approach to Personality-Based Turn-Taking

  • Kathrin Janowski
  • Elisabeth André

Embodied conversational agents, which are increasingly prevalent in our society, require turn-taking mechanisms that not only generate fluent conversations but are also consistent with the personality and interpersonal stance required in the given context. We present a decision-theoretic approach for deriving the turn-taking behavior of such an agent from the personality it is meant to convey. For this we gathered relevant theories from psychology and communications research, as well as related systems employing utility-based reasoning. On this basis we describe the construction of an influence diagram which decides between acting and waiting based on those actions’ expected utility for the agent’s personality-related interaction goals. To test our approach, we integrated our model into an application which simulates conversations between two virtual characters. We then evaluated our prototype by presenting videos of those conversations in an online survey. Our results confirmed that differences in an agent’s speaking behavior, generated from different Extraversion configurations in our model, lead to the intended perceptions of its Extraversion, Agreeableness and Status.

PDF

ECAI Conference 2014 Conference Paper

Modeling Gaze Mechanisms for Grounding in HRI

  • Gregor Mehlmann
  • Kathrin Janowski
  • Tobias Baur 0001
  • Markus Häring
  • Elisabeth André
  • Patrick Gebhard

Grounding is essential in human interaction and crucial for social robots collaborating with humans. Gaze plays versatile roles for establishing, maintaining and repairing the common ground. It is combined with parallel modalities and involved in several processes for behavior generation and recognition. We present a uniform modeling approach focusing on the multi-modal, parallel and bidirectional aspects of gaze and their interleaving with the dialog logic.

Details

JAAMAS Journal 2013 Journal Article

Investigating culture-related aspects of behavior for virtual characters

  • Birgit Endrass
  • Elisabeth André
  • Yukiko Nakano

Abstract In this paper, culture-related behaviors are investigated on several channels of communication for virtual characters. Prototypical behaviors were formalized in computational models based on a literature review as well as a corpus analysis, exemplifying the German and Japanese cultures. Therefore, aspects of verbal behavior, communication management and nonverbal behavior were taken into account. In evaluation studies conducted in the targeted cultures, each aspect’s impact on human observers was tested. With it, we investigated for which of the aspects, observers prefer agent behavior that was designed to resemble their own cultural background.

Details DOI

AAMAS Conference 2013 Conference Paper

Traveller: An Intercultural Training System with Intelligent Agents

  • Samuel Mascarenhas
  • André Silva
  • Ana Paiva
  • RUTH AYLETT
  • Felix Kistler
  • Elisabeth André
  • Nick Degens
  • Gert Jan Hofstede

There is a growing demand for new forms of intercultural training. We describe a demonstration of an agent-based application designed to teach young adults (18-25) general patterns of behaviour that can distinguish a broad range of cultures. Training is done through an interactive-story telling approach where the user must go through a series of critical incidents, interacting with agents capable of simulating different synthetic cultures in their behaviour.

PDF

AAMAS Conference 2009 Conference Paper

Culture-specific Communication Management for Virtual Agents

  • Birgit Endrass
  • Matthias Rehm
  • Elisabeth André

Human interaction depends on several individual factors such as personality, social relations, age or gender. But also the society we live in influences our behaviour. Thus culture affects the way communication is led. As virtual agents interact in a more and more human-like manner, culture-specific behaviour should also be taken into account. In this paper, we investigate communication management as one aspect of communication. Our findings in culture related differences are based on a video corpus that was recorded in Germany and Japan as well as on findings described in the literature. To this end, the use of pauses in speech as well as the occurrence of overlapping speech was analyzed and integrated into a demonstrator using virtual agents. In a preliminary study, we investigated whether subjects perceive a difference between agent dialogs that are in line with culturespecific findings and agent dialogs that are not.

PDF

AAMAS Conference 2009 Conference Paper

Emotional Input for Character-based Interactive Storytelling

  • Marc Cavazza
  • David Pizzi
  • Fred Charles
  • Thurid Vogt
  • Elisabeth André

In most Interactive Storytelling systems, user interaction is based on natural language communication with virtual agents, either through isolated utterances or through dialogue. Natural language communication is also an essential element of interactive narratives in which the user is supposed to impersonate one of the story’s characters. Whilst techniques for narrative generation and agent behaviour have made significant progress in recent years, natural language processing remains a bottleneck hampering the scalability of Interactive Storytelling systems. In this paper, we introduce a novel interaction technique based solely on emotional speech recognition. It allows the user to take part in dialogue with virtual actors without any constraints on style or expressivity, by mapping the recognised emotional categories to narrative situations and virtual characters feelings. Our Interactive Storytelling system uses an emotional planner to drive characters’ behaviours. The main feature of this approach is that characters’ feelings are part of the planning domain and are at the heart of narrative representations. The emotional speech recogniser analyses the speech signal to produce a variety of features which can be used to define ad-hoc categories on which to train the system. The content of our interactive narrative is an adaptation of one chapter of the XIXth century classic novel, Madame Bovary, which is well suited to a formalisation in terms of characters’ feelings. At various stages of the narrative, the user can address the main character or respond to her, impersonating her lover. The emotional category extracted from the user utterance can be analysed in terms of the current narrative context, which includes characters’ beliefs, feelings and expectations, to produce a specific influence on the target character, which will become visible through a change in its behaviour, achieving a high level of realism for the interaction. A limited number of emotional categories is sufficient to drive the narrative across multiple courses of actions, since it comprises over thirty narrative functions. We report results from a fully implemented prototype, both in terms of proof of concept and of usability through a preliminary user study.

PDF

AAAI Conference 1996 Conference Paper

Coping with Temporal Constraints in Multimedia Presentation Planning

  • Elisabeth André

Computer-based presentation systems enable the realization of effective and dynamic presentation styles that incorporate multiple media. Obvious examples are animated user interface agents which verbally comment on multimedia objects displayed on the screen while performing cross-media and cross-window pointing gestures. The design of such presentations must account for the temporal coordination of media output and the agent’ s behavior. In this paper we describe a new presentation system which not only creates the multimedia objects to be presented, but also generates a script for presenting the material to the user. In our system, this script is forwarded to an animated presentation agent running the presentation. The paper details the kernel of the system which is a component for planning temporally coordinated multimedia. Figure 1: Verbal Annotation of Graphical Objects

PDF Details

AIJ Journal 1993 Journal Article

Plan-based integration of natural language and graphics generation

  • Wolfgang Wahlster
  • Elisabeth André
  • Wolfgang Finkler
  • Hans-Jürgen Profitlich
  • Thomas Rist

Multimodal interfaces combining natural language and graphics take advantage of both the individual strength of each communication mode and the fact that several modes can be employed in parallel. The central claim of this paper is that the generation of a multimodal presentation can be considered as an incremental planning process that aims to achieve a given communicative goal. We describe the multimodal presentation system WIP which allows the generation of alternate presentations of the same content taking into account various contextual factors. We discuss how the plan-based approach to presentation design can be exploited so that graphics generation influences the production of text and vice versa. We show that well-known concepts from the area of natural language processing like speech acts, anaphora, and rhetorical relations take on an extended meaning in the context of multimodal communication. Finally, we discuss two detailed examples illustrating and reinforcing our theoretical claims.

Details DOI
v2026.09.13