Avi Rosenfeld

AAMAS Conference 2021 Conference Paper

Better Metrics for Evaluating Explainable Artificial Intelligence

• Avi Rosenfeld

This paper presents objective metrics for how explainable artificial intelligence (XAI) can be quantified. Through an overview of current trends, we show that many explanations are generated post-hoc and independent of the agent’s logical process, which in turn creates explanations with limited meaning as they lack transparency and fidelity. While user studies are a known basis for evaluating XAI, studies that do not consider objective metrics for evaluating XAI may have limited meaning and may suffer from confirmation bias, particularly if they use low fidelity explanations unnecessarily. To avoid this issue, this paper suggests a paradigm shift in evaluating XAI that focuses on metrics that quantify the explanation itself and its appropriateness given the XAI goal. We suggest four such metrics based on performance differences, D, between the explanation’s logic and the agent’s actual performance, the number of rules, R, outputted by the explanation, the number of features, F, used to generate that explanation, and the stability, S, of the explanation. We believe that user studies that focus on these metrics in their evaluations are inherently more valid and should be integrated in future XAI research.

JAAMAS Journal 2019 Journal Article

Explainability in human–agent systems

• Avi Rosenfeld
• Ariella Richardson

Abstract This paper presents a taxonomy of explainability in human–agent systems. We consider fundamental questions about the Why, Who, What, When and How of explainability. First, we define explainability, and its relationship to the related terms of interpretability, transparency, explicitness, and faithfulness. These definitions allow us to answer why explainability is needed in the system, whom it is geared to and what explanations can be generated to meet this need. We then consider when the user should be presented with this information. Last, we consider how objective and subjective measures can be used to evaluate the entire system. This last question is the most encompassing as it will need to evaluate all other issues regarding explainability.

IJCAI Conference 2017 Conference Paper

Evaluating Market User Interfaces for Electric Vehicle Charging using Bid2Charge

• Sebastian Stein
• Enrico H. Gerding
• Adrian Nedea
• Avi Rosenfeld
• Nicholas R. Jennings

We consider settings where electric vehicle drivers participate in a market mechanism to charge their vehicles. Existing work typically assumes that participants are fully rational and can report their charging preferences accurately. However, this may not be reasonable in settings with non-experts. To explore this, we design a novel game called Bid2Charge and compare a fully expressive interface that covers the entire space of preferences to two restricted interfaces that offer fewer possible reports. We show that restricting the users' preferences significantly reduces deliberation times while also leading to an increase in utility by up to 70%.

AIJ Journal 2017 Journal Article

Making friends on the fly: Cooperating with new teammates

• Samuel Barrett
• Avi Rosenfeld
• Sarit Kraus
• Peter Stone

Robots are being deployed in an increasing variety of environments for longer periods of time. As the number of robots grows, they will increasingly need to interact with other robots. Additionally, the number of companies and research laboratories producing these robots is increasing, leading to the situation where these robots may not share a common communication or coordination protocol. While standards for coordination and communication may be created, we expect that robots will need to additionally reason intelligently about their teammates with limited information. This problem motivates the area of ad hoc teamwork in which an agent may potentially cooperate with a variety of teammates in order to achieve a shared goal. This article focuses on a limited version of the ad hoc teamwork problem in which an agent knows the environmental dynamics and has had past experiences with other teammates, though these experiences may not be representative of the current teammates. To tackle this problem, this article introduces a new general-purpose algorithm, PLASTIC, that reuses knowledge learned from previous teammates or provided by experts to quickly adapt to new teammates. This algorithm is instantiated in two forms: 1) PLASTIC-Model – which builds models of previous teammates' behaviors and plans behaviors online using these models and 2) PLASTIC-Policy – which learns policies for cooperating with previous teammates and selects among these policies online. We evaluate PLASTIC on two benchmark tasks: the pursuit domain and robot soccer in the RoboCup 2D simulation domain. Recognizing that a key requirement of ad hoc teamwork is adaptability to previously unseen agents, the tests use more than 40 previously unknown teams on the first task and 7 previously unknown teams on the second. While PLASTIC assumes that there is some degree of similarity between the current and past teammates' behaviors, no steps are taken in the experimental setup to make sure this assumption holds. The teammates were created by a variety of independent developers and were not designed to share any similarities. Nonetheless, the results show that PLASTIC was able to identify and exploit similarities between its current and past teammates' behaviors, allowing it to quickly adapt to new teammates.

JAIR Journal 2017 Journal Article

Market Interfaces for Electric Vehicle Charging

• Sebastian Stein
• Enrico H. Gerding
• Adrian Nedea
• Avi Rosenfeld
• Nicholas R. Jennings

We consider settings where owners of electric vehicles (EVs) participate in a market mechanism to charge their vehicles. Existing work on such mechanisms has typically assumed that participants are fully rational and can report their preferences accurately via some interface to the mechanism or to a software agent participating on their behalf. However, this may not be reasonable in settings with non-expert human end-users.Thus, our overarching aim in this paper is to determine experimentally if a fully expressive market interface that enables accurate preference reports is suitable for the EV charging domain, or, alternatively, if a simpler, restricted interface that reduces the space of possible options is preferable. In doing this, we measure the performance of an interface both in terms of how it helps participants maximise their utility and how it affects deliberation time. Our secondary objective is to contrast two different types of restricted interfaces that vary in how they restrict the space of preferences that can be reported. To enable this analysis, we develop a novel game that replicates key features of an abstract EV charging scenario. In two experiments with over 300 users, we show that restricting the users' preferences significantly reduces the time they spend deliberating (by up to half in some cases). An extensive usability survey confirms that this restriction is furthermore associated with a lower perceived cognitive burden on the users. More surprisingly, at the same time, using restricted interfaces leads to an increase in the users' performance compared to the fully expressive interface (by up to 70%). We also show that some restricted interfaces have the desirable effect of reducing the energy consumption of their users by up to 20% while achieving the same utility as other interfaces. Finally, we find that a reinforcement learning agent displays similar performance trends to human users, enabling a novel methodology for evaluating market interfaces.

AAMAS Conference 2016 Conference Paper

Bid2Charge: Market User Interface Design for Electric Vehicle Charging

• Sebastian Stein
• Enrico H. Gerding
• Adrian Nedea
• Avi Rosenfeld
• Nicholas R. Jennings

We consider settings where owners of electric vehicles (EVs) participate in a market mechanism to charge their vehicles. Existing work on such mechanisms has typically assumed that participants are fully rational and can report their preferences accurately to the mechanism or to a software agent participating on their behalf. However, this may not be reasonable in settings with non-expert human end-users. To explore this, we compare a fully expressive interface that covers the entire space of preferences to two restricted interfaces that reduce the space of possible options. To enable this analysis, we develop a novel game that replicates key features of an abstract EV charging scenario. In two extensive evaluations with over 300 users, we show that restricting the users’ preferences significantly reduces the time they spend deliberating. More surprisingly, it also leads to an increase in their utility compared to the fully expressive interface (up to 70%). Finally, we find that a reinforcement learning agent displays similar performance trends, enabling a novel methodology for evaluating market interfaces.

AAAI Conference 2015 Conference Paper

A Hybrid Approach of Classifier and Clustering for Solving the Missing Node Problem

• Sigal Sina
• Avi Rosenfeld
• Sarit Kraus
• Navot Akiva

An important area of social network research is identifying missing information which is not explicitly represented in the network or is not visible to all. In this paper, we propose a novel Hybrid Approach of Classifier and Clustering, which we refer to as HACC, to solve the missing node identification problem in social networks. HACC utilizes a classifier as a preprocessing step in order to integrate all known information into one similarity measure and then uses a clustering algorithm to identify missing nodes. Specifically, we used the information on the network structure, attributes about known users (nodes) and pictorial information to evaluate HACC and found that it performs significantly better than other missing node algorithms. We also argue that HACC is a general approach and domain independent and can be easily applied to other domains. We support this claim by evaluating HACC on a second authorship identification domain as well.

JAAMAS Journal 2015 Journal Article

Autonomous agents and human cultures in the trust–revenge game

• Amos Azaria
• Ariella Richardson
• Avi Rosenfeld

Abstract Autonomous agents developed by experts are embedded with the capability to interact well with people from different cultures. When designing expert agents intended to interact with autonomous agents developed by non-game theory agents (NGTE), it is beneficial to obtain insights on the behavior of these NGTE agents. Is the behavior of these NGTE agents similar to human behavior from different cultures? This is an important question as such a quality would allow an expert agent interacting with NGTE agents to model them using the same methods that are used to model humans from different cultures. To study this point, we evaluated NGTE agents behavior using a game called the Trust–Revenge game, which is known in social science for capturing different human tendencies. The Trust–Revenge game has a unique subgame-perfect equilibrium strategy profile, however, very rarely do people follow it. We compared the behavior of autonomous agents to the actions of several human demographic groups—one of which is similar to the designers of the autonomous agents. We claim that autonomous agents are similar to human players from various cultures. This enables the use of approaches, developed for handling cultural diversity among humans, to be applied for interaction with NGTE agents. This paper also analyzes additional aspects of autonomous agents behavior and whether composing autonomous agents affects human behavior.

AAAI Conference 2015 Conference Paper

Crowdsourcing Complex Workflows under Budget Constraints

• Long Tran-Thanh
• Trung Dong Huynh
• Avi Rosenfeld
• Sarvapali Ramchurn
• Nicholas Jennings

We consider the problem of task allocation in crowdsourcing systems with multiple complex workflows, each of which consists of a set of inter-dependent micro-tasks. We propose Budgeteer, an algorithm to solve this problem under a budget constraint. In particular, our algorithm first calculates an efficient way to allocate budget to each workflow. It then determines the number of inter-dependent micro-tasks and the price to pay for each task within each workflow, given the corresponding budget constraints. We empirically evaluate it on a well-known crowdsourcing-based text correction workflow using Amazon Mechanical Turk, and show that Budgeteer can achieve similar levels of accuracy to current benchmarks, but is on average 45% cheaper.

JAAMAS Journal 2015 Journal Article

Guest Editorial

• Sarvapali D. Ramchurn
• Avi Rosenfeld
• Joel E. Fischer

JAAMAS Journal 2015 Journal Article

NegoChat-A: a chat-based negotiation agent with bounded rationality

• Avi Rosenfeld
• Inon Zuckerman
• Sarit Kraus

Abstract To date, a variety of automated negotiation agents have been created. While each of these agents has been shown to be effective in negotiating with people in specific environments, they typically lack the natural language processing support required to enable real-world types of interactions. To address this limitation, we present NegoChat-A, an agent that incorporates several significant research contributions. First, we found that simply modifying existing agents to include an natural language processing module is insufficient to create these agents. Instead, agents that support natural language must have strategies that allow for partial agreements and issue-by-issue interactions. Second, we present NegoChat-A’s negotiation algorithm. This algorithm is based on bounded rationality, and specifically anchoring and aspiration adaptation theory. The agent begins each negotiation interaction by proposing a full offer, which serves as its anchor. Assuming this offer is not accepted, the agent then proceeds to negotiate via partial agreements, proposing the next issue for negotiation based on people’s typical urgency, or order of importance. We present a rigorous evaluation of NegoChat-A, showing its effectiveness in two different negotiation roles.

AAAI Conference 2015 Conference Paper

When Suboptimal Rules

• Avshalom Elmalech
• David Sarne
• Avi Rosenfeld
• Eden Erez

This paper represents a paradigm shift in what advice agents should provide people. Contrary to what was previously thought, we empirically show that agents that dispense optimal advice will not necessary facilitate the best improvement in people’s strategies. Instead, we claim that agents should at times suboptimally advise. We provide results demonstrating the effectiveness of a suboptimal advising approach in extensive experiments in two canonical mixed agent-human advice-giving domains. Our proposed guideline for suboptimal advising is to rely on the level of intuitiveness of the optimal advice as a measure for how much the suboptimal advice presented to the user should drift from the optimal value.

AAAI Conference 2014 Conference Paper

Generating Content for Scenario-Based Serious-Games Using CrowdSourcing

• Sigal Sina
• Avi Rosenfeld
• Sarit Kraus

Scenario-based serious-games have become an important tool for teaching new skills and capabilities. An important factor in the development of such systems is reducing the time and cost overheads in manually creating content for these scenarios. To address this challenge, we present Scenario- Gen, an automatic method for generating content about everyday activities through combining computer science techniques with the crowd. ScenarioGen uses the crowd in three different ways: to capture a database of scenarios of everyday activities, to generate a database of likely replacements for specific events within that scenario, and to evaluate the resulting scenarios. We evaluated ScenarioGen in 6 different content domains and found that it was consistently rated as coherent and consistent as the originally captured content. We also compared ScenarioGen’s content to that created by traditional planning techniques. We found that both methods were equally effective in generating coherent and consistent scenarios, yet ScenarioGen’s content was found to be more varied and easier to create.

AAAI Conference 2013 Conference Paper

Teamwork with Limited Knowledge of Teammates

• Samuel Barrett
• Peter Stone
• Sarit Kraus
• Avi Rosenfeld

While great strides have been made in multiagent teamwork, existing approaches typically assume extensive information exists about teammates and how to coordinate actions. This paper addresses how robust teamwork can still be created even if limited or no information exists about a specific group of teammates, as in the ad hoc teamwork scenario. The main contribution of this paper is the first empirical evaluation of an agent cooperating with teammates not created by the authors, where the agent is not provided expert knowledge of its teammates. For this purpose, we develop a generalpurpose teammate modeling method and test the resulting ad hoc team agent’s ability to collaborate with more than 40 unknown teams of agents to accomplish a benchmark task. These agents were designed by people other than the authors without these designers planning for the ad hoc teamwork setting. A secondary contribution of the paper is a new transfer learning algorithm, TwoStageTransfer, that can improve results when the ad hoc team agent does have some limited observations of its current teammates.

AAMAS Conference 2011 Conference Paper

ADAPT: Abstraction Hierarchies to Succinctly Model Teamwork

• Meirav Hadad
• Avi Rosenfeld

In this paper we present a lightweight teamwork implementation through use of abstraction hierarchies. The basis of this implementation is ADAPT, which supports A utonomous D ynamic A gent P lanning for T eamwork. ADAPT's novelty stems from how it succinctly decomposes teamwork problems into two separate planners: a task network for the set of activities to be performed by a specific agent and a separate group network for addressing team organization factors. Because abstract search techniques are the basis for creating these two components, ADAPT agents are able to effectively address teamwork in dynamic environments without explicitly enumerating the entire set of possible team states. During run-time, ADAPT agents then expand the teamwork states that are necessary for task completion through an association algorithm to dynamically link its task and group planners. As a result, ADAPT uses far fewer team states than existing teamwork models. We describe how ADAPT was implemented within a commercial training and simulation application, and present evidence detailing its success in concisely and effectively modeling teamwork.

AAAI Conference 2011 Conference Paper

Identifying Missing Node Information in Social Networks

• Ron Eyal
• Sarit Kraus
• Avi Rosenfeld

In recent years, social networks have surged in popularity as one of the main applications of the Internet. This has generated great interest in researching these networks by various fields in the scientific community. One key aspect of social network research is identifying important missing information which is not explicitly represented in the network, or is not visible to all. To date, this line of research typically focused on what connections were missing between nodes, or what is termed the "Missing Link Problem". This paper introduces a new Missing Nodes Identification problem where missing members in the social network structure must be identified. Towards solving this problem, we present an approach based on clustering algorithms combined with measures from missing link research. We show that this approach has beneficial results in the missing nodes identification process and we measure its performance in several different scenarios.

AAMAS Conference 2011 Conference Paper

Using Aspiration Adaptation Theory to Improve Learning

• Avi Rosenfeld
• Sarit Kraus

Creating agents that properly simulate and interact with people is critical for many applications. Towards creating these agents, models are needed that quickly and accurately predict how people behave in a variety of domains and problems. This paper explores how one bounded rationality theory, Aspiration Adaptation Theory (AAT), can be used to aid in this task. We extensively studied two types of problems - a relatively simple optimization problem and two complex negotiation problems. We compared the predictive capabilities of traditional learning methods with those where we added key elements of AAT and other optimal and bounded rationality models. Within the extensive empirical studies we conducted, we found that machine learning models combined with AAT were most effective in quickly and accurately predicting people's behavior.

JAAMAS Journal 2010 Journal Article

Modeling agents based on aspiration adaptation theory

• Avi Rosenfeld
• Sarit Kraus

Abstract Creating agents that realistically simulate and interact with people is an important problem. In this paper we present strong empirical evidence that such agents should be based on bounded rationality, and specifically on key elements from Aspiration Adaptation Theory (AAT). First, we analyzed the strategies people described they would use to solve two relatively basic optimization problems involving one and two parameters. Second, we studied the agents a different group of people wrote to solve these same problems. We then studied two realistic negotiation problems involving five and six parameters. Again, first we studied the negotiation strategies people used when interacting with other people. Then we studied two state of the art automated negotiation agents and negotiation sessions between these agents and people. We found that in both the optimizing and negotiation problems the overwhelming majority of automated agents and people used key elements from AAT, even when optimal solutions, machine learning techniques for solving multiple parameters, or bounded techniques other than AAT could have been implemented. We discuss the implications of our findings including suggestions for designing more effective agents for game and simulation environments.

IJCAI Conference 2009 Conference Paper

• Avi Rosenfeld
• Sarit Kraus

Effectively modeling an agent’s cognitive model is an important problem in many domains. In this paper, we explore the agents people wrote to operate within optimization problems. We claim that the overwhelming majority of these agents used strategies based on bounded rationality, even when optimal solutions could have been implemented. Particularly, we believe that many elements from Aspiration Adaptation Theory (AAT) are useful in quantifying these strategies. To support these claims, we present extensive empirical results from over a hundred agents programmed to perform in optimization problems involving solving for one and two variables.

KER Journal 2009 Journal Article

A Study of Dynamic Coordination Mechanisms

• Avi Rosenfeld

AIJ Journal 2008 Journal Article

A study of mechanisms for improving robotic group performance

• Avi Rosenfeld
• Gal A. Kaminka
• Sarit Kraus
• Onn Shehory

Many collaborative multi-robot application domains have limited areas of operation that cause spatial conflicts between robotic teammates. These spatial conflicts can cause the team's productivity to drop with the addition of robots. This phenomenon is impacted by the coordination methods used by the team-members, as different coordination methods yield radically different productivity results. However, selecting the best coordination method to be used by teammates is a formidable task. This paper presents techniques for creating adaptive coordination methods to address this challenge. We first present a combined coordination cost measure, CCC, to quantify the cost of group interactions. Our measure is useful for facilitating comparison between coordination methods, even when multiple cost factors are considered. We consistently find that as CCC values grow, group productivity falls. Using the CCC, we create adaptive coordination techniques that are able to dynamically adjust the efforts spent on coordination to match the number of perceived coordination conflicts in a group. We present two adaptation heuristics that are completely distributed and require no communication between robots. Using these heuristics, robots independently estimate their combined coordination cost (CCC), adjust their coordination methods to minimize it, and increase group productivity. We use simulated robots to perform thousands of experiment trials to demonstrate the efficacy of our approach. We show that using adaptive coordination methods create a statistically significant improvement in productivity over static methods, regardless of the group size.

IJCAI Conference 2005 Conference Paper

Measuring the Cost of Robotic Communication

• Avi Rosenfeld
• Gal A Kaminka
• Sarit

ECAI Conference 2004 Conference Paper

Adaptive Robot Coordination Using Interference Metrics

• Avi Rosenfeld
• Gal A. Kaminka
• Sarit Kraus