Iyad Rahwan

JAAMAS Journal 2026 Journal Article

Guest Editorial: Argumentation in Multi-Agent Systems

• Iyad Rahwan

AAMAS Conference 2026 Conference Paper

The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems

• Prateek Gupta
• Qiankun Zhong
• Hiromu Yakura
• Thomas Eisenmann
• Iyad Rahwan

A growing body of multi-agent studies with Large Language Models (LLMs) explores how norms and cooperation emerge in mixedmotive scenarios, where pursuing individual gain can undermine the collective good. While prior work has explored these dynamics in both richly contextualized simulations and simplified gametheoretic environments, most LLM systems featuring common-pool resource(CPR)gamesprovideagentswithexplicitrewardfunctions directly tied to their actions. In contrast, human cooperation often emerges without explicit knowledge of the payoff structure or how individual actions translate into long-run outcomes, relying instead on heuristics, communication, and enforcement. We introduce a CPRsimulationframeworkthatremovesexplicitrewardsignalsand embeds cultural-evolutionary mechanisms: social learning (adopting strategies and beliefs from successful peers) and norm-based punishment, grounded in Ostrom’s principles of resource governance. Agents also individually learn from the consequences of harvesting, monitoring, and punishing via environmental feedback, enabling norms to emerge endogenously. We establish the validity of our simulation by reproducing key findings from existing studies on human behavior. Building on this, we examine norm evolution across a 2 × 2 grid of environmental and social initialisations (resource-rich vs. resource-scarce; altruistic vs. selfish) and benchmark how agentic societies comprised of different LLMs perform under these conditions. Our results reveal systematic model differences in sustaining cooperation and norm formation, positioning the framework as a rigorous testbed for studying emergent norms in mixed-motive LLM societies. Such analysis can inform the design of AI systems deployed in social and organizational contexts, where alignment with cooperative norms is critical for stability, fairness, and effective governance of AI-mediated environments. ∗Equal contribution This work is licensed under a Creative Commons Attribution International 4. 0 License. Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026), C. Amato, L. Dennis, V. Mascardi, J. Thangarajah (eds.), May 25 – 29, 2026, Paphos, Cyprus. © 2026 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org). https: //doi. org/10. 65109/CZDC3237 Iyad Rahwan Center for Humans and Machines Max-Planck Institute for Human Development Berlin, Germany

AAMAS Conference 2025 Conference Paper

When Is It Acceptable to Break the Rules? Knowledge Representation of Moral Judgements Based on Empirical Data (Extended Abstract)

• Edmond Awad
• Sydney Levine
• Andrea Loreggia
• Nicholas Mattei
• Iyad Rahwan
• Francesca Rossi
• Kartik Talamadupula
• Joshua Tenenbaum

This paper explores how humans make contextual moral judgments to inform the development of AI systems capable of balancing rulefollowing with flexibility. We investigate the limitations of rigid constraints in AI, which can hinder morally acceptable actions in specific contexts, unlike humans who can override rules when appropriate. We propose a preference-based graphical model inspired by dual-process theories of moral judgment and conduct a study on human decisions about breaking the social norm of "no cutting in line. " Our model outperforms standard machine learning methods in predicting human judgments and offers a generalizable framework for modeling moral decision-making across various contexts. This short paper summarizes the main findings of our paper published in the journal Autonomous Agents and Multi-Agent Systems. [2]

JAIR Journal 2021 Journal Article

Superintelligence Cannot be Contained: Lessons from Computability Theory

• Manuel Alfonseca
• Manuel Cebrian
• Antonio Fernandez Anta
• Lorenzo Coviello
• Andrés Abeliuk
• Iyad Rahwan

Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potentially catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that total containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) impossible. This article is part of the special track on AI and Society.

AAAI Conference 2018 Conference Paper

A Voting-Based System for Ethical Decision Making

• Ritesh Noothigattu
• Snehalkumar Gaikwad
• Edmond Awad
• Sohan Dsouza
• Iyad Rahwan
• Pradeep Ravikumar
• Ariel Procaccia

We present a general approach to automating ethical decisions, drawing on machine learning and computational social choice. In a nutshell, we propose to learn a model of societal preferences, and, when faced with a specific ethical dilemma at runtime, efficiently aggregate those preferences to identify a desirable choice. We provide a concrete algorithm that instantiates our approach; some of its crucial steps are informed by a new theory of swap-dominance efficient voting rules. Finally, we implement and evaluate a system for ethical decision making in the autonomous vehicle domain, using preference data collected from 1. 3 million people through the Moral Machine website.

IJCAI Conference 2018 Conference Paper

TuringBox: An Experimental Platform for the Evaluation of AI Systems

• Ziv Epstein
• Blakeley H. Payne
• Judy Hanwen Shen
• Casey Jisoo Hong
• Bjarke Felbo
• Abhimanyu Dubey
• Matthew Groh
• Nick Obradovich

We introduce TuringBox, a platform to democratize the study of AI. On one side of the platform, AI contributors upload existing and novel algorithms to be studied scientifically by others. On the other side, AI examiners develop and post machine intelligence tasks to evaluate and characterize the outputs of algorithms. We outline the architecture of such a platform, and describe two interactive case studies of algorithmic auditing on the platform.

KR Conference 2014 Short Paper

Interval Methods for Judgment Aggregation in Argumentation

• Richard Booth
• Edmond Awad
• Iyad Rahwan

In the present paper, we embark on a broader study of JA in argumentation. We define a general family of aggregation operators called interval methods and show that they contain existing operators as instances. Interval methods always satisfy a strong version of Independence, but will usually fail Collective Rationality. But despite this important barrier, we are able to fully axiomatize interval methods in terms of a set of fundamental postulates. Then, building on Caminada and Pigozzi’s down-admissible + up-complete (DAUC) construction, we present an approach to transform any interval method into one satisfying Collective Rationality while preserving a weaker and more reasonable form of independence known as Directionality. Given a set of conflicting arguments, there can exist multiple plausible opinions about which arguments should be accepted, rejected, or deemed undecided. Recent work explored some operators for deciding how multiple such judgments should be aggregated. Here, we generalize this line of study by introducing a family of operators called interval aggregation methods, which contain existing operators as instances. While these methods fail to output a complete labelling in general, we show that it is possible to transform a given aggregation method into one that does always yield collectively rational labellings. This employs the downadmissible and up-complete constructions of Caminada and Pigozzi. For interval methods, collective rationality is attained at the expense of a strong Independence postulate, but we show that an interesting weakening of the Independence postulate is retained. Preliminaries We assume a countably infinite set U of argument names, from which all possible argumentation frameworks are built. Definition 1 An argumentation framework (AF for short) A “ pArgs, áq is a pair consisting of a finite set Args Ď U of arguments and an attack relation áĎ Args ˆ Args. Sometimes we use Args A and áA to denote the arguments and attack relation of a given AF A.

AAMAS Conference 2012 Conference Paper

A Storage Pricing Mechanism for Learning Agents in the Masdar City Smart Grid

• Fatimah Ishowo-Oloko
• Perukrishnen Vytelingum
• NICK JENNINGS
• Iyad Rahwan

Masdar City in the United Arab Emirates is designed to be the first modern city powered solely by renewable energy. However, the stochastic nature of renewable energy generators has remained a major challenge in their sole and largescale deployment. Traditional approaches couple large-scale storage systems to renewable generators while more recent approaches also study how emerging technologies such as electric vehicles and micro-batteries can be used as consumerside storage. Future smart grids are however likely to contain both forms of storage. We present a novel model of joint-storage management that allows both renewable energy suppliers and consumers to coordinate in a decentralized manner by gradually adopting storage abilities. For this model, we present a dynamic storage-pricing mechanism that makes use of the storage information from the renewable supplier to generate daily, real-time electricity prices which are communicated to the consumers.

AAMAS Conference 2012 Conference Paper

Quantifying Disagreement in Argument-based Reasoning

• Richard Booth
• Martin Caminada
• Mikolaj Mikołaj
• Iyad Rahwan

An argumentation framework can be seen as expressing, in an abstract way, the conflicting information of an underlying logical knowledge base. This conflicting information often allows for the presence of more than one possible reasonable position (extension/labelling) which one can take. A relevant question, therefore, is how much these positions differ from each other. In the current paper, we will examine the issue of how to define meaningful measures of distance between the (complete) labellings of a given argumentation framework. We provide concrete distance measures based on argument-wise label difference, as well as based on the notion of critical sets, and examine their properties.

KER Journal 2011 Journal Article

Logical mechanism design

• Iyad Rahwan
• Kate Larson

Abstract Game theory is becoming central to the design and analysis of computational mechanisms in which multiple entities interact strategically. The tools of mechanism design are used extensively to engineer incentives for truth revelation into resource allocation (e.g. combinatorial auctions) and preference aggregation protocols (e.g. voting). We argue that mechanism design can also be useful in the design of logical inference procedures. In particular, it can help us understand and engineer inference procedures when knowledge is distributed among self-interested agents. We set a research agenda for this emerging area, and point to some early research efforts.

KER Journal 2011 Journal Article

Representing and classifying arguments on the Semantic Web

• Iyad Rahwan
• Bita Banihashemi
• Chris Reed
• Douglas Walton
• Sherief Abdallah

Abstract Until recently, little work has been dedicated to the representation and interchange of informal, semi-structured arguments of the type found in natural language prose and dialogue. To redress this, the research community recently initiated work towards an Argument Interchange Format (AIF). The AIF aims to facilitate the exchange of semi-structured arguments among different argument analysis and argumentation-support tools. In this paper, we present a Description Logic ontology for annotating arguments, based on a new reification of the AIF and founded in Walton's theory of argumentation schemes. We demonstrate how this ontology enables a new kind of automated reasoning over argument structures, which complements classical reasoning about argument acceptability. In particular, Web Ontology Language reasoning enables significantly enhanced querying of arguments through automatic scheme classifications, instance classification, inference of indirect support in chained argument structures, and inference of critical questions. We present the implementation of a pilot Web-based system for authoring and querying argument structures using the proposed ontology.

AAMAS Conference 2010 Conference Paper

Agreeing on Plans Through Iterated Disputes

• Alexandros Belesiotis
• Michael Rovatsos
• Iyad Rahwan

Autonomous agents transcend their individual capabilities by cooperating towards achieving shared goals. The different viewpoints agents have on the environment cause disagreements about the anticipated effects of plans. Reaching agreement requires the resolution of such inconsistencies and the alignment of the agents' viewpoints. We present a dialogue protocol that enables agents to discuss candidate plans and reach agreements. The dialogue is based on an argumentation process in the language of situation calculus. Agreement is reached through persuasion, thereby aligning the planning beliefs of the agents. We describe our abstract iterated dialogue protocol, and extend it for the specific problem of arguing about plans. We show that our method always terminates and produces sound results. Furthermore, we detail a set of heuristics to simplify reasoning and reduce the exchanged information.

AAMAS Conference 2010 Conference Paper

Collective Argument Evaluation as Judgement Aggregation

• Iyad Rahwan
• Fernando Tohme

A conflicting knowledge base can be seen abstractly as aset of arguments and a binary relation characterising conflict among them. There may be multiple plausible waysto evaluate conflicting arguments. In this paper, we ask: {\em given a set of agents, each with a legitimate subjective evaluation of a set of arguments, how can they reach a collectiveevaluation of those arguments? } After formally defining thisproblem, we extensively analyse an argument-wise pluralityvoting rule, showing that it suffers a fundamental limitation. Then we demonstrate, through a general impossibilityresult, that this limitation is more fundamentally rooted. Finally, we show how this impossibility result can be circumvented by additional domain restrictions.

IJCAI Conference 2009 Conference Paper

• Iyad Rahwan
• Kate Larson
• Fernando Tohmé

Recently, Argumentation Mechanism Design (ArgMD) was introduced as a new paradigm for studying argumentation among self-interested agents using game-theoretic techniques. Preliminary results showed a condition under which a direct mechanism based on Dung’s grounded semantics is strategy-proof (i. e. truth enforcing). But these early results dealt with a highly restricted form of agent preferences, and assumed agents can only hide, but not lie about, arguments. In this paper, we characterise strategy-proofness under grounded semantics for a more realistic preference class (namely, focal arguments). We also provide the first analysis of the case where agents can lie.

AIJ Journal 2009 Journal Article

Dialogue games that agents play within a society

• Nishan C. Karunatillake
• Nicholas R. Jennings
• Iyad Rahwan
• Peter McBurney

Human societies have long used the capability of argumentation and dialogue to overcome and resolve conflicts that may arise within their communities. Today, there is an increasing level of interest in the application of such dialogue games within artificial agent societies. In particular, within the field of multi-agent systems, this theory of argumentation and dialogue games has become instrumental in designing rich interaction protocols and in providing agents with a means to manage and resolve conflicts. However, to date, much of the existing literature focuses on formulating theoretically sound and complete models for multi-agent systems. Nonetheless, in so doing, it has tended to overlook the computational implications of applying such models in agent societies, especially ones with complex social structures. Furthermore, the systemic impact of using argumentation in multi-agent societies and its interplay with other forms of social influences (such as those that emanate from the roles and relationships of a society) within such contexts has also received comparatively little attention. To this end, this paper presents a significant step towards bridging these gaps for one of the most important dialogue game types; namely argumentation-based negotiation (ABN). The contributions are three fold. First, we present a both theoretically grounded and computationally tractable ABN framework that allows agents to argue, negotiate, and resolve conflicts relating to their social influences within a multi-agent society. In particular, the model encapsulates four fundamental elements: (i) a scheme that captures the stereotypical pattern of reasoning about rights and obligations in an agent society, (ii) a mechanism to use this scheme to systematically identify social arguments to use in such contexts, (iii) a language and a protocol to govern the agent interactions, and (iv) a set of decision functions to enable agents to participate in such dialogues. Second, we use this framework to devise a series of concrete algorithms that give agents a set of ABN strategies to argue and resolve conflicts in a multi-agent task allocation scenario. In so doing, we exemplify the versatility of our framework and its ability to facilitate complex argumentation dialogues within artificial agent societies. Finally, we carry out a series of experiments to identify how and when argumentation can be useful for agent societies. In particular, our results show: a clear inverse correlation between the benefit of arguing and the resources available within the context; that when agents operate with imperfect knowledge, an arguing approach allows them to perform more effectively than a non-arguing one; that arguing earlier in an ABN interaction presents a more efficient method than arguing later in the interaction; and that allowing agents to negotiate their social influences presents both an effective and an efficient method that enhances their performance within a society.

JAAMAS Journal 2008 Journal Article

Intentional learning agent architecture

• Budhitama Subagdja
• Liz Sonenberg
• Iyad Rahwan

Abstract Dealing with changing situations is a major issue in building agent systems. When the time is limited, knowledge is unreliable, and resources are scarce, the issue becomes more challenging. The BDI (Belief-Desire-Intention) agent architecture provides a model for building agents that addresses that issue. The model can be used to build intentional agents that are able to reason based on explicit mental attitudes, while behaving reactively in changing circumstances. However, despite the reactive and deliberative features, a classical BDI agent is not capable of learning. Plans as recipes that guide the activities of the agent are assumed to be static. In this paper, an architecture for an intentional learning agent is presented. The architecture is an extension of the BDI architecture in which the learning process is explicitly described as plans. Learning plans are meta-level plans which allow the agent to introspectively monitor its mental states and update other plans at run time. In order to acquire the intricate structure of a plan, a process pattern called manipulative abduction is encoded as a learning plan. This work advances the state of the art by combining the strengths of learning and BDI agent frameworks in a rich language for describing deliberation processes and reactive execution. It enables domain experts to specify learning processes and strategies explicitly, while allowing the agent to benefit from procedural domain knowledge expressed in plans.

AAMAS Conference 2008 Conference Paper

Mechanism Design for Abstract Argumentation

• Iyad Rahwan
• Kate Larson

Since their introduction by Dung over a decade ago, abstract argumentation frameworks have received increasing interest in artificial intelligence as a convenient model for reasoning about general characteristics of argument. Such a framework consists of a set of arguments and a binary defeat relation among them. Various semantic and computational approaches have been developed to characterise the acceptability of individual arguments in a given argumentation framework. However, little work exists on understanding the strategic aspects of abstract argumentation among self-interested agents. In this paper, we introduce (game-theoretic) argumentation mechanism design (ArgMD), which enables the design and analysis of argumentation mechanisms for self-interested agents. We define the notion of a direct-revelation argumentation mechanism, in which agents must decide which arguments to reveal simultaneously. We then design a particular direct argumentation mechanism and prove that it is strategy proof under specific conditions; that is, the strategy profile in which each agent reveals its arguments truthfully is a dominant strategy equilibrium.

AAAI Conference 2008 Conference Paper

Pareto Optimality in Abstract Argumentation

• Iyad Rahwan

Since its introduction in the mid-nineties, Dung’s theory of abstract argumentation frameworks has been influential in artificial intelligence. Dung viewed arguments as abstract entities with a binary defeat relation among them. This enabled extensive analysis of different (semantic) argument acceptance criteria. However, little attention was given to comparing such criteria in relation to the preferences of selfinterested agents who may have conflicting preferences over the final status of arguments. In this paper, we define a number of agent preference relations over argumentation outcomes. We then analyse different argument evaluation rules taking into account the preferences of individual agents. Our framework and results inform the mediator (e. g. judge) to decide which argument evaluation rule (i. e. semantics) to use given the type of agent population involved.

IS Journal 2007 Journal Article

Guest Editors' Introduction: Argumentation Technology

• Iyad Rahwan
• Peter McBurney

This introduction to the special issue on argumentation technology discusses argumentation's role in modern computing. It also identifies the challenges that the community must meet before it can achieve widespread deployment of argumentation technologies.

AIJ Journal 2007 Journal Article

Laying the foundations for a World Wide Argument Web

• Iyad Rahwan
• Fouad Zablith
• Chris Reed

This paper lays theoretical and software foundations for a World Wide Argument Web (WWAW): a large-scale Web of inter-connected arguments posted by individuals to express their opinions in a structured manner. First, we extend the recently proposed Argument Interchange Format (AIF) to express arguments with a structure based on Walton's theory of argumentation schemes. Then, we describe an implementation of this ontology using the RDF Schema Semantic Web-based ontology language, and demonstrate how our ontology enables the representation of networks of arguments on the Semantic Web. Finally, we present a pilot Semantic Web-based system, ArgDF, through which users can create arguments using different argumentation schemes and can query arguments using a Semantic Web query language. Manipulation of existing arguments is also handled in ArgDF: users can attack or support parts of existing arguments, or use existing parts of an argument in the creation of new arguments. ArgDF also enables users to create new argumentation schemes. As such, ArgDF is an open platform not only for representing arguments, but also for building interlinked and dynamic argument networks on the Semantic Web. This initial public-domain tool is intended to seed a variety of future applications for authoring, linking, navigating, searching, and evaluating arguments on the Web.

AAAI Conference 2007 Conference Paper

On the Benefits of Exploiting Underlying Goals in Argument-based Negotiation

• Iyad Rahwan
• Liz Sonenberg

Interest-based negotiation (IBN) is a form of negotiation in which agents exchange information about their underlying goals, with a view to improving the likelihood and quality of a deal. While this intuition has been stated informally in much previous literature, there is no formal analysis of the types of deals that can be reached through IBN and how they differ from those reachable using (classical) alternating offer bargaining. This paper bridges this gap by providing a formal framework for analysing the outcomes of IBN dialogues, and begins by analysing a specific IBN protocol.

AAAI Conference 2007 Conference Paper

Towards Large Scale Argumentation Support on the Semantic Web

• Iyad Rahwan

This paper lays theoretical and software foundations for a World Wide Argument Web (WWAW): a large-scale Web of inter-connected arguments posted by individuals to express their opinions in a structured manner. First, we extend the recently proposed Argument Interchange Format (AIF) to express arguments with a structure based on Walton’s theory of argumentation schemes. Then, we describe an implementation of this ontology using the RDF Schema language, and demonstrate how our ontology enables the representation of networks of arguments on the Semantic Web. Finally, we present a pilot Semantic Web-based system, ArgDF, through which users can create arguments using different argumentation schemes and can query arguments using a Semantic Web query language. Users can also attack or support parts of existing arguments, use existing parts of an argument in the creation of new arguments, or create new argumentation schemes. As such, this initial public-domain tool is intended to seed a variety of future applications for authoring, linking, navigating, searching, and evaluating arguments on the Web.

AAMAS Conference 2006 Conference Paper

An Argumentation-based Approach for Practical Reasoning

• Iyad Rahwan
• Leila Amgoud

AAMAS Conference 2006 Conference Paper

Managing Social Influences through Argumentation-Based Negotiation

• Nishan Karunatillake
• NICK JENNINGS
• Iyad Rahwan
• Sarvapali Ramchurn

KER Journal 2006 Journal Article

Towards an argument interchange format

• CARLOS CHESÑEVAR
• MCGINNIS
• Sanjay Modgil
• Iyad Rahwan
• Chris Reed
• GUILLERMO SIMARI
• MATTHEW SOUTH
• GERARD VREESWIJK

The theory of argumentation is a rich, interdisciplinary area of research straddling the fields of artificial intelligence, philosophy, communication studies, linguistics and psychology. In the last few years, significant progress has been made in understanding the theoretical properties of different argumentation logics. However, one major barrier to the development and practical deployment of argumentation systems is the lack of a shared, agreed notation or ‘interchange format’ for argumentation and arguments. In this paper, we describe a draft specification for an argument interchange format (AIF) intended for representation and exchange of data between various argumentation tools and agent-based applications. It represents a consensus ‘abstract model’ established by researchers across fields of argumentation, artificial intelligence and multi-agent systems. In its current form, this specification is intended as a starting point for further discussion and elaboration by the community, rather than an attempt at a definitive, all-encompassing model. However, to demonstrate proof of concept, a use case scenario is briefly described. Moreover, three concrete realizations or ‘reifications’ of the abstract model are illustrated.

AAMAS Conference 2005 Conference Paper

ArgumentBased Negotiation in a Social Context

• Nishan Karunatillake
• Nicholas Jennings
• Iyad Rahwan
• Timothy Norman

KER Journal 2003 Journal Article

Argumentation-based negotiation

• Iyad Rahwan
• Sarvapali D. Ramchurn
• Nicholas R. Jennings
• Peter McBurney
• Simon Parsons
• Liz Sonenberg

Negotiation is essential in settings where autonomous agents have conflicting interests and a desire to cooperate. For this reason, mechanisms in which agents exchange potential agreements according to various rules of interaction have become very popular in recent years as evident, for example, in the auction and mechanism design community. However, a growing body of research is now emerging which points out limitations in such mechanisms and advocates the idea that agents can increase the likelihood and quality of an agreement by exchanging arguments which influence each others' states. This community further argues that argument exchange is sometimes essential when various assumptions about agent rationality cannot be satisfied. To this end, in this article, we identify the main research motivations and ambitions behind work in the field. We then provide a conceptual framework through which we outline the core elements and features required by agents engaged in argumentation-based negotiation, as well as the environment that hosts these agents. For each of these elements, we survey and evaluate existing proposed techniques in the literature and highlight the major challenges that need to be addressed if argument-based negotiation research is to reach its full potential.

AAMAS Conference 2003 Conference Paper

Towards Interest-Based Negotiation

• Iyad Rahwan
• Liz Sonenberg
• Frank Dignum