Marija Slavkovik

AAMAS Conference 2025 Conference Paper

Contesting Black-Box AI Decisions

• Virginia Dignum
• Loizos Michael
• Juan Carlos Nieves
• Marija Slavkovik
• Julliett Suarez
• Andreas Theodorou

The “right to contest” decisions that have consequences on individuals or the society is a well-established democratic right. Contesting a decision is not a matter of simply providing an explanation, but rather of assessing whether the decision and the explanation are permissible against an organization’s governance framework. Yet, albeit the popularity of adjacent fields, little work has been explicitly done on contesting AI decisions. In this paper, we propose that formal argumentation can be used to formulate contestations of decisions made by artificial agents. We extend the discourse on socio-ethical values in AI by conceptualizing our argumentation framework as a formal dialogue, enabling the interaction between humans and agents as decisions are being contested.

EUMAS Conference 2025 Short Paper

Designing Value-Aligned Traffic Agents through Conflict Sensitivity

• Astrid Rakow
• Maike Schwammberger
• Marija Slavkovik
• Gleifer Vaz Alves
• Joe Collenette

JAAMAS Journal 2024 Journal Article

Finding middle grounds for incoherent horn expressions: the moral machine case

• Ana Ozaki
• Anum Rehman
• Marija Slavkovik

Abstract Smart devices that operate in a shared environment with people need to be aligned with their values and requirements. We study the problem of multiple stakeholders informing the same device on what the right thing to do is. Specifically, we focus on how to reach a middle ground among the stakeholders inevitably incoherent judgments on what the rules of conduct for the device should be. We formally define a notion of middle ground and discuss the main properties of this notion. Then, we identify three sufficient conditions on the class of Horn expressions for which middle grounds are guaranteed to exist. We provide a polynomial time algorithm that computes middle grounds, under these conditions. We also show that if any of the three conditions is removed then middle grounds for the resulting (larger) class may not exist. Finally, we implement our algorithm and perform experiments using data from the Moral Machine Experiment. We present conflicting rules for different countries and how the algorithm finds the middle ground in this case.

LAMAS&SR Workshop 2023 Workshop Paper

Probabilistic Judgment Aggregation with Conditional Independence Constraints

• Magdalena Ivanovska
• Marija Slavkovik

Probabilistic judgment aggregation is concerned with aggregating judgments about probabilities of logically related issues. It takes as input imprecise probabilistic judgments over the issues given by a group of agents and defines rules of aggregating the individual judgments into a collective opinion representative for the group. The process of aggregation can be subject to constraints, i.e., aggregation rules can be required to satisfy certain properties. We explore how probabilistic independence constraints can be incorporated into the aggregation process.

JAIR Journal 2023 Journal Article

The Jiminy Advisor: Moral Agreements among Stakeholders Based on Norms and Argumentation

• Beishui Liao
• Pere Pardo
• Marija Slavkovik
• Leendert van der Torre

An autonomous system is constructed by a manufacturer, operates in a society subject to norms and laws, and interacts with end users. All of these actors are stakeholders affected by the behavior of the autonomous system. We address the challenge of how the ethical views of such stakeholders can be integrated in the behavior of an autonomous system. We propose an ethical recommendation component called Jiminy which uses techniques from normative systems and formal argumentation to reach moral agreements among stakeholders. A Jiminy represents the ethical views of each stakeholder by using normative systems, and has three ways of resolving moral dilemmas that involve the opinions of the stakeholders. First, the Jiminy considers how the arguments of the stakeholders relate to one another, which may already resolve the dilemma. Secondly, the Jiminy combines the normative systems of the stakeholders such that the combined expertise of the stakeholders may resolve the dilemma. Thirdly, and only if these two other methods have failed, the Jiminy uses context-sensitive rules to decide which of the stakeholders take preference over the others. At the abstract level, these three methods are characterized by adding arguments, adding attacks between arguments, and revising attacks between arguments. We show how a Jiminy can be used not only for ethical reasoning and collaborative decision-making, but also to provide explanations about ethical behavior.

AIJ Journal 2022 Journal Article

AI Journal Special Issue on Ethics for Autonomous Systems

• Michael Fisher
• Sven Koenig
• Marija Slavkovik

AAMAS Conference 2021 Conference Paper

Egalitarian Judgment Aggregation

• Sirin Botan
• Ronald de Haan
• Marija Slavkovik
• Zoi Terzopoulou

Egalitarian considerations play a central role in many areas of social choice theory. Applications of egalitarian principles range from ensuring everyone gets an equal share of a cake when deciding how to divide it, to guaranteeing balance with respect to gender or ethnicity in committee elections. Yet, the egalitarian approach has received little attention in judgment aggregation—a powerful framework for aggregating logically interconnected issues. We make the first steps towards filling that gap. We introduce axioms capturing two classical interpretations of egalitarianism in judgment aggregation and situate these within the context of existing axioms in the pertinent framework of belief merging. We then explore the relationship between these axioms and several notions of strategyproofness from social choice theory at large. Finally, a novel egalitarian judgment aggregation rule stems from our analysis; we present complexity results concerning both outcome determination and strategic manipulation for that rule.

JAIR Journal 2020 Journal Article

The Complexity Landscape of Outcome Determination in Judgment Aggregation

• Ulle Endriss
• Ronald de Haan
• Jérôme Lang
• Marija Slavkovik

We provide a comprehensive analysis of the computational complexity of the outcome determination problem for the most important aggregation rules proposed in the literature on logic-based judgment aggregation. Judgment aggregation is a powerful and flexible framework for studying problems of collective decision making that has attracted interest in a range of disciplines, including Legal Theory, Philosophy, Economics, Political Science, and Artificial Intelligence. The problem of computing the outcome for a given list of individual judgments to be aggregated into a single collective judgment is the most fundamental algorithmic challenge arising in this context. Our analysis applies to several different variants of the basic framework of judgment aggregation that have been discussed in the literature, as well as to a new framework that encompasses all existing such frameworks in terms of expressive power and representational succinctness.

IJCAI Conference 2019 Conference Paper

Answer Set Programming for Judgment Aggregation

• Ronald de Haan
• Marija Slavkovik

Judgment aggregation (JA) studies how to aggregate truth valuations on logically related issues. Computing the outcome of aggregation procedures is notoriously computationally hard, which is the likely reason that no implementation of them exists as of yet. However, even hard problems sometimes need to be solved. The worst-case computational complexity of answer set programming (ASP) matches that of most problems in judgment aggregation. We take advantage of this and propose a natural and modular encoding of various judgment aggregation procedures and related problems in JA into ASP. With these encodings, we achieve two results: (1) paving the way towards constructing a wide range of new benchmark instances (from JA) for answer set solving algorithms; and (2) providing an automated tool for researchers in the area of judgment aggregation.

AAMAS Conference 2017 Conference Paper

Complexity Results for Aggregating Judgments using Scoring or Distance-Based Procedures

• Ronald de Haan
• Marija Slavkovik

Judgment aggregation is an abstract framework for studying collective decision making by aggregating individual opinions on logically related issues. Important types of judgment aggregation methods are those of scoring and distance-based methods, many of which can be seen as generalisations of voting rules. An important question to investigate for judgment aggregation methods is how hard it is to find a collective decision by applying these methods. In this article we study the complexity of this“winner determination”problem for some scoring and distance-based judgment aggregation procedures. Such procedures aggregate judgments by assigning values to judgment sets. Our work fills in some of the last gaps in the complexity landscape for winner determination in judgment aggregation. Our results reaffirm that aggregating judgments is computationally hard and strongly point towards the necessity of analyzing approximation methods or parameterized algorithms in judgment aggregation. CCS Concepts •Theory of computation → Problems, reductions and completeness; •Computing methodologies → Artificial intelligence;

AAAI Conference 2016 Conference Paper

Agenda Separability in Judgment Aggregation

• Jérôme Lang
• Marija Slavkovik
• Srdjan Vesic

One of the better studied properties for operators in judgment aggregation is independence, which essentially dictates that the collective judgment on one issue should not depend on the individual judgments given on some other issue(s) in the same agenda. Independence, although considered a desirable property, is too strong, because together with mild additional conditions it implies dictatorship. We propose here a weakening of independence, named agenda separability: a judgment aggregation rule satisfies it if, whenever the agenda is composed of several independent sub-agendas, the resulting collective judgment sets can be computed separately for each sub-agenda and then put together. We show that this property is discriminant, in the sense that among judgment aggregation rules so far studied in the literature, some satisfy it and some do not. We briefly discuss the implications of agenda separability on the computation of judgment aggregation rules.

AAMAS Conference 2012 Conference Paper

Distance-based Rules for Weighted Judgment Aggregation

• Marija Slavkovik
• Wojciech Jamroga

Cooperating agents need to reach group decisions on several logically related issues. These decision-making problems are studied in social choice theory by the discipline of judgment aggregation. Judgment aggregation produces group decisions by aggregating individual answers to binary questions, however existing aggregation rules are defined for a very restricted setting, insufficient for aggregating opinions in a computer science contexts. We propose a family of distance-based judgment aggregation rules and study their properties.

AAMAS Conference 2012 Conference Paper

Selecting judgment aggregation rules for NAO robots: an experimental approach

• Vijayalakshmi Ganesan
• Marija Slavkovik
• Sergio Sousa
• Leendert van der Torre

Social choice rules can be used to reach group decisions in multiagent systems. We consider judgment aggregation, the problem of aggregating answers to binary logically related questions. In general ”fairness” is usually considered to be the main concern when selecting a social choice rule, however we believe that in judgment aggregation often a more relevant property is how efficient the rule is in truth tracking, that is, how often does it return the correct answer to the binary questions. Whereas “fairness” can be studied axiomatically, truth tracking efficiency needs to be studied experimentally. We accomplish the experimental analysis by constructing a multi-robot system.