Maria Chli

EUMAS Conference 2025 Short Paper

Strategic Infrastructure Design via Multi-Agent Congestion Games with Joint Placement and Pricing

• Niloofar Aminikalibar
• Farzaneh Farhadi
• Maria Chli

AAMAS Conference 2023 Conference Paper

Budget-Feasible Mechanism Design for Cost-Benefit Optimization in Gradual Service Procurement

• Farzaneh Farhadi
• Maria Chli
• Nicholas R. Jennings

We consider a procurement problem where a software agent procures multiple services from self-interested providers with private costs and uncertain reliabilities to complete a budget-limited task before a strict deadline. Over the last decade, several truthful budgetfeasible procurement mechanisms have been developed to extract the true cost information from strategic providers. Most of these mechanisms have focused on maximizing the procurer’s value (e. g. , the task’s success probability), and hence procuring as many services as the budget allows, even if the returned benefit is lower than the incurred cost. In this paper, however, we focus on the more realistic objective of balancing the cost-benefit tradeoff and propose a novel approach for designing budget-feasible mechanisms that invoke services gradually over time and whenever they are cost-optimal. A major barrier to achieving this goal was the strong dependencies among the decision variables caused by budget constraints. We overcome this barrier by proposing a conservative decomposable approximation to budget constraints. This is the first such approximation technique, which opens a path toward designing budget-feasible mechanisms for contingent planning problems.

JAIR Journal 2023 Journal Article

Optimal and Efficient Auctions for the Gradual Procurement of Strategic Service Provider Agents

• Farzaneh Farhadi
• Maria Chli
• Nicholas R. Jennings

We consider an outsourcing problem where a software agent procures multiple services from providers with uncertain reliabilities to complete a computational task before a strict deadline. The service consumer’s goal is to design an outsourcing strategy (defining which services to procure and when) so as to maximize a specific objective function. This objective function can be different based on the consumer’s nature; a socially-focused consumer often aims to maximize social welfare, while a self-interested consumer often aims to maximize its own utility. However, in both cases, the objective function depends on the providers’ execution costs, which are privately held by the self-interested providers and hence may be misreported to influence the consumer’s decisions. For such settings, we develop a unified approach to design truthful procurement auctions that can be used by both socially-focused and, separately, self-interested consumers. This approach benefits from our proposed weighted threshold payment scheme which pays the provably minimum amount to make an auction with a monotone outsourcing strategy incentive compatible. This payment scheme can handle contingent outsourcing plans, where additional procurement happens gradually over time and only if the success probability of the already hired providers drops below a time-dependent threshold. Using a weighted threshold payment scheme, we design two procurement auctions that maximize, as well as two low-complexity heuristic-based auctions that approximately maximize, the consumer’s expected utility and expected social welfare, respectively. We demonstrate the effectiveness and strength of our proposed auctions through both game-theoretical and empirical analysis.

AAMAS Conference 2022 Conference Paper

Fully-Autonomous, Vision-based Traffic Signal Control: From Simulation to Reality

• Deepeka Garg
• Maria Chli
• George Vogiatzis

Ineffective traffic signal control is one of the major causes of congestion in urban road networks. Dynamically changing traffic conditions and live traffic state estimation are fundamental challenges that limit the ability of the existing signal infrastructure in rendering individualized signal control in real-time. We use deep reinforcement learning (DRL) to address these challenges. Due to economic and safety constraints associated with training such agents in the real world, a practical approach is to do so in simulation before deployment. Domain randomisation is an effective technique for bridging the reality gap and ensuring effective transfer of simulation-trained agents to the real world. In this paper, we develop a fully-autonomous, vision-based DRL agent that achieves adaptive signal control in the face of complex, imprecise, and dynamic traffic environments. Our agent uses live visual data (i. e. a stream of real-time RGB footage) from an intersection to extensively perceive and subsequently act upon the traffic environment. Employing domain randomisation, we examine our agent’s generalisation capabilities under varying traffic conditions in both the simulation and the real-world environments. In a diverse validation set independent of training data, our traffic control agent reliably adapted to novel traffic situations and demonstrated a positive transfer to previously unseen real intersections despite being trained entirely in simulation.

AAMAS Conference 2019 Conference Paper

Domain Adaptation for Reinforcement Learning on the Atari

• Thomas Carr
• Maria Chli
• George Vogiatzis

Deep Reinforcement learning is a powerful machine learning paradigm that has had significant success across a wide range of control problems. This success often requires long training times to achieve. Observing that many problems share similarities, it is likely that much of the training done could be redundant if knowledge could be efficiently and appropriately shared across tasks. In this paper we demonstrate a novel adversarial domain adaptation approach to transfer state knowledge between domains and tasks on the Atari game suite. We show how this approach can successfully transfer across very different visual domains of the Atari platform. We focus on semantically related games that involve returning a ball with the user controlled agent. Our experiments demonstrate that our method reduces the number of samples required to successfully train an agent to play an Atari game.

AAMAS Conference 2019 Conference Paper

Traffic3D: A New Traffic Simulation Paradigm

• Deepeka Garg
• Maria Chli
• George Vogiatzis

The field of Deep Reinforcement Learning has evolved significantly over the last few years. However, an important and not yet fullyattained goal is to produce intelligent agents which can be successfully taken out of the laboratory and employed in the real-world. Intelligent agents that are successfully deployable in real-world settings require substantial prior exposure to their intended environments. When this is not practical or possible, the agents benefit from being trained and tested on powerful test-beds, effectively replicating the real-world. To achieve traffic management at an unprecedented level of efficiency, in this work, we demonstrate a significantly richer new traffic simulation environment; Traffic3D, a platform to effectively simulate and evaluate a variety of 3D road traffic scenarios, closely mimicking real-world traffic characteristics, including faithful simulation of individual vehicle behavior, precise physics of movement and photo-realism. In addition to deep reinforcement learning, Traffic3D also facilitates research in several other domains such as imitation learning, learning by interaction, visual question answering, object detection and segmentation, unsupervised representation learning and procedural generation.

IJCAI Conference 2013 Conference Paper

An Intelligent Broker Agent for Energy Trading: An MDP Approach

• Rodrigue T. Kuate
• Minghua He
• Maria Chli
• Hai H. Wang

This paper details the development and evaluation of AstonTAC, an energy broker that successfully participated in the 2012 Power Trading Agent Competition (Power TAC). AstonTAC buys electrical energy from the wholesale market and sells it in the retail market. The main focus of the paper is on the broker’s bidding strategy in the wholesale market. In particular, it employs Markov Decision Processes (MDP) to purchase energy at low prices in a day-ahead power wholesale market, and keeps energy supply and demand balanced. Moreover, we explain how the agent uses Non-Homogeneous Hidden Markov Model (NHHMM) to forecast energy demand and price. An evaluation and analysis of the 2012 Power TAC finals show that AstonTAC is the only agent that can buy energy at low price in the wholesale market and keep energy imbalance low.

AAMAS Conference 2012 Conference Paper

Using the Max-Sum Algorithm for Supply Chain Formation in Dynamic Multi-Unit Environments

• Michael Winsper
• Maria Chli

The max-sum loopy belief propagation (LBP) algorithm was shown in [4] to produce strong results in a simple decentralised supply chain formation (SCF) scenario where goods are traded in single units. In this paper, we demonstrate the performance of LBP in a multi-unit SCF scenario with additional constraints. We also provide experimental analysis of LBP’s performance in dynamic scenarios where the properties and composition of participants are altered while the algorithm is running. Our results suggest that LBP continues to produce strong solutions in multi-unit scenarios, and that performance remains solid in a dynamic setting.

AAMAS Conference 2010 Conference Paper

A Probabilistic Model for Trust and Reputation

• George Vogiatzis
• Ian MacGillivray
• Maria Chli

This paper concerns the problem of agent trust in an electronic market place. We maintain that agent trust involves making decisions under uncertainty and therefore the phenomenon should be modelled probabilistically. We therefore propose a probabilistic framework that models agent interactions as a Hidden Markov Model (HMM). The observations of the HMM are the interaction outcomes and the hidden state is the underlying probability of a good outcome. The task of deciding whether to interact with another agent reduces to probabilistic inference of the current state of that agent given all previous interaction outcomes. The model is extended to include a probabilistic reputation system which involves agents gathering opinions about other agents and fusing them with their own beliefs. Our system is fully probabilistic and hence delivers the following improvements with respect to previous work: (a) the model assumptions are faithfully translated into algorithms; our system is optimal under those assumptions. (b) It can account for agents whose behaviour is not static with time (c) it can estimate the rate with which an agent's behaviour changes. The system is shown to significantly outperform previous state-of-the-art methods in several numerical experiments.

AAMAS Conference 2010 Conference Paper

An agent-based simulation of lock-in dynamics in a duopoly

• Michael Garlick
• Maria Chli

We create an agent-based simulation to explore consumerlock-in in a duopoly of experience goods (goods with characteristics that are difficult to determine in advance, butascertained upon consumption). We model heterogeneousagents using simple assumptions, where agents choose between products based upon personal experience and neighbours' decisions. We test strategies to break a lock-in througha free give-away and advertising. We find that, under ourassumptions, breaking a lock-in required the formation ofregions where the competitor product was adopted, likenedto a niche in the market.

ECAI Conference 2010 Conference Paper

Decentralised Supply Chain Formation: A Belief Propagation-based Approach

• Michael Winsper
• Maria Chli

Decentralised supply chain formation involves determining the set of producers within a network able to supply goods to one or more consumers at the lowest cost. This problem is frequently tackled using auctions and negotiations. In this paper we show how it can be cast as an optimisation of a pairwise cost function. Optimising this class of functions is NP-hard but good approximations to the global minimum can be obtained using Loopy Belief Propagation (LBP). Here we detail a LBP-based approach to the supply chain formation problem, involving decentralised message-passing between potential participants. Our approach is evaluated against a well-known double-auction method and an optimal centralised technique, showing several improvements: it obtains better solutions for most networks that admit a competitive equilibriumCompetitive equilibrium as defined in [3] is used as a means of classifying results on certain networks to allow for minor inefficiencies in their auction protocol and agent bidding strategies.