Author name cluster

Markos Papageorgiou

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

AAMAS Conference 2025 Conference Paper

Conditional Max-Sum for Asynchronous Multiagent Decision Making

Dimitrios Troullinos
Georgios Chalkiadakis
Ioannis Papamichail
Markos Papageorgiou

In this paper we present a novel approach for multiagent decision making in dynamic environments based on Factor Graphs and the Max-Sum algorithm, considering asynchronous variable reassignments and distributed message-passing among agents. Motivated by the challenging domain of lane-free traffic where automated vehicles can communicate and coordinate as agents, we propose a more realistic communication framework for Factor Graph formulations that satisfies the above-mentioned restrictions, along with Conditional Max-Sum: an extension of Max-Sum with a revised message-passing process that is better suited for asynchronous settings. The overall application in lane-free traffic can be viewed as a hybrid system where the Factor Graph formulation undertakes the strategic decision making of vehicles, that of desired lateral alignment in a coordinated manner; and acts on top of a rule-based method we devise that provides a structured representation of the lane-free environment for the factors, while also handling the underlying control of vehicles regarding core operations and safety. Our experimental evaluation showcases the capabilities of the proposed framework in problems with intense coordination needs when compared to a domain-specific baseline without communication, and an increased adeptness of Conditional Max-Sum with respect to the standard algorithm.

PDF

AAAI Conference 2025 Conference Paper

Deep Implicit Imitation Reinforcement Learning in Heterogeneous Action Settings

Iason Chrysomallis
Georgios Chalkiadakis
Ioannis Papamichail
Markos Papageorgiou

Implicit imitation reinforcement learning (IIRL) is a framework that aims to aid a trainee agent’s learning process via observing the state transitions of a mentor, but without access to the latter's action information. Standard IIRL assumes a shared Markov decision process (MDP) between the mentor and trainee, consequently implying an identical action space. This restriction imposes limitations on the applicability of implicit imitation frameworks in real-life scenarios where, possibly due to variations in physical characteristics, the mentor agent may possess distinct own actions, thereby creating a heterogeneous action setting. In this work, we extend the deep implicit imitation Q-networks (DIIQN) method -an online, model-free, deep RL algorithm for implicit imitation- to allow for heterogeneous action sets between mentor and trainee agents. Equipped with our heterogeneous actions DIIQN (HA-DIIQN) method, a trainee agent can harvest the benefits of IIRL even in heterogeneous action settings, achieving accelerated learning and outperforming non-optimal mentor agents.

PDF Details DOI

ECAI Conference 2023 Conference Paper

Deep Reinforcement Learning with Implicit Imitation for Lane-Free Autonomous Driving

Iason Chrysomallis
Dimitrios Troullinos
Georgios Chalkiadakis
Ioannis Papamichail
Markos Papageorgiou

Implicit imitation assumes that learning agents observe only the state transitions of an agent they use as a mentor, and try to recreate them based on their own abilities and knowledge of their environment. In this paper, we put forward a deep implicit imitation Q-network (DIIQN) model, which incorporates ideas from three well-known Deep Q-Network (DQN) variants. As such, we enable a novel implicit imitation method for online, model-free deep reinforcement learning. Our thorough experimentation in the complex environment of the emerging lane-free traffic paradigm, verifies the benefits of our approach. Specifically, we show that deep implicit imitation RL dramatically accelerates the learning process when compared to a “vanilla” DQN method; and, unlike explicit imitation reinforcement learning, it is able to outperform mentor performance without resorting to additional information, such as the mentor’s actions.

Details

EUMAS Conference 2022 Conference Paper

Collaborative Decision Making for Lane-Free Autonomous Driving in the Presence of Uncertainty

Pavlos Geronymakis
Dimitrios Troullinos
Georgios Chalkiadakis
Markos Papageorgiou

Abstract The recently introduced lane-free traffic paradigm removes the restrictions of the traffic lanes, so that autonomous vehicles can move anywhere laterally across the road’s width. Previous research in this domain has employed the celebrated max-plus message-passing algorithm in order to allow the coordination of all (connected and autonomous) vehicles in the environment. However, when allowing for the realistic perspective that there exist vehicles that are unable or unwilling to communicate with others, the uncertainty introduced renders the aforementioned coordination approach ineffective. To combat this, in this paper we adjust the Max-plus algorithm accordingly so that agents using max-plus for coordination can also observe and take into consideration independent agents via emulated messages. We put forward different methods to form these messages—namely the Maximax, Maximin, Hurwicz, Minimax Regret and Laplace decision-making criteria. Finally, we provide a thorough evaluation of our approach, including a detailed comparison of all criteria used for message-forming.

Details

AAMAS Conference 2022 Conference Paper

Max-sum with Quadtrees for Continuous DCOPs with Application to Lane-Free Autonomous Driving

Dimitrios Troullinos
Georgios Chalkiadakis
Vasilis Samoladas
Markos Papageorgiou

In this paper we put forward a novel extension of the classic Max- Sum algorithm to the framework of Continuous Distributed Constrained Optimization Problems (Continuous DCOPs), in which we model the exchanged messages by means of a popular geometric algorithm, Quadtrees. As such, the discretization process is dynamic and embedded in the internal Max-Sum operations (addition and marginal maximization). We apply our Max-Sum with Quadtrees approach to Lane-Free Autonomous Driving in a highway populated with vehicles. Our experimental evaluation verifies the efficiency of our approach in this challenging dynamic coordination domain, demonstrating its superior performance with respect to the standard Max-Sum algorithm.

PDF

IJCAI Conference 2022 Conference Paper

Max-Sum with Quadtrees for Decentralized Coordination in Continuous Domains

Dimitrios Troullinos
Georgios Chalkiadakis
Vasilis Samoladas
Markos Papageorgiou

In this paper we put forward a novel extension of the classic Max-Sum algorithm to the framework of Continuous Distributed Constrained Optimization Problems (Continuous DCOPs), by utilizing a popular geometric algorithm, namely Quadtrees. In its standard form, Max-Sum can only solve Continuous DCOPs with an a priori discretization procedure. Existing Max-Sum extensions to continuous multiagent coordination domains require additional assumptions regarding the form of the factors, such as access to the gradient, or the ability to model them as continuous piecewise linear functions. Our proposed approach has no such requirements: we model the exchanged messages with Quadtrees, and, as such, the discretization procedure is dynamic and embedded in the internal Max-Sum operations (addition and marginal maximization). We apply Max-Sum with Quadtrees to lane-free autonomous driving. Our experimental evaluation showcases the effectiveness of our approach in this challenging coordination domain.

PDF Details DOI

AAMAS Conference 2021 Conference Paper

Collaborative Multiagent Decision Making for Lane-Free Autonomous Driving

Dimitrios Troullinos
Georgios Chalkiadakis
Ioannis Papamichail
Markos Papageorgiou

This paper addresses the problem of collaborative multi-agent autonomous driving of connected and automated vehicles (CAVs) in lane-free highway scenarios. We eliminate the lane-changing task, i. e. , CAVs may be located in any arbitrary lateral position within the road boundaries, hence allowing for better utilization of the available road capacity. As a consequence, vehicles operate in a much more complex environment, and the need for the individual CAVs to select actions that are efficient for the group as a whole is highly desired. We formulate this environment as a multiagent collaboration problem represented via a coordination graph, thus decomposing the problem with local utility functions, based on the interactions between vehicles. We produce a tractable and scalable solution by estimating the joint action of all vehicles via the anytime max-plus algorithm, with local utility functions provided by potential fields, designed to promote collision avoidance. Specifically, the fields have an ellipsoid form that is most suitable for lane-free highway environments. This novel use of max-plus with potential fields gives rise to a coordinated control policy that exploits only local information specific to each CAV. Our experimental evaluation confirms the effectiveness of our approach: lane-free movement allows for increased traffic flow rates, and vehicles are able to achieve speeds that are both high and close to their desired ones, even in demanding environments with high traffic flow.

PDF