Victor R. Lesser

JAAMAS Journal 2026 Journal Article

Learning Situation-Specific Coordination in Cooperative Multi-agent Systems

• M. V. Nagendra Prasad
• Victor R. Lesser

Abstract Achieving effective cooperation in a multi-agent system is a difficult problem for a number of reasons such as limited and possibly out-dated views of activities of other agents and uncertainty about the outcomes of interacting non-local tasks. In this paper, we present a learning system called COLLAGE, that endows the agents with the capability to learn how to choose the most appropriate coordination strategy from a set of available coordination strategies. COLLAGE relies on meta-level information about agents' problem solving situations to guide them towards a suitable choice for a coordination strategy. We present empirical results that strongly indicate the effectiveness of the learning algorithm.

JAAMAS Journal 2026 Journal Article

Modeling uncertainty and its implications to sophisticated control Tæms agents

• Thomas A. Wagner
• Anita Raja
• Victor R. Lesser

JAAMAS Journal 2026 Journal Article

Reflections on the Nature of Multi-Agent Coordination and Its Implications for an Agent Architecture

• Victor R. Lesser

Abstract The development of enabling infrastructure for the next generation of multi-agent systems consisting of large numbers of agents and operating in open environments is one of the key challenges for the multi-agent community. Current infrastructure support does not materially assist in the development of sophisticated agent coordination strategies. It is the need for and the development of such a high-level support structure that will be the focus of this paper. A domain-independent (generic) agent architecture is proposed that wraps around an agent's problem-solving component in order to make problem solving responsive to real-time constraints, available network resources, and the need to coordinate—both in the large and small—with problem-solving activities of other agents. This architecture contains five components, local agent scheduling, multi-agent coordination, organizational design, detection and diagnosis, and on-line learning, that are designed to interact so that a range of different situation-specific coordination strategies can be implemented and adapted as the situation evolves. The presentation of this architecture is followed by a more detailed discussion on the interaction among these components and the research questions that need to be answered to understand the appropriateness of this architecture for the next generation of multi-agent systems.

ICML Conference 2020 Conference Paper

ROMA: Multi-Agent Reinforcement Learning with Emergent Roles

• Tonghan Wang 0001
• Heng Dong 0001
• Victor R. Lesser
• Chongjie Zhang

The role concept provides a useful tool to design and understand complex multi-agent systems, which allows agents with a similar role to share similar behaviors. However, existing role-based methods use prior domain knowledge and predefine role structures and behaviors. In contrast, multi-agent reinforcement learning (MARL) provides flexibility and adaptability, but less efficiency in complex tasks. In this paper, we synergize these two paradigms and propose a role-oriented MARL framework (ROMA). In this framework, roles are emergent, and agents with similar roles tend to share their learning and to be specialized on certain sub-tasks. To this end, we construct a stochastic role embedding space by introducing two novel regularizers and conditioning individual policies on roles. Experiments show that our method can learn specialized, dynamic, and identifiable roles, which help our method push forward the state of the art on the StarCraft II micromanagement benchmark. Demonstrative videos are available at https: //sites. google. com/view/romarl/.

AAMAS Conference 2019 Conference Paper

Ethically Aligned Multi-agent Coordination to Enhance Social Welfare

• Han Yu
• Zhiqi Shen
• Lizhen Cui
• Yongqing Zheng
• Victor R. Lesser

In multi-agent systems (MASs), the complex interactions among self-interested agents can be modelled as stochastic games. Existing decision support approaches dealing with such situations focus on minimizing individual agent’s regret through outperforming other agents in the competitive aspect of the game. Such an approach often results in social welfare not being maximized in the process. In this paper, we propose the regret-minimization-social-welfare-maximization (RMSM) approach. It contains a novel method to quantify how an agent’s sacrifice increases and decreases over time based on queueing system dynamics. In this way, ensuring fairness of distribution of sacrifice among agents and compensating for their previous sacrifices can be translated into maintaining the stability of a queueing system.

IJCAI Conference 2018 Conference Paper

Building Ethics into Artificial Intelligence

• Han Yu
• Zhiqi Shen
• Chunyan Miao
• Cyril Leung
• Victor R. Lesser
• Qiang Yang

As artificial intelligence (AI) systems become increasingly ubiquitous, the topic of AI governance for ethical decision-making by AI has captured public imagination. Within the AI research community, this topic remains less familiar to many researchers. In this paper, we complement existing surveys, which largely focused on the psychological, social and legal discussions of the topic, with an analysis of recent advances in technical solutions for AI governance. By reviewing publications in leading AI conferences including AAAI, AAMAS, ECAI and IJCAI, we propose a taxonomy which divides the field into four areas: 1) exploring ethical dilemmas; 2) individual ethical decision frameworks; 3) collective ethical decision frameworks; and 4) ethics in human-AI interactions. We highlight the intuitions and key techniques used in each approach, and discuss promising future research directions towards successful integration of ethical AI systems into human societies.

IJCAI Conference 2013 Conference Paper

A Reputation Management Approach for Resource Constrained Trustee Agents

• Han Yu
• Chunyan Miao
• Bo An
• Cyril Leung
• Victor R. Lesser

Trust is an important mechanism enabling agents to self-police open and dynamic multi-agent systems (ODMASs). Trusters evaluate the reputation of trustees based on their past observed performance, and use this information to guide their future interaction decisions. Existing trust models tend to concentrate trusters’ interactions on a small number of highly reputable trustees to minimize risk exposure. When a trustee’s servicing capacity is limited, such an approach may cause long delays for trusters and subsequently damage the reputation of trustees. To mitigate this problem, we propose a reputation management approach for trustee agents based on distributed constraint optimization. It helps a trustee to make situation-aware decisions on which incoming requests to serve and prevent the resulting reputation score from being affected by factors out of the trustee’s control. The approach is evaluated through theoretical analysis and within a simulated, highly dynamic multi-agent environment. The results show that it can achieve close to optimally efficient utilization of the trustee agents’ collective capacity in an ODMAS, promotes fair treatment of trustee agents based on their behavior, and significantly outperforms related work in enhancing social welfare.

AAMAS Conference 2013 Conference Paper

Biasing the Behavior of Organizationally Adept Agents

• Daniel Corkill
• Chongjie Zhang
• Bruno da Silva
• Yoonheui Kim
• Daniel Garant
• Victor R. Lesser
• Xiaoqin Zhang

An organizationally adept agent (OAA) adjusts its behavior when given annotated organizational guidelines. More importantly, it can also determine when such guidelines become ineffective and proactively adapt its behavior to better achieve organizational objectives. We present the high-level aspects of this architecture and analyze its effectiveness using call-center OAAs striving to extinguish fires in RoboCup Rescue scenarios.

UAI Conference 2011 Conference Paper

Compact Mathematical Programs For DEC-MDPs With Structured Agent Interactions

• Hala Mostafa
• Victor R. Lesser

To deal with the prohibitive complexity of calculating policies in Decentralized MDPs, researchers have proposed models that exploit structured agent interactions. Settings where most agent actions are independent except for few actions that affect the transitions and/or rewards of other agents can be modeled using Event-Driven Interactions with Complex Rewards (EDI-CR). Finding the optimal joint policy can be formulated as an optimization problem. However, existing formulations are too verbose and/or lack optimality guarantees. We propose a compact Mixed Integer Linear Program formulation of EDI-CR instances. The key insight is that most action sequences of a group of agents have the same effect on a given agent. This allows us to treat these sequences similarly and use fewer variables. Experiments show that our formulation is more compact and leads to faster solution times and better solutions than existing formulations.

JAAMAS Journal 2006 Journal Article

Modeling Uncertainty and its Implications to Sophisticated Control in Tæms Agents

• Thomas A. Wagner
• Anita Raja
• Victor R. Lesser

Abstract Open environments are characterized by their uncertainty and non-determinism. Agents need to adapt their task processing to available resources, deadlines, the goal criteria specified by the clients as well their current problem solving context in order to survive in these environments. If there were no resource constraints, then an optimal Markov Decision Process based policy would obviously be the best way for complex problem solving agents to make scheduling decisions. However in many agent systems, these scheduling decisions have to be made on-line or in soft real-time, making the off-line policy computationally infeasible in open environments. The hybrid planner/scheduler used to control Task Analysis, Environment Modeling, and Simulation (TÆMS) agents is the Design-to-Criteria (DTC) agent scheduler. Design-to-Criteria scheduling is the soft real-time process of custom building a plan/schedule to meet an agent’s current objectives which are expressed as dynamic goal criteria (including real-time deadlines), using task models that describe alternate ways to achieve tasks and subtasks. Recent advances in Design-to-Criteria control include the addition of uncertainty to the TÆMS computational task models analyzed by the scheduler and the incorporation of uncertainty in the scheduling process. As we show, the use of uncertainty in TÆMS and Design-to-Criteria enables agents to make better control decisions in uncertain environments. Design-to-Criteria uses a heuristic approach for on-line scheduling of medium granularity tasks. It approximates the analysis used to generate an optimal policy by heuristically reasoning about the implications of uncertainty in task execution. The addition of uncertainty has also spawned a post-scheduling contingency analysis step for situations in which an agent must produce a result by a given deadline (deadline critical situations) and where the added computational cost is worth the expense. We describe the uncertainty representation in TÆMS and how it improves task models and the scheduling process, and provide empirical examples of reasoning about uncertainty in action. We also evaluate the performance of our heuristic-based approach to agent control using the performance of the policy generated by an optimal controller as the benchmark.

AAMAS Conference 2002 Conference Paper

AgentSpeak(XL): Efficient Intention Selection in BDI Agents via Decision-Theoretic Task Scheduling

• Rafael H. Bordini
• Ana L. C. Bazzan
• Rafael de O. Jannone
• Daniel M. Basso
• Rosa M. Vicari
• Victor R. Lesser

ICAPS Conference 1996 Conference Paper

A Cooperative Repair Method for a Distributed Scheduling System

• Daniel E. Neiman
• Victor R. Lesser

For some time, we have been studying the issues involved in job-shop scheduling in an environment of cooperative distributed agents, none of which has a complete view of the resources available, or of the tasks to be scheduled. Schedules produced cooperatively by such distributed agents using constraint satisfaction methods are often not optimal because of the inherent asynchronicity of the distributed scheduling process, the bounded rationality of the scheduling agents, and the difficulty in completely integrating meta-level heuristics into an agent’s local scheduling processes. This paper describes a modification to distributed scheduling in which the loosely coupled distributed processing methods are supplemented with a tightly coupled parallel repair process. We explore the implications on the repair process of a distributed environment in which the designer of the repair algorithm must address issues of agent communication and organization. We describe a search algorithm and a set of heuristics for guiding the repair process and present some experimental results in the context of the Distributed ARM, an airline resource scheduling system.

AIJ Journal 1995 Journal Article

IPUS: an architecture for the integrated processing and understanding of signals

• Victor R. Lesser
• S.Hamid Nawab
• Frank I. Klassner

The Integrated Processing and Understanding of Signals (IPUS) architecture is presented as a framework that exploits formal signal processing models to structure the bidirectional interaction between front-end signal processing and signal understanding processes. This architecture is appropriate for complex environments, which are characterized by variable signal-to-noise ratios, unpredictable source behaviors, and the simultaneous occurrence of objects whose signal signatures can distort each other. A key aspect of this architecture is that front-end signal processing is dynamically modifiable in response to scenario changes and to the need to reanalyze ambiguous or distorted data. The architecture tightly integrates the search for the appropriate front-end signal processing configuration with the search for plausible interpretations. In our opinion, this dual search, informed by formal signal processing theory, is a necessary component of perceptual systems that must interact with complex environments. To explain this architecture in detail, we discuss examples of its use in an implemented system for acoustic signal interpretation.

AAAI Conference 1994 Conference Paper

Exploiting Meta-Level Information in a Distributed Scheduling System

• Daniel E. Neiman
• Victor R. Lesser

In this paper, we study the problem of achieving efficient interaction in a distributed scheduling system whose scheduling agents may borrow resources from one another. Specifically, we expand on Sycara’ s use of resource texture measures in a distributed scheduling system with a central resource monitor for each resource type and apply it to the decentralized case. We show how analysis of the abstracted resource requirements of remote agents can guide an agent’ s choice of local scheduling activities not only in determining local constraint tightness, but also in identifying activities that reduce global uncertainty. We also exploit meta-level information to allow the scheduling agents to make reasoned decisions about when to attempt to solve impasses locally through backtracking and constraint relaxation and when to request resources from remote agents. Finally, we describe the current state of negotiation in our system and discuss plans for integrating a more sophisticated cost model into the negotiation protocol. This work is presented in the context of the Distributed Airport Resource Management System, a multi-agent system for solving airport ground service scheduling problems.

IJCAI Conference 1993 Conference Paper

Understanding the Role of Negotiation in Distributed Search Among Heterogeneous Agents

• Susan E. Lander
• Victor R. Lesser

In our research, we explore the role of negotiation for conflict resolution in distributed search among heterogeneous and reusable agents. We present negotiated search, an algorithm that explicitly recognizes and exploits conflict to direct search activity across a set of agents. In negotiated search, loosely coupled agents interleave the tasks of 1) local search for a solution to some subproblem; 2) integration of local subproblem solutions into a shared solution; 3) information exchange to define and refine the shared search space of the agents; and 4) assessment and reassessment of emerging solutions. Negotiated search is applicable to diverse application areas and problem-solving environments. It requires only basic search operators and allows maximum flexibility in the distribution of those operators. These qualities make the algorithm particularly appropriate for the integration of heterogeneous agents into application systems. The algorithm is implemented in a multi-agent framework, TEAM, that provides the infrastructure required for communication and cooperation.

IJCAI Conference 1989 Conference Paper

Focus of Control Through Goal Relationships

• Victor R. Lesser
• Daniel D. Corkill
• Robert C. Whitehair
• Joseph A. Hernandez

Goal relationships resulting from the initial data and subsequent processing can he used to dynamically construct a partial topology of the solution space based on what appear to be feasible solutions. This structure can be used to make control decisions that significantly reduce the amount of search required to solve a problem in a complex domain. We examine the utility of this approach in the context of a multi-level, cooperative knowledge source model of problem solving. We present a taxonomy of goal relationships for constructing partial topologies of the solution space and show that mechanisms using this information can be built as natural extensions of an integrated data-directed and goal directed archi tecture. Examples and performance results demonstrating how these additions improve the system's ability to evaluate potential activities are provided.

IJCAI Conference 1987 Conference Paper

Using Partial Global Plans to Coordinate Distributed Problem Solvers

• Edmund H. Durfee
• Victor R. Lesser

Communicating problem solvers can cooperate in various ways, such as negotiating over task assignments, exchanging partial solutions to converge on global results, and planning interactions that help each other perform their tasks better. We introduce a new framework that supports different styles of cooperation by using partial global plana to specify effective, coordinated actions for groups of problem solvers. In this framework, problem solvers summarise their local plans into node-plant that they selectively exchange to dynamically model network activity and to develop partial global plans. However, because network and problem characteristics can change and communication channels have delays and limited capacity, problem solvers' models and partial global plans may be incomplete, out-of-date, and inconsistent. Our mechanisms allow problem solvers to agree on consistent partial global plans when possible, and to locally form partial global plans that lead to satisfactory cooperation even in rapidly changing environments where complete agreement is impossible. In this paper, we describe the mechanisms, knowledge representations, and algorithms that we have developed for generating and maintaining partial global plans in a distributed system. We use experiments to illustrate how these mechanisms improve and promote cooperation in a variety of styles.

IJCAI Conference 1985 Conference Paper

Increasing Coherence in a Distributed Problem-Solving Network

• Edmund H. Durfee
• Victor R. Lesser
• Daniel D. Corkill

IJCAI Conference 1983 Conference Paper

The Use of Meta-Level Control for Coordination in a Distributed Problem Solving Network

• Daniel D. Corkill
• Victor R. Lesser