Algorithmic Contract Design with Reinforcement Learning Agents

Designing incentive mechanisms for multi-agent systems in stochastic and dynamic environments is a critical challenge, as system outcomes emerge from the complex interplay of agent learning and environmentaluncertainty. Existingprincipal–multi-agentcontract design methods often assume static settings or ignore learning dynamics, limiting their applicability in multi-agent reinforcement learning (MARL). Furthermore, the contract design space is highly constrained by feasibility requirements, such as individual rationality and incentive compatibility, making it difficult to explore. Weintroducetheprincipal-MARLcontractdesignproblem, where a principal optimizes both recruitment and incentive contracts evaluated via MARL. To address this problem, we propose Constrained ParetoMaximumEntropySearch(cPMES), amulti-objectiveBayesian optimization framework that treats feasibility as an explicit objective and selects designs based on information gain over the Pareto front. Experiments in social dilemma environments demonstrate thatcPMESefficientlyidentifiesfeasible, high-performingcontracts, significantly improving coordination and system-level rewards.

Authors

• David Molina Concha University of Toronto
• Kyeonghyeon Park Korea Advanced Institute of Science and Technology
• Hyun-Rok Lee Inha University
• Taesik Lee Korea Advanced Institute of Science & Technology
• Chi-Guhn Lee University of Toronto

Keywords

• Multi-agent systems
• Reinforcement Learning
• Contract Design