AAMAS Conference 2026 Conference Paper
Algorithmic Contract Design with Reinforcement Learning Agents
- David Molina Concha
- Kyeonghyeon Park
- Hyun-Rok Lee
- Taesik Lee
- Chi-Guhn Lee
Designing incentive mechanisms for multi-agent systems in stochastic and dynamic environments is a critical challenge, as system outcomes emerge from the complex interplay of agent learning and environmentaluncertainty. Existingprincipal–multi-agentcontract design methods often assume static settings or ignore learning dynamics, limiting their applicability in multi-agent reinforcement learning (MARL). Furthermore, the contract design space is highly constrained by feasibility requirements, such as individual rationality and incentive compatibility, making it difficult to explore. Weintroducetheprincipal-MARLcontractdesignproblem, where a principal optimizes both recruitment and incentive contracts evaluated via MARL. To address this problem, we propose Constrained ParetoMaximumEntropySearch(cPMES), amulti-objectiveBayesian optimization framework that treats feasibility as an explicit objective and selects designs based on information gain over the Pareto front. Experiments in social dilemma environments demonstrate thatcPMESefficientlyidentifiesfeasible, high-performingcontracts, significantly improving coordination and system-level rewards.