The Multi-Agent Off-Switch Game

The off-switch game framework has been instrumental in understanding corrigibility—the property that AI agents should allow human oversight and intervention. In single-agent settings, uncertainty about human preferences naturally incentivizes agents to defer to human judgment. However, as AI systems increasingly operate in multi-agent environments, a crucial question arises: does corrigibility compose across multiple agents? We introduce the multi-agent off-switch game and demonstrate that individually corrigible agents can become collectively incorrigible when strategic interactions are considered. Through formal analysis and illustrative examples, we show that corrigibility is not compositional and identify conditions under which group incorrigibility emerges. Our results highlight fundamental challenges for AI safety in multi-agent settings and suggest the need for new approaches that explicitly address collective dynamics.

Authors

• Akash Agrawal ML Alignment & Theory Scholars
• Soroush Ebadian Pivotal Research & University of Toronto
• Lewis Hammond Cooperative AI Foundation & University of Oxford

Keywords

• Corrigibility
• Off-switch game
• Alignment
• Multi-agent systems