Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models

Policy-Space Response Oracles (PSRO) have enabled the computation of approximate Nash equilibria in complex games. However, standard implementations rely on Deep Reinforcement Learning oracles, producing "black-box" neural network policies that are opaque, difficult to verify, and sample-inefficient. We introduce Code-Space Response Oracles (CSRO), a framework that tasks a Large Language Model (LLM) to synthesize code policies. CSRO reframes best-response computation as a code generation task, producing policies as executable, human-readable Python code. We demonstrate that CSRO, particularly when augmented with evolutionary refinement (AlphaEvolve), achieves performance competitive with baselines while offering superior interpretability and leveraging the LLM’s pretraining knowledge.

Authors

• Daniel Hennes Google DeepMind
• Zun Li Google DeepMind
• John Schultz Google DeepMind
• Marc Lanctot Google DeepMind

Keywords

• Multi-Agent Reinforcement Learning
• Game Theory
• Large Language Models
• Code Synthesis