Author name cluster

Bingyun Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

AAMAS Conference 2026 Conference Paper

Towards Foresighted AI Cooperators with LLM-driven Decision-Time Planning

Yuheng Jing
Kai Li
Bingyun Liu
Ziwen Zhang
Zhe Wu
Yifan Zhang
Junliang Xing
Jian Cheng

In multi-agent systems, building agents capable of seamlessly collaborating with unknown partners is a long-standing research goal. Existing approaches primarily generate a diverse population of partners and then train an agent against this population to master various cooperation conventions. However, these approaches are often hindered in two aspects: (1) They heavily rely on taskspecific training; (2) Their trained agents lack adaptability at test time. In this paper, we investigate how to leverage Large Language Models (LLMs) to build agents capable of foresighted coordination, addressing the challenges faced by existing work. To facilitate structured reasoning mechanisms, we introduce DTPAgent, a novel LLM-driven Decision-Time Planning (DTP) framework. Within this framework, LLMs, without relying on task-specific training and solely through in-context learning, estimate the partner policy and the transition-reward function to model the full dynamics of the environment. Based on these LLM-driven modelings, DTPAgent This work is licensed under a Creative Commons Attribution International 4. 0 License. Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026), C. Amato, L. Dennis, V. Mascardi, J. Thangarajah (eds.), May 25 – 29, 2026, Paphos, Cyprus. © 2026 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org). https: //doi. org/10. 65109/XALP4331 simulates a range of possible trajectories to dynamically search for the most advantageous policy at each timestep. We demonstrate on the popular benchmark, Overcooked, that DTPAgent, built with small-scale LLMs, effectively outperforms various types of baselines when faced with unseen partners. Our DTPAgent also exhibits a scalable property that existing agents lack: the ability to translate test-time computation into improved performance.

PDF

AAAI Conference 2025 Conference Paper

An Open-Ended Learning Framework for Opponent Modeling

Yuheng Jing
Kai Li
Bingyun Liu
Haobo Fu
Qiang Fu
Junliang Xing
Jian Cheng

Opponent Modeling (OM) aims to enhance decision-making by modeling other agents in multi-agent environments. Existing works typically learn opponent models against a pre-designated fixed set of opponents during training. However, this will cause poor generalization when facing unknown opponents during testing, as previously unseen opponents can exhibit out-of-distribution (OOD) behaviors that the learned opponent models cannot handle. To tackle this problem, we introduce a novel Open-Ended Opponent Modeling (OEOM) framework, which continuously generates opponents with diverse strengths and styles to reduce the possibility of OOD situations occurring during testing. Founded on population-based training and information-theoretic trajectory space diversity regularization, OEOM generates a dynamic set of opponents. This set is then fed to any OM approaches to train a potentially generalizable opponent model. Upon this, we further propose a simple yet effective OM approach that naturally fits within the OEOM framework. This approach is based on in-context reinforcement learning and learns a Transformer that dynamically recognizes and responds to opponents based on their trajectories. Extensive experiments in cooperative, competitive, and mixed environments demonstrate that OEOM is an approach-agnostic framework that improves generalizability compared to training against a fixed set of opponents, regardless of OM approaches or testing opponent settings. The results also indicate that our proposed approach generally outperforms existing OM baselines.

PDF Details DOI

ICML Conference 2025 Conference Paper

Offline Opponent Modeling with Truncated Q-driven Instant Policy Refinement

Yuheng Jing
Kai Li 0022
Bingyun Liu
Ziwen Zhang
Haobo Fu
Qiang Fu 0016
Junliang Xing
Jian Cheng 0001

Offline Opponent Modeling (OOM) aims to learn an adaptive autonomous agent policy that dynamically adapts to opponents using an offline dataset from multi-agent games. Previous work assumes that the dataset is optimal. However, this assumption is difficult to satisfy in the real world. When the dataset is suboptimal, existing approaches struggle to work. To tackle this issue, we propose a simple and general algorithmic improvement framework, Truncated Q-driven Instant Policy Refinement (TIPR), to handle the suboptimality of OOM algorithms induced by datasets. The TIPR framework is plug-and-play in nature. Compared to original OOM algorithms, it requires only two extra steps: (1) Learn a horizon-truncated in-context action-value function, namely Truncated Q, using the offline dataset. The Truncated Q estimates the expected return within a fixed, truncated horizon and is conditioned on opponent information. (2) Use the learned Truncated Q to instantly decide whether to perform policy refinement and to generate policy after refinement during testing. Theoretically, we analyze the rationale of Truncated Q from the perspective of No Maximization Bias probability. Empirically, we conduct extensive comparison and ablation experiments in four representative competitive environments. TIPR effectively improves various OOM algorithms pretrained with suboptimal datasets.

Details

IJCAI Conference 2024 Conference Paper

Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent

Hang Xu
Kai Li
Bingyun Liu
Haobo Fu
Qiang Fu
Junliang Xing
Jian Cheng

Counterfactual regret minimization (CFR) is a family of algorithms for effectively solving imperfect-information games. It decomposes the total regret into counterfactual regrets, utilizing local regret minimization algorithms, such as Regret Matching (RM) or RM+, to minimize them. Recent research establishes a connection between Online Mirror Descent (OMD) and RM+, paving the way for an optimistic variant PRM+ and its extension PCFR+. However, PCFR+ assigns uniform weights for each iteration when determining regrets, leading to substantial regrets when facing dominated actions. This work explores minimizing weighted counterfactual regret with optimistic OMD, resulting in a novel CFR variant PDCFR+. It integrates PCFR+ and Discounted CFR (DCFR) in a principled manner, swiftly mitigating negative effects of dominated actions and consistently leveraging predictions to accelerate convergence. Theoretical analyses prove that PDCFR+ converges to a Nash equilibrium, particularly under distinct weighting schemes for regrets and average strategies. Experimental results demonstrate PDCFR+'s fast convergence in common imperfect-information games. The code is available at https: //github. com/rpSebastian/PDCFRPlus.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Opponent Modeling with In-context Search

Yuheng Jing
Bingyun Liu
Kai Li
Yifan Zang
Haobo Fu
Qiang Fu
Junliang Xing
Jian Cheng

Opponent modeling is a longstanding research topic aimed at enhancing decision-making by modeling information about opponents in multi-agent environments. However, existing approaches often face challenges such as having difficulty generalizing to unknown opponent policies and conducting unstable performance. To tackle these challenges, we propose a novel approach based on in-context learning and decision-time search named Opponent Modeling with In-context Search (OMIS). OMIS leverages in-context learning-based pretraining to train a Transformer model for decision-making. It consists of three in-context components: an actor learning best responses to opponent policies, an opponent imitator mimicking opponent actions, and a critic estimating state values. When testing in an environment that features unknown non-stationary opponent agents, OMIS uses pretrained in-context components for decision-time search to refine the actor's policy. Theoretically, we prove that under reasonable assumptions, OMIS without search converges in opponent policy recognition and has good generalization properties; with search, OMIS provides improvement guarantees, exhibiting performance stability. Empirically, in competitive, cooperative, and mixed environments, OMIS demonstrates more effective and stable adaptation to opponents than other approaches. See our project website at https: //sites. google. com/view/nips2024-omis.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Towards Offline Opponent Modeling with In-context Learning

Yuheng Jing
Kai Li 0022
Bingyun Liu
Yifan Zang 0001
Haobo Fu
Qiang Fu 0016
Junliang Xing
Jian Cheng 0001

Opponent modeling aims at learning the opponent's behaviors, goals, or beliefs to reduce the uncertainty of the competitive environment and assist decision-making. Existing work has mostly focused on learning opponent models online, which is impractical and inefficient in practical scenarios. To this end, we formalize an Offline Opponent Modeling (OOM) problem with the objective of utilizing pre-collected offline datasets to learn opponent models that characterize the opponent from the viewpoint of the controlled agent, which aids in adapting to the unknown fixed policies of the opponent. Drawing on the promises of the Transformers for decision-making, we introduce a general approach, Transformer Against Opponent (TAO), for OOM. Essentially, TAO tackles the problem by harnessing the full potential of the supervised pre-trained Transformers' in-context learning capabilities. The foundation of TAO lies in three stages: an innovative offline policy embedding learning stage, an offline opponent-aware response policy training stage, and a deployment stage for opponent adaptation with in-context learning. Theoretical analysis establishes TAO's equivalence to Bayesian posterior sampling in opponent modeling and guarantees TAO's convergence in opponent policy recognition. Extensive experiments and ablation studies on competitive environments with sparse and dense rewards demonstrate the impressive performance of TAO. Our approach manifests remarkable prowess for fast adaptation, especially in the face of unseen opponent policies, confirming its in-context learning potency.

Details