Dynamically Increasing Agents Set-Size in Bayesian Multi-agent Multi-armed Bandits Framework

Modern systems usually face environments that change over time and often unpredictably, making static or fixed agent learning strategies inadequate for sustained performance. The Likelihood- Adaptive Multi-Agent Systems (LA-MAS) introduces a novel framework for lifelong adaptation in Multi-Armed Bandits under nonstationaryenvironments. Incontrasttootherdynamicenvironmentbased approaches, LA-MAS maintains and incrementally expands a dynamic pool of agent models, each with its own Q-table and learning policy, specialized for distinct environmental characteristics. LA-MAS uses Bayesian inference to compute the likelihood with which each agent explains recently observed rewards, updating a probability distribution over all agents. When all current agents are unable to identify the current environment due to the low likelihoods, the framework detects a changepoint and automatically adds a new agent while retaining memory of previous agents for rapid re-adaptation when the environment recurs. Our empirical results on both synthetic and real-world server environments showed LA-MAS’s ability to achieve a lower regret of around 50% and faster environment detection compared to state-of-the-art methods.

Authors

• Mohammad Essa Alsomali Lancaster University
• Leandro Soriano Marcolino VinUniversity
• Barry Porter Lancaster University
• Roberto Rodrigues-Filho Federal University of Santa Catarina

Keywords

• On-line Learning
• Multi-Armed Bandits
• Bayesian Inference