Author name cluster

Daniel Hennes

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

28 papers

2 author rows

IJCAI Conference 2025 Conference Paper

Combining Deep Reinforcement Learning and Search with Generative Models for Game-Theoretic Opponent Modeling

Zun Li
Marc Lanctot
Kevin R. McKee
Luke Marris
Ian Gemp
Daniel Hennes
Paul Muller
Kate Larson

Opponent modeling methods typically involve two crucial steps: building a belief distribution over opponents' strategies, and exploiting this opponent model by playing a best response. However, existing approaches typically require domain-specific heurstics to come up with such a model, and algorithms for approximating best responses are hard to scale in large, imperfect information domains. In this work, we introduce a scalable and generic multiagent training regime for opponent modeling using deep game-theoretic reinforcement learning. We first propose Generative Best Respoonse (GenBR), a best response algorithm based on Monte-Carlo Tree Search (MCTS) with a learned deep generative model that samples world states during planning. This new method scales to large imperfect information domains and can be plug and play in a variety of multiagent algorithms. We use this new method under the framework of Policy Space Response Oracles (PSRO), to automate the generation of an offline opponent model via iterative game-theoretic reasoning and population-based training. We propose using solution concepts based on bargaining theory to build up an opponent mixture, which we find identifying profiles that are near the Pareto frontier. Then GenBR keeps updating an online opponent model and reacts against it during gameplay. We conduct behavioral studies where human participants negotiate with our agents in Deal-or-No-Deal, a class of bilateral bargaining games. Search with generative modeling finds stronger policies during both training time and test time, enables online Bayesian co-player prediction, and can produce agents that achieve comparable social welfare and Nash bargaining score negotiating with humans as humans trading among themselves.

PDF Details DOI

ICML Conference 2025 Conference Paper

Mastering Board Games by External and Internal Planning with Language Models

John Schultz
Jakub Adámek
Matej Jusup
Marc Lanctot
Michael Kaisers
Sarah Perrin
Daniel Hennes
Jeremy Shar

Advancing planning and reasoning capabilities of Large Language Models (LLMs) is one of the key prerequisites towards unlocking their potential for performing reliably in complex and impactful domains. In this paper, we aim to demonstrate this across board games (Chess, Fischer Random / Chess960, Connect Four, and Hex), and we show that search-based planning can yield significant improvements in LLM game-playing strength. We introduce, compare and contrast two major approaches: In external search, the model guides Monte Carlo Tree Search (MCTS) rollouts and evaluations without calls to an external game engine, and in internal search, the model is trained to generate in-context a linearized tree of search and a resulting final choice. Both build on a language model pre-trained on relevant domain knowledge, reliably capturing the transition and value functions in the respective environments, with minimal hallucinations. We evaluate our LLM search implementations against game-specific state-of-the-art engines, showcasing substantial improvements in strength over the base model, and reaching Grandmaster-level performance in chess while operating closer to the human search budget. Our proposed approach, combining search with domain knowledge, is not specific to board games, hinting at more general future applications.

Details

AAMAS Conference 2024 Conference Paper

Approximating the Core via Iterative Coalition Sampling

Ian Gemp
Marc Lanctot
Luke Marris
Yiran Mao
Edgar Duéñez-Guzmán
Sarah Perrin
Andras Gyorgy
Romuald Elie

The core is a central solution concept in cooperative game theory, defined as the set of feasible allocations or payments such that no subset of agents has incentive to break away and form their own subgroup or coalition. However, it has long been known that the core (and approximations, such as the least-core) are hard to compute. This limits our ability to analyze cooperative games in general, and to fully embrace cooperative game theory contributions in domains such as explainable AI (XAI), where the core can complement the Shapley values to identify influential features or instances supporting predictions by black-box models. We propose novel iterative algorithms for computing variants of the core, which avoid the computational bottleneck of many other approaches; namely solving large linear programs. As such, they scale better to very large problems as we demonstrate across different classes of cooperative games, including weighted voting games, induced subgraph games, and marginal contribution networks. We also explore our algorithms in the context of XAI, providing further evidence of the power of the core for such applications. This work is licensed under a Creative Commons Attribution International 4. 0 License. Proc. of the 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2024), N. Alechina, V. Dignum, M. Dastani, J. S. Sichman (eds.), May 6 – 10, 2024, Auckland, New Zealand. © 2024 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org).

PDF

TMLR Journal 2023 Journal Article

Population-based Evaluation in Repeated Rock-Paper-Scissors as a Benchmark for Multiagent Reinforcement Learning

Marc Lanctot
John Schultz
Neil Burch
Max Olan Smith
Daniel Hennes
Thomas Anthony
Julien Perolat

Progress in fields of machine learning and adversarial planning has benefited significantly from benchmark domains, from checkers and the classic UCI data sets to Go and Diplomacy. In sequential decision-making, agent evaluation has largely been restricted to few interactions against experts, with the aim to reach some desired level of performance (e.g. beating a human professional player). We propose a benchmark for multiagent learning based on repeated play of the simple game Rock, Paper, Scissors along with a population of forty-three tournament entries, some of which are intentionally sub-optimal. We describe metrics to measure the quality of agents based both on average returns and exploitability. We then show that several RL, online learning, and language model approaches can learn good counter-strategies and generalize well, but ultimately lose to the top-performing bots, creating an opportunity for research in multiagent learning.

PDF Details

AAMAS Conference 2023 Conference Paper

Search-Improved Game-Theoretic Multiagent Reinforcement Learning in General and Negotiation Games

Zun Li
Marc Lanctot
Kevin R. McKee
Luke Marris
Ian Gemp
Daniel Hennes
Kate Larson
Yoram Bachrach

Multiagent reinforcement learning (MARL) has benefited significantly from population-based and game-theoretic training regimes. One approach, Policy-Space Response Oracles (PSRO), employs standard reinforcement learning to compute response policies via approximate best responses and combines them via meta-strategy selection. We augment PSRO by adding a novel search procedure with generative sampling of world states, and introduce two new meta-strategy solvers based on the Nash bargaining solution. We evaluate PSRO’s ability to compute approximate Nash equilibrium, and its performance in negotiation games: Colored Trails and Dealor-no-Deal. We conduct behavioral studies where human participants negotiate with our agents (𝑁 = 346). Search with generative modeling finds stronger policies during both training time and test time, enables online Bayesian co-player prediction, and can produce agents that achieve comparable social welfare negotiating with humans as humans trading among themselves.

PDF

JAIR Journal 2022 Journal Article

Evolutionary Dynamics and Phi-Regret Minimization in Games

Georgios Piliouras
Mark Rowland
Shayegan Omidshafiei
Romuald Elie
Daniel Hennes
Jerome Connor
Karl Tuyls

Regret has been established as a foundational concept in online learning, and likewise has important applications in the analysis of learning dynamics in games. Regret quantifies the difference between a learner’s performance against a baseline in hindsight. It is well known that regret-minimizing algorithms converge to certain classes of equilibria in games; however, traditional forms of regret used in game theory predominantly consider baselines that permit deviations to deterministic actions or strategies. In this paper, we revisit our understanding of regret from the perspective of deviations over partitions of the full mixed strategy space (i.e., probability distributions over pure strategies), under the lens of the previously-established Φ-regret framework, which provides a continuum of stronger regret measures. Importantly, Φ-regret enables learning agents to consider deviations from and to mixed strategies, generalizing several existing notions of regret such as external, internal, and swap regret, and thus broadening the insights gained from regret-based analysis of learning algorithms. We prove here that the well-studied evolutionary learning algorithm of replicator dynamics (RD) seamlessly minimizes the strongest possible form of Φ-regret in generic 2 × 2 games, without any modification of the underlying algorithm itself. We subsequently conduct experiments validating our theoretical results in a suite of 144 2 × 2 games wherein RD exhibits a diverse set of behaviors. We conclude by providing empirical evidence of Φ-regret minimization by RD in some larger games, hinting at further opportunity for Φ-regret based study of such algorithms from both a theoretical and empirical perspective.

PDF Details DOI

ICLR Conference 2022 Conference Paper

NeuPL: Neural Population Learning

Siqi Liu 0002
Luke Marris
Daniel Hennes
Josh Merel
Nicolas Heess
Thore Graepel

Learning in strategy games (e.g. StarCraft, poker) requires the discovery of diverse policies. This is often achieved by iteratively training new policies against existing ones, growing a policy population that is robust to exploit. This iterative approach suffers from two issues in real-world games: a) under finite budget, approximate best-response operators at each iteration needs truncating, resulting in under-trained good-responses populating the population; b) repeated learning of basic skills at each iteration is wasteful and becomes intractable in the presence of increasingly strong opponents. In this work, we propose Neural Population Learning (NeuPL) as a solution to both issues. NeuPL offers convergence guarantees to a population of best-responses under mild assumptions. By representing a population of policies within a single conditional model, NeuPL enables transfer learning across policies. Empirically, we show the generality, improved performance and efficiency of NeuPL across several test domains. Most interestingly, we show that novel strategies become more accessible, not less, as the neural population expands.

Details

JAIR Journal 2021 Journal Article

Game Plan: What AI can do for Football, and What Football can do for AI

Karl Tuyls
Shayegan Omidshafiei
Paul Muller
Zhe Wang
Jerome Connor
Daniel Hennes
Ian Graham
William Spearman

The rapid progress in artificial intelligence (AI) and machine learning has opened unprecedented analytics possibilities in various team and individual sports, including baseball, basketball, and tennis. More recently, AI techniques have been applied to football, due to a huge increase in data collection by professional teams, increased computational power, and advances in machine learning, with the goal of better addressing new scientific challenges involved in the analysis of both individual players’ and coordinated teams’ behaviors. The research challenges associated with predictive and prescriptive football analytics require new developments and progress at the intersection of statistical learning, game theory, and computer vision. In this paper, we provide an overarching perspective highlighting how the combination of these fields, in particular, forms a unique microcosm for AI research, while offering mutual benefits for professional teams, spectators, and broadcasters in the years to come. We illustrate that this duality makes football analytics a game changer of tremendous value, in terms of not only changing the game of football itself, but also in terms of what this domain can mean for the field of AI. We review the state-of-the-art and exemplify the types of analysis enabled by combining the aforementioned fields, including illustrative examples of counterfactual analysis using predictive models, and the combination of game-theoretic analysis of penalty kicks with statistical learning of player attributes. We conclude by highlighting envisioned downstream impacts, including possibilities for extensions to other sports (real and virtual).

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

Which priors matter? Benchmarking models for learning latent dynamics

Aleksandar Botev
Andrew Jaegle
Peter Wirnsberger
Daniel Hennes
Irina Higgins

Learning dynamics is at the heart of many important applications of machine learning (ML), such as robotics and autonomous driving. In these settings, ML algorithms typically need to reason about a physical system using high dimensional observations, such as images, without access to the underlying state. Recently, several methods have proposed to integrate priors from classical mechanics into ML models to address the challenge of physical reasoning from images. In this work, we take a sober look at the current capabilities of these models. To this end, we introduce a suite consisting of 17 datasets with visual observations based on physical systems exhibiting a wide range of dynamics. We conduct a thorough and detailed comparison of the major classes of physically inspired methods alongside several strong baselines. While models that incorporate physical priors can often learn latent spaces with desirable properties, our results demonstrate that these methods fail to significantly improve upon standard techniques. Nonetheless, we find that the use of continuous and time-reversible dynamics benefits models of all classes.

PDF Details

ICLR Conference 2020 Conference Paper

A Generalized Training Approach for Multiagent Learning

Paul Muller
Shayegan Omidshafiei
Mark Rowland 0001
Karl Tuyls
Julien Pérolat
Siqi Liu 0002
Daniel Hennes
Luke Marris

This paper investigates a population-based training regime based on game-theoretic principles called Policy-Spaced Response Oracles (PSRO). PSRO is general in the sense that it (1) encompasses well-known algorithms such as fictitious play and double oracle as special cases, and (2) in principle applies to general-sum, many-player games. Despite this, prior studies of PSRO have been focused on two-player zero-sum games, a regime where in Nash equilibria are tractably computable. In moving from two-player zero-sum games to more general settings, computation of Nash equilibria quickly becomes infeasible. Here, we extend the theoretical underpinnings of PSRO by considering an alternative solution concept, α-Rank, which is unique (thus faces no equilibrium selection issues, unlike Nash) and applies readily to general-sum, many-player settings. We establish convergence guarantees in several games classes, and identify links between Nash equilibria and α-Rank. We demonstrate the competitive performance of α-Rank-based PSRO against an exact Nash solver-based PSRO in 2-player Kuhn and Leduc Poker. We then go beyond the reach of prior PSRO applications by considering 3- to 5-player poker games, yielding instances where α-Rank achieves faster convergence than approximate Nash solvers, thus establishing it as a favorable general games solver. We also carry out an initial empirical validation in MuJoCo soccer, illustrating the feasibility of the proposed approach in another complex domain.

Details

ICML Conference 2020 Conference Paper

Fast computation of Nash Equilibria in Imperfect Information Games

Rémi Munos
Julien Pérolat
Jean-Baptiste Lespiau
Mark Rowland 0001
Bart De Vylder
Marc Lanctot
Finbarr Timbers
Daniel Hennes

We introduce and analyze a class of algorithms, called Mirror Ascent against an Improved Opponent (MAIO), for computing Nash equilibria in two-player zero-sum games, both in normal form and in sequential form with imperfect information. These algorithms update the policy of each player with a mirror-ascent step to maximize the value of playing against an improved opponent. An improved opponent can be a best response, a greedy policy, a policy improved by policy gradient, or by any other reinforcement learning or search techniques. We establish a convergence result of the last iterate to the set of Nash equilibria and show that the speed of convergence depends on the amount of improvement offered by these improved policies. In addition, we show that under some condition, if we use a best response as improved policy, then an exponential convergence rate is achieved.

Details

ICRA Conference 2019 Conference Paper

Active Multi-Contact Continuous Tactile Exploration with Gaussian Process Differential Entropy

Danny Driess
Daniel Hennes
Marc Toussaint

In the present work, we propose an active tactile exploration framework to obtain a surface model of an unknown object utilizing multiple contacts simultaneously. To incorporate these multiple contacts, the exploration strategy is based on the differential entropy of the underlying Gaussian process implicit surface model, which formalizes the exploration with multiple contacts within an information theoretic context and additionally allows for nonmyopic multi-step planning. In contrast to many previous approaches, the robot continuously slides along the surface with its end-effectors to gather the tactile stimuli, instead of touching it at discrete locations. This is realized by closely integrating the surface model into the compliant controller framework. Furthermore, we extend our recently proposed sliding based tactile exploration approach to handle non-convex objects. In the experiments, it is shown that multiple contacts simultaneously leads to a more efficient exploration of complex, non-convex objects, not only in terms of time, but also with respect to the total moved distance of all end-effectors. Finally, we demonstrate our methodology with a real PR2 robot that explores an object with both of its arms.

Details

ICRA Conference 2018 Conference Paper

Learning to Control Redundant Musculoskeletal Systems with Neural Networks and SQP: Exploiting Muscle Properties

Danny Driess
Heiko Zimmermann
Simon Wolfen
Dan Suissa
Daniel F. B. Häufle
Daniel Hennes
Marc Toussaint
Syn Schmitt

Modeling biomechanical musculoskeletal systems reveals that the mapping from muscle stimulations to movement dynamics is highly nonlinear and complex, which makes it difficult to control those systems with classical techniques. In this work, we not only investigate whether machine learning approaches are capable of learning a controller for such systems. We are especially interested in the question if the structure of the musculoskeletal apparatus exhibits properties that are favorable for the learning task. In particular, we consider learning a control policy from target positions to muscle stimulations. To account for the high actuator redundancy of biomechanical systems, our approach uses a learned forward model represented by a neural network and sequential quadratic programming to obtain the control policy, which also enables us to alternate the co-contraction level and hence allows to change the stiffness of the system and to include optimality criteria like small muscle stimulations. Experiments on both a simulated musculoskeletal model of a human arm and a real biomimetic muscle-driven robot show that our approach is able to learn an accurate controller despite high redundancy and nonlinearity, while retaining sample efficiency.

Details

ICRA Conference 2017 Conference Paper

Gaussian process estimation of odometry errors for localization and mapping

Javier Hidalgo-Carrióo
Daniel Hennes
Jakob Schwendner
Frank Kirchner

Since early in robotics the performance of odometry techniques has been of constant research for mobile robots. This is due to its direct influence on localization. The pose error grows unbounded in dead-reckoning systems and its uncertainty has negative impacts in localization and mapping (i. e. SLAM). The dead-reckoning performance in terms of residuals, i. e. the difference between the expected and the real pose state, is related to the statistical error or uncertainty in probabilistic motion models. A novel approach to model odometry errors using Gaussian processes (GPs) is presented. The methodology trains a GP on the residual between the non-linear parametric motion model and the ground truth training data. The result is a GP over odometry residuals which provides an expected value and its uncertainty in order to enhance the belief with respect to the parametric model. The localization and mapping benefits from a comprehensive GP-odometry residuals model. The approach is applied to a planetary rover in an unstructured environment. We show that our approach enhances visual SLAM by efficiently computing image frames and effectively distributing keyframes.

Details

IROS Conference 2017 Conference Paper

NOctoSLAM: Fast octree surface normal mapping and registration

Joscha-David Fossel
Karl Tuyls
Benjamin Schnieders
Daniel Claes
Daniel Hennes

In this paper, we introduce a SLAM front end called NOctoSLAM. The approach adopts an octree-based map representation that implicitly enables source and reference data association for point to plane ICP registration. Additionally, the data structure is used to group map points to approximate surface normals. The multi-resolution capability of octrees, achieved by aggregating information in parent nodes, enables us to compensate for spatially unbalanced sensor data typically provided by multi-line lidar sensors. The octree-based data association is only approximate, but our empirical evaluation shows that NOctoSLAM achieves the same pose estimation accuracy as a comparable, point cloud based approach. However, NOctoSLAM can perform twice as many registration iterations per time unit. In contrast to point cloud based surface normal maps, where the map update duration depends on the current map size, we achieve a constant map update duration including surface normal recalculation. Therefore, NOctoSLAM does not require elaborate and environment dependent data filters. The results of our experiments show a mean positional error of 0. 029 m and 0. 019 rad, with a low standard deviation of 0. 005 m and 0. 006 rad, outperforming the state-of-the-art by remaining accurate while running online.

Details

ECAI Conference 2016 Conference Paper

Space Debris Removal: A Game Theoretic Analysis

Richard Klíma
Daan Bloembergen
Rahul Savani
Karl Tuyls
Daniel Hennes
Dario Izzo

We analyse active space debris removal efforts from a strategic, game-theoretic perspective. An active debris removal mission is a costly endeavour that has a positive effect (or risk reduction) for all satellites in the same orbital band. This leads to a dilemma: each actor (space agency, private stakeholder, etc.) has an incentive to delay its actions and wait for others to respond. The risk of the latter action is that, if everyone waits the joint outcome will be catastrophic leading to what in game theory is referred to as the 'tragedy of the commons'. We introduce and thoroughly analyse this dilemma using simulation and empirical game theory in a two player setting.

Details

JAIR Journal 2015 Journal Article

Evolutionary Dynamics of Multi-Agent Learning: A Survey

Daan Bloembergen
Karl Tuyls
Daniel Hennes
Michael Kaisers

The interaction of multiple autonomous agents gives rise to highly dynamic and nondeterministic environments, contributing to the complexity in applications such as automated financial markets, smart grids, or robotics. Due to the sheer number of situations that may arise, it is not possible to foresee and program the optimal behaviour for all agents beforehand. Consequently, it becomes essential for the success of the system that the agents can learn their optimal behaviour and adapt to new situations or circumstances. The past two decades have seen the emergence of reinforcement learning, both in single and multi-agent settings, as a strong, robust and adaptive learning paradigm. Progress has been substantial, and a wide range of algorithms are now available. An important challenge in the domain of multi-agent learning is to gain qualitative insights into the resulting system dynamics. In the past decade, tools and methods from evolutionary game theory have been successfully employed to study multi-agent learning dynamics formally in strategic interactions. This article surveys the dynamical models that have been derived for various multi-agent reinforcement learning algorithms, making it possible to study and compare them qualitatively. Furthermore, new learning algorithms that have been introduced using these evolutionary game theoretic tools are reviewed. The evolutionary models can be used to study complex strategic interactions. Examples of such analysis are given for the domains of automated trading in stock markets and collision avoidance in multi-robot systems. The paper provides a roadmap on the progress that has been achieved in analysing the evolutionary dynamics of multi-agent learning by highlighting the main results and accomplishments.

PDF Details DOI

IJCAI Conference 2015 Conference Paper

Interplanetary Trajectory Planning with Monte Carlo Tree Search

Daniel Hennes
Dario Izzo

Planning an interplanetary trajectory is a very complex task, traditionally accomplished by domain experts using computer-aided design tools. Recent advances in trajectory optimization allow automation of part of the trajectory design but have yet to provide an efficient way to select promising planetary encounter sequences. In this work, we present a heuristic-free approach to automated trajectory planning (including the encounter sequence planning) based on Monte Carlo Tree Search (MCTS). We discuss a number of modifications to traditional MCTS unique to the domain of interplanetary trajectory planning and provide results on the Rosetta and Cassini-Huygens interplanetary mission design problems. The resulting heuristic-free method is found to be orders of magnitude more efficient with respect to a standard tree search with heuristicbased pruning which is the current state-of-the art in this domain.

PDF Details

TIST Journal 2015 Journal Article

Metastrategies in Large-Scale Bargaining Settings

Daniel Hennes
Steven De Jong
Karl Tuyls
Ya’akov (Kobi) Gal

This article presents novel methods for representing and analyzing a special class of multiagent bargaining settings that feature multiple players, large action spaces, and a relationship among players’ goals, tasks, and resources. We show how to reduce these interactions to a set of bilateral normal-form games in which the strategy space is significantly smaller than the original settings while still preserving much of their structural relationship. The method is demonstrated using the Colored Trails (CT) framework, which encompasses a broad family of games and has been used in many past studies. We define a set of heuristics (metastrategies) in multiplayer CT games that make varying assumptions about players’ strategies, such as boundedly rational play and social preferences. We show how these CT settings can be decomposed into canonical bilateral games such as the Prisoners’ Dilemma, Stag Hunt, and Ultimatum games in a way that significantly facilitates their analysis. We demonstrate the feasibility of this approach in separate CT settings involving one-shot and repeated bargaining scenarios, which are subsequently analyzed using evolutionary game-theoretic techniques. We provide a set of necessary conditions for CT games for allowing this decomposition. Our results have significance for multiagent systems researchers in mapping large multiplayer CT task settings to smaller, well-known bilateral normal-form games while preserving some of the structure of the original setting.

Details DOI

AAMAS Conference 2013 Conference Paper

OctoSLAM: A 3D Mapping Approach to Situational Awareness of Unmanned Aerial Vehicles

Joscha-David Fossel
Daniel Hennes
Sjriek Alers
Daniel Claes
Karl Tuyls

Unmanned aerial vehicles (UAVs) have recently become widely available to the research community. A common vision is that such (semi-)autonomous airborne agents can be beneficial in numerous scenarios, e. g. urban search and rescue. However, when deploying computationally restricted UAVs in these real life scenarios, various challenges from multiple research domains arise. These include situational awareness, controlling, planning, and learning. The focus of this demonstration is on situational awareness of agents capable of 6D motion, in particular UAVs. We propose the integration of 2D laser range finder, altitude, and attitude sensor data to compose 3D maps of the environment. Experiments show significant improvement in the localization and representation accuracy over current 2D map methods.

PDF

AAMAS Conference 2012 Conference Paper

CALU: Collision Avoidance with Localization Uncertainty

Daniel Claes
Daniel Hennes
Karl Tuyls
Wim Meeussen

CALU is a multi-robot collision avoidance system based on the velocity obstacle paradigm. In contrast to previous approaches, we alleviate the strong requirement for perfect sensing (i. e. global positioning) using Adaptive Monte-Carlo Localization on a per-agent level.

PDF

IROS Conference 2012 Conference Paper

Collision avoidance under bounded localization uncertainty

Daniel Claes
Daniel Hennes
Karl Tuyls
Wim Meeussen

We present a multi-mobile robot collision avoidance system based on the velocity obstacle paradigm. Current positions and velocities of surrounding robots are translated to an efficient geometric representation to determine safe motions. Each robot uses on-board localization and local communication to build the velocity obstacle representation of its surroundings. Our close and error-bounded convex approximation of the localization density distribution results in collision-free paths under uncertainty. While in many algorithms the robots are approximated by circumscribed radii, we use the convex hull to minimize the overestimation in the footprint. Results show that our approach allows for safe navigation even in densely packed environments.

Details

AAMAS Conference 2012 Conference Paper

MITRO: an augmented mobile telepresence robot with assisted control

Sjriek Alers
Daan Bloembergen
Max B
uuml; gler
Daniel Hennes
Karl Tuyls

We present MITRO: Maastricht Intelligent Telepresence RObot, a custom-built robot system specifically designed for augmented telepresence with assisted control. Telepresence robots can be deployed in a wide range of application domains, and augmented presence with assisted control can greatly improve the experience for the user.

PDF

AAMAS Conference 2012 Conference Paper

Multi-robot collision avoidance with localization uncertainty

Daniel Hennes
Daniel Claes
Wim Meeussen
Karl Tuyls

This paper describes a multi-robot collision avoidance system based on the velocity obstacle paradigm. In contrast to previous approaches, we alleviate the strong requirement for perfect sensing (i. e. global positioning) using Adaptive Monte-Carlo Localization on a per-agent level. While such methods as Optimal Reciprocal Collision Avoidance guarantee local collision-free motion for a large number of robots, given perfect knowledge of positions and speeds, a realistic implementation requires further extensions to deal with inaccurate localization and message passing delays. The presented algorithm bounds the error introduced by localization and combines the computation for collision-free motion with localization uncertainty. We provide an open source implementation using the Robot Operating System (ROS). The system is tested and evaluated with up to eight robots in simulation and on four differential drive robots in a real-world situation.

PDF

AAMAS Conference 2011 Conference Paper

Bee-Inspired Foraging In An Embodied Swarm

Sjriek Alers
Daan Bloembergen
Daniel Hennes
Steven De Jong
Michael Kaisers
Nyree Lemmens
Karl Tuyls
Gerhard Weiss

We show the emergence of Swarm Intelligence in physical robots. We transfer an optimization algorithm which is based on beeforaging behavior to a robotic swarm. In simulation this algorithm has already been shown to be more effective, scalable and adaptive than algorithms inspired by ant foraging. In addition to this advantage, bee-inspired foraging does not require (de-)centralized simulation of environmental parameters (e. g. pheromones).

PDF

IROS Conference 2011 Conference Paper

Hierarchies of octrees for efficient 3D mapping

Kai M. Wurm
Daniel Hennes
Dirk Holz
Radu Bogdan Rusu
Cyrill Stachniss
Kurt Konolige
Wolfram Burgard

The on-chip fabrication and manipulation of microstructures are expected to be applied for single cell analysis system such as cell manipulation and measurement tools. In this paper, we previously present a methodology for fabricating and assembling microstructures inside a microfluidic channel. By the illumination of patterned UV-ray through the mask under a microscope, microstructures with arbitrary shape are made of the photo-crosslinkable resin inside microfluidic device. The microstructures are fabricated at the desired place inside microfluidic channel and manipulated by optical tweezers. Based on the technique which can manipulate multiple points simultaneously by high-speed scanning of a single laser with galvanometer mirror, a rotational microstructure made of a microgear and a rotation axis is assembled and rotated. We also report two methods of solution replacement inside microfluidic channel which reduces viscosity of solvent in order to improve manipulation performance. By adjusting the concentration of photo-crosslinkable resin and replacing solution components, the viscosity of solvent inside channel can be changed. The manipulation speed of the rotational microstructure increases when the viscosity of solvent decreases, because the viscosity resistance for the movement of microstructure is weaker inside lower viscosity solvent. We fabricate rotational microstructures inside lower viscosity solvent and evaluate the movement efficiency compared with microstructures inside former high viscosity solvent.

Details

AAMAS Conference 2011 Conference Paper

Metastrategies in the Colored Trails Game

Steven De Jong
Daniel Hennes
Karl Tuyls
Ya'akov (Kobi) Gal

This paper presents a novel method to describe and analyze strategic interactions in settings that include multiple actors, many possible actions and relationships among goals, tasks and resources. It shows how to reduce these large interactions to a set of bilateral normal-form games in which the strategy space is significantly smaller than the original setting, while still preserving many of its strategic characteristics. We demonstrate this technique on the Colored Trails (CT) framework, which encompasses a broad family of games defining multi-agent interactions and has been used in many past studies. We define a set of representative heuristics in a three-player CT setting. Choosing players' strategies from this set, the original CT setting is analytically decomposed into canonical bilateral social dilemmas, i. e. , Prisoners' Dilemma, Stag Hunt and Ultimatum games. We present a set of criteria for generating strategically interesting CT games and empirically show that they indeed decompose into bilateral social dilemmas if players play according to the heuristics. Our results have significance for multi-agent systems researchers in mapping large multi-player task settings to well-known bilateral normal-form games in a way that facilitates the analysis of the original setting.

PDF

AAMAS Conference 2009 Conference Paper

State-Coupled Replicator Dynamics

Daniel Hennes
Karl Tuyls
Matthias Rauterberg

This paper introduces a new model, i. e. state-coupled replicator dynamics, expanding the link between evolutionary game theory and multiagent reinforcement learning to multistate games. More precisely, it extends and improves previous work on piecewise replicator dynamics, a combination of replicators and piecewise models. The contributions of the paper are twofold. One, we identify and explain the major shortcomings of piecewise replicators, i. e. discontinuities and occurrences of qualitative anomalies. Two, this analysis leads to the proposal of the new model for learning dynamics in stochastic games, named state-coupled replicator dynamics. The preceding formalization of piecewise replicators general in the number of agents and states - is factored into the new approach. Finally, we deliver a comparative study of ﬁnite action-set learning automata to piecewise and state-coupled replicator dynamics. Results show that statecoupled replicators model learning dynamics in stochastic games more accurately than their predecessor, the piecewise approach.

PDF