Vincent J. Koeman

AAMAS Conference 2019 Conference Paper

Evaluating the Effectiveness of Multi-Agent Organisational Paradigms in a Real-Time Strategy Environment: Engineering Multiagent Systems Track

• Buster A. Bernstein
• Jasper C. M. Geurtz
• Vincent J. Koeman

We study the impact of using different organisational paradigms on the design and implementation of a Multi-Agent System (MAS) for Real-Time Strategy (RTS) games. We examine systems designed and implemented according to a specific paradigm on their performance in a practical scenario, as well as examining software-engineering concepts like size and complexity entailed by the according implementations. In contrast to related theoretical work, we deal with the practical constraints and implications of the paradigms by targeting the prototypical RTS game StarCraft: Brood War. Through careful analysis of this environment, agent systems for four separate paradigms that operate at different levels of autonomy and communication are designed, implemented, and evaluated by thousands of instrumented runs. One of the main findings is that using a central processing agent, e. g. in a market-based approach, increases task performance, but at the cost of increased code complexity.

AAMAS Conference 2019 Conference Paper

Recognising and Explaining Bidding Strategies in Negotiation Support Systems

• Vincent J. Koeman
• Koen V. Hindriks
• Jonathan Gratch
• Catholijn M. Jonker

To improve a negotiator’s ability to recognise bidding strategies, we pro-actively provide explanations that are based on the opponent’s bids and the negotiator’s guesses about the opponent’s strategy. We introduce an aberration detection mechanism for recognising strategies and the notion of an explanation matrix. The aberration detection mechanism identifies when a bid falls outside the range of expected behaviour for a specific strategy. The explanation matrix is used to decide when to provide what explanations. We evaluated our work experimentally in a task in which participants are asked to identify their opponent’s strategy in the environment of a negotiation support system, namely the Pocket Negotiator (PN). We implemented our explanation mechanism in the PN and experimented with different explanation matrices. As the number of correct guesses increases with explanations, indirectly, these experiments show the effectiveness of our aberration detection mechanism. Our experiments with over 100 participants show that suggesting consistent strategies is more effective than explaining why observed behaviour is inconsistent.

AAMAS Conference 2018 Conference Paper

StarCraft as a Testbed for Engineering Complex Distributed Systems Using Cognitive Agent Technology

• Vincent J. Koeman
• Harm J. Griffioen
• Danny C. Plenge
• Koen V. Hindriks

It has been argued that the evaluation of cognitive agent systems requires richer benchmark problems. We think that real-time strategy (RTS) games can offer such a testbed, as AI for RTS requires the design of complicated strategies for coordinating hundreds of units that need to solve a range of challenges. Therefore, in this paper, we report on the design and development of the first multi-agent connector that provides full access to StarCraft (Brood War). We provide a new interface that is dedicated to a multi-agent approach by connecting each unit in the game to a cognitive agent. Two main challenges are addressed in this work. First, we decide on the right level of abstraction for unit control by means of agents, designing for instance the percepts that are available to units. Second, a sufficient level of performance needs to be ensured in order to allow a large variety of multi-agent implementations to be successful at tackling challenges of RTS AI. The resulting open-source connector readily supports the hundreds of agents that can come and go during the game. Based on the development of the connector and its initial use by over 200 students, we gained valuable insights.

IJCAI Conference 2017 Conference Paper

Omniscient Debugging for Cognitive Agent Programs

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

For real-time programs reproducing a bug by rerunning the system is likely to fail, making fault localization a time-consuming process. Omniscient debugging is a technique that stores each run in such a way that it supports going backwards in time. However, the overhead of existing omniscient debugging implementations for languages like Java is so large that it cannot be effectively used in practice. In this paper, we show that for agent-oriented programming practical omniscient debugging is possible. We design a tracing mechanism for efficiently storing and exploring agent program runs. We are the first to demonstrate that this mechanism does not affect program runs by empirically establishing that the same tests succeed or fail. Usability is supported by a trace visualization method aimed at more effectively locating faults in agent programs.

IJCAI Conference 2017 Conference Paper

Omniscient Debugging for GOAL Agents in Eclipse (Demonstration)

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

The main goal of our demonstration is to show how omniscient debugging can be applied in practice to cognitive agents. A concrete implementation of the mechanisms proposed in Koeman et. al [2017] has been created for the GOAL agent programming language in the Eclipse environment, integrated with the source-level debugger of Koeman et. al [2016], thus fully implementing the proposal within a state-of-the-art setting. The implementation will be used together with typical agent programs to demonstrate its practical use.

AAMAS Conference 2016 Conference Paper

Automating Failure Detection in Cognitive Agent Programs

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

Debugging is notoriously difficult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specification of test conditions and show that the test language is sufficiently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.

JAAMAS Journal 2016 Journal Article

Designing a source-level debugger for cognitive agent programs

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

Abstract When an agent program exhibits unexpected behaviour, a developer needs to locate the fault by debugging the agent’s source code. The process of fault localisation requires an understanding of how code relates to the observed agent behaviour. The main aim of this paper is to design a source-level debugger that supports single-step execution of a cognitive agent program. Cognitive agents execute a decision cycle in which they process events and derive a choice of action from their beliefs and goals. Current state-of-the-art debuggers for agent programs provide insight in how agent behaviour originates from this cycle but less so in how it relates to the program code. As relating source code to generated behaviour is an important part of the debugging task, arguably, a developer also needs to be able to suspend an agent program on code locations. We propose a design approach for single-step execution of agent programs that supports both code-based as well as cycle-based suspension of an agent program. This approach results in a concrete stepping diagram ready for implementation and is illustrated by a diagram for both the Goal and Jason agent programming languages, and a corresponding full implementation of a source-level debugger for Goal in the Eclipse development environment. The evaluation that was performed based on this implementation shows that agent programmers prefer a source-level debugger over a purely cycle-based debugger.

AAMAS Conference 2016 Conference Paper

Using Automatic Failure Detection for Cognitive Agents in Eclipse (Demonstration)

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker