Koen V. Hindriks

JAAMAS Journal 2026 Journal Article

Agent Programming in 3APL

• Koen V. Hindriks
• Frank S. De Boer
• John-Jules Ch. Meyer

Abstract An intriguing and relatively new metaphor in the programming community is that of an intelligent agent. The idea is to view programs as intelligent agents acting on our behalf. By using the metaphor of intelligent agents the programmer views programs as entities which have a mental state consisting of beliefs and goals. The computational behaviour of an agent is explained in terms of the decisions the agent makes on the basis of its mental state. It is assumed that this way of looking at programs may enhance the design and development of complex computational systems. To support this new style of programming, we propose the agent programming language 3APL. 3APL has a clear and formally defined semantics. The operational semantics of the language is defined by means of transition systems. 3APL is a combination of imperative and logic programming. From imperative programming the language inherits the full range of regular programming constructs, including recursive procedures, and a notion of state-based computation. States of agents, however, are belief or knowledge bases, which are different from the usual variable assignments of imperative programming. From logic programming, the language inherits the proof as computation model as a basic means of computation for querying the belief base of an agent. These features are well-understood and provide a solid basis for a structured agent programming language. Moreover, on top of that 3APL agents use so-called practical reasoning rules which extend the familiar recursive rules of imperative programming in several ways. Practical reasoning rules can be used to monitor and revise the goals of an agent, and provide an agent with reflective capabilities. Applying the metaphor of intelligent agents means taking a design stance. From this perspective, a program is taken as an entity with a mental state, which acts pro-actively and reactively, and has reflective capabilities. We illustrate how the metaphor of intelligent agents is supported by the programming language. We also discuss the design of control structures for rule-based agent languages. A control structure provides a solution to the problem of which goals and which rules an agent should select. We provide a concrete and intuitive ordering on the practical reasoning rules on which such a selection mechanism can be based. The ordering is based on the metaphor of intelligent agents. Furthermore, we provide a language with a formal semantics for programming control structures. The main idea is not to integrate this language into the agent language itself, but to provide the facilities for programming control structures at a meta level. The operational semantics is accordingly specified at the meta level, by means of a meta transition system.

IROS Conference 2025 Conference Paper

Beware of the Tablet: A Dominant Distractor in Human-Robot Interaction

• Linlin Cheng
• Artem V. Belopolsky
• Mark de Bruijn
• Koen V. Hindriks

The present study aims at investigating how humans engage with common communication modalities—speech, tablet, and gesture—when interacting with a humanoid robot. To explore this, we designed a live interaction experiment using a congruence paradigm, where participants engaged with a robot presenting two out of three modalities simultaneously: one as the primary cue and the other as a distracting cue. We measured participants’ task performance (response time, error rate) and fixation distribution (fixation count and duration proportions) across different roles (primary, distracting, neither) and areas of interest (face, tablet, gesture). Additionally, we compared fixation patterns between the performance and baseline phases. Our findings reveal that while the tablet is the most effective modality for task engagement, it also serves as a strong attentional distractor, dominating gaze allocation regardless of its informational value. This underscores the importance of carefully balancing tablet integration in HRI design. Notably, our results demonstrate that gaze patterns alone do not fully reveal attentional focus, emphasizing the need to consider both overt and covert cognitive processes in multimodal HRI. These insights provide valuable guidelines for designing more effective and engaging human-robot interactions.

IROS Conference 2025 Conference Paper

Can Real-Time Lipreading Improve Speech Recognition? A Systematic Exploration Using Human-Robot Interaction Data

• Sander Goetzee
• Yue Li 0044
• Koen V. Hindriks

Speech recognition in Human-Robot Interaction (HRI) fully relies on audio-based Automatic Speech Recognition. However, speech recognition that relies solely on audio faces significant challenges in noisy environments and may lead to poor performance in such environments. One approach to address this is to also use lipreading in combination with traditional speech recognition. Recent work has shown that audiovisual speech recognition (AVSR) can achieve a Word Error Rate (WER) of only 0. 9% on the dataset LRS3. In this paper, we assess the potential of combining audio with lipreading on a social robot platform, Pepper, which has not yet been widely tested for AVSR. Given that prior research has focused on non-robotic domains, it remains unclear whether such models can generalize well to social robot environments. We systematically evaluate and compare the performance of established offline and real-time audiovisual models with their audio-only counterparts. The experiments were conducted in both a controlled laboratory setting and a dynamic and noisy public environment. We evaluated the data using WER and also measured the inference latency of real-time models via Real-Time Factor and Words Per Second rates. The results demonstrate real-time performance for audio-only speech recognition across all latency metrics and near real-time performance for models that combine audio with lipreading. We also explored factors that might influence the inference performance of these models to understand how much video contributes to the audio. This includes factors related to (1) environmental and temporal variations, (2) model behavior, and (3) implementation choices. Our findings indicate that for now the audio-only models outperform the audiovisual models on a social robot platform, in contrast to what has been reported in the benchmarked literature. We conclude that more work is still needed to benefit from lipreading in HRI.

ICRA Conference 2025 Conference Paper

Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approach

• Muhan Hou
• Koen V. Hindriks
• A. E. Eiben
• Kim Baraka

Transfer Learning (TL) is a powerful tool that enables robots to transfer learned policies across different environments, tasks, or embodiments. To further facilitate this process, efforts have been made to combine it with Learning from Demonstrations (LfD) for more flexible and efficient policy transfer. However, these approaches are almost exclusively limited to offline demonstrations collected before policy transfer starts, which may suffer from the intrinsic issue of covariance shift brought by LfD and harm the performance of policy transfer. Meanwhile, extensive work in the learning-from-scratch setting has shown that online demonstrations can effectively alleviate covariance shift and lead to better policy performance with improved sample efficiency. This work combines these insights to introduce online demonstrations into a policy transfer setting. We present Policy Transfer with Online Demonstrations, an active LfD algorithm for policy transfer that can optimize the timing and content of queries for online episodic expert demonstrations under a limited demonstration budget. We evaluate our method in eight robotic scenarios, involving policy transfer across diverse environment characteristics, task objectives, and robotic embodiments, with the aim to transfer a trained policy from a source task to a related but different target task. The results show that our method significantly outperforms all baselines in terms of average success rate and sample efficiency, compared to two canonical LfD methods with offline demonstrations and one active LfD method with online demonstrations. Additionally, we conduct preliminary sim-to-real tests of the transferred policy on three transfer scenarios in the real-world environment, demonstrating the policy effectiveness on a real robot manipulator.

AAMAS Conference 2019 Conference Paper

A Child and a Robot Getting Acquainted - Interaction Design for Eliciting Self-Disclosure

• Mike Ligthart
• Timo Fernhout
• Mark A. Neerincx
• Kelly L. A. van Bindsbergen
• Martha A. Grootenhuis
• Koen V. Hindriks

In order to facilitate a sustainable long-term interaction between a child and a robot they need to get acquainted with one another. In this paper we discuss the foundation, the rationale, and the evaluation (N = 75) of our design for an autonomous robot conversational partner that engages with Dutch children (8-11 y. o.) in a getting acquainted interaction. The main objective of the robot is to elicit children to self-disclose. Firstly, we discuss five interaction design patterns (IDPs) that proved to be successful in autonomously eliciting and processing self-disclosures. Secondly, we compared two robot behavior profiles. The behavior profiles can be relatively considered as being more and less energetic. We manipulated the movement speed, the speech rate and volume, the use of high/low energy language, waiting time before responding, and the order of high/low energy activities. Results show that the less energetic behavior profile significantly leads to more self-disclosure.

IS Journal 2019 Journal Article

A Formal Graphical Language of Interdependence in Teamwork

• Changyun Wei
• Koen V. Hindriks
• M. Birna van Riemsdijk
• Catholijn M. Jonker

Agents in teamwork may be highly interdependent on each other, and the awareness of interdependences is an important requirement for designing and consequently implementing a multiagent system. In this article, we propose a formal graphical and domain-independent language that can facilitate the identification of comprehensive interdependences among the agents in teamwork. Moreover, a formal semantics is also introduced to precisely express and explain the properties of a graphical structure. The novel feature of the graphical language is that it complements the Interdependence Analysis Color Scheme in a way that explicitly models negative influences and, in addition, provides a visual-communication aid for developers. To demonstrate the applicability and sufficiency of the graphical language in a variety of domains, our case studies include a multirobot scenario and a human-robot scenario.

IROS Conference 2019 Conference Paper

Enthusiastic Robots Make Better Contact

• Elie Saad
• Joost Broekens
• Mark A. Neerincx
• Koen V. Hindriks

This paper presents the design and evaluation of human-like welcoming behaviors for a humanoid robot to draw the attention of passersby by following a three-step model: (1) selecting a target (person) to engage, (2) executing behaviors to draw the target’s attention, and (3) monitoring the attentive response. A computer vision algorithm was developed to select the person, start the behaviors and monitor the response automatically. To vary the robot’s enthusiasm when engaging passersby, a waving gesture was designed as basic welcoming behavioral element, which could be successively combined with an utterance and an approach movement. This way, three levels of enthusiasm were implemented: Mild (waving), moderate (waving and utterance) and high (waving, utterance and approach movement). The three levels of welcoming behaviors were tested with a Pepper robot at the entrance of a university building. We recorded data and observation sheets from several hundreds of passersby (N =364) and conducted post-interviews with randomly selected passersby (N =28). The level selection was done at random for each participant. The passersby indicated that they appreciated the robot at the entrance and clearly recognized its role as a welcoming robot. In addition, the robot proved to draw more attention when showing high enthusiasm (i. e. , more welcoming behaviors), particularly for female passersby.

AAMAS Conference 2019 Conference Paper

Recognising and Explaining Bidding Strategies in Negotiation Support Systems

• Vincent J. Koeman
• Koen V. Hindriks
• Jonathan Gratch
• Catholijn M. Jonker

To improve a negotiator’s ability to recognise bidding strategies, we pro-actively provide explanations that are based on the opponent’s bids and the negotiator’s guesses about the opponent’s strategy. We introduce an aberration detection mechanism for recognising strategies and the notion of an explanation matrix. The aberration detection mechanism identifies when a bid falls outside the range of expected behaviour for a specific strategy. The explanation matrix is used to decide when to provide what explanations. We evaluated our work experimentally in a task in which participants are asked to identify their opponent’s strategy in the environment of a negotiation support system, namely the Pocket Negotiator (PN). We implemented our explanation mechanism in the PN and experimented with different explanation matrices. As the number of correct guesses increases with explanations, indirectly, these experiments show the effectiveness of our aberration detection mechanism. Our experiments with over 100 participants show that suggesting consistent strategies is more effective than explaining why observed behaviour is inconsistent.

AAMAS Conference 2018 Conference Paper

StarCraft as a Testbed for Engineering Complex Distributed Systems Using Cognitive Agent Technology

• Vincent J. Koeman
• Harm J. Griffioen
• Danny C. Plenge
• Koen V. Hindriks

It has been argued that the evaluation of cognitive agent systems requires richer benchmark problems. We think that real-time strategy (RTS) games can offer such a testbed, as AI for RTS requires the design of complicated strategies for coordinating hundreds of units that need to solve a range of challenges. Therefore, in this paper, we report on the design and development of the first multi-agent connector that provides full access to StarCraft (Brood War). We provide a new interface that is dedicated to a multi-agent approach by connecting each unit in the game to a cognitive agent. Two main challenges are addressed in this work. First, we decide on the right level of abstraction for unit control by means of agents, designing for instance the percepts that are available to units. Second, a sufficient level of performance needs to be ensured in order to allow a large variety of multi-agent implementations to be successful at tackling challenges of RTS AI. The resulting open-source connector readily supports the hundreds of agents that can come and go during the game. Based on the development of the connector and its initial use by over 200 students, we gained valuable insights.

IJCAI Conference 2017 Conference Paper

Omniscient Debugging for Cognitive Agent Programs

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

For real-time programs reproducing a bug by rerunning the system is likely to fail, making fault localization a time-consuming process. Omniscient debugging is a technique that stores each run in such a way that it supports going backwards in time. However, the overhead of existing omniscient debugging implementations for languages like Java is so large that it cannot be effectively used in practice. In this paper, we show that for agent-oriented programming practical omniscient debugging is possible. We design a tracing mechanism for efficiently storing and exploring agent program runs. We are the first to demonstrate that this mechanism does not affect program runs by empirically establishing that the same tests succeed or fail. Usability is supported by a trace visualization method aimed at more effectively locating faults in agent programs.

IJCAI Conference 2017 Conference Paper

Omniscient Debugging for GOAL Agents in Eclipse (Demonstration)

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

The main goal of our demonstration is to show how omniscient debugging can be applied in practice to cognitive agents. A concrete implementation of the mechanisms proposed in Koeman et. al [2017] has been created for the GOAL agent programming language in the Eclipse environment, integrated with the source-level debugger of Koeman et. al [2016], thus fully implementing the proposal within a state-of-the-art setting. The implementation will be used together with typical agent programs to demonstrate its practical use.

AAMAS Conference 2016 Conference Paper

A Survey of Opponent Modeling Techniques in Automated Negotiation

• Tim Baarslag
• Mark J. C. Hendrikx
• Koen V. Hindriks
• Catholijn M. Jonker

EUMAS Conference 2016 Invited Paper

An Introduction to the Pocket Negotiator: A General Purpose Negotiation Support System

• Catholijn M. Jonker
• Reyhan Aydogan
• Tim Baarslag
• Joost Broekens
• Christian A. Detweiler
• Koen V. Hindriks
• Alina Huldtgren
• Wouter Pasman

Abstract The Pocket Negotiator (PN) is a negotiation support system developed at TU Delft as a tool for supporting people in bilateral negotiations over multi-issue negotiation problems in arbitrary domains. Users are supported in setting their preferences, estimating those of their opponent, during the bidding phase and sealing the deal. We describe the overall architecture, the essentials of the underlying techniques, the form that support takes during the negotiation phases, and we share evidence of the effectiveness of the Pocket Negotiator.

AAMAS Conference 2016 Conference Paper

Automating Failure Detection in Cognitive Agent Programs

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

Debugging is notoriously difficult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specification of test conditions and show that the test language is sufficiently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.

ECAI Conference 2016 Conference Paper

Boolean Negotiation Games

• Nils Bulling
• Koen V. Hindriks

We propose a new strategic model of negotiation, called Boolean negotiation games. Our model is inspired by Boolean games and the alternating offers model of bargaining. It offers a computationally grounded model for studying properties of negotiation protocols in a qualitative setting. Boolean negotiation games can yield agreements that are more beneficial than stable solutions (Nash equilibria) of the underlying Boolean game.

JAAMAS Journal 2016 Journal Article

Designing a source-level debugger for cognitive agent programs

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

Abstract When an agent program exhibits unexpected behaviour, a developer needs to locate the fault by debugging the agent’s source code. The process of fault localisation requires an understanding of how code relates to the observed agent behaviour. The main aim of this paper is to design a source-level debugger that supports single-step execution of a cognitive agent program. Cognitive agents execute a decision cycle in which they process events and derive a choice of action from their beliefs and goals. Current state-of-the-art debuggers for agent programs provide insight in how agent behaviour originates from this cycle but less so in how it relates to the program code. As relating source code to generated behaviour is an important part of the debugging task, arguably, a developer also needs to be able to suspend an agent program on code locations. We propose a design approach for single-step execution of agent programs that supports both code-based as well as cycle-based suspension of an agent program. This approach results in a concrete stepping diagram ready for implementation and is illustrated by a diagram for both the Goal and Jason agent programming languages, and a corresponding full implementation of a source-level debugger for Goal in the Eclipse development environment. The evaluation that was performed based on this implementation shows that agent programmers prefer a source-level debugger over a purely cycle-based debugger.

AAMAS Conference 2016 Conference Paper

Using Automatic Failure Detection for Cognitive Agents in Eclipse (Demonstration)

• Vincent J. Koeman
• Koen V. Hindriks
• Catholijn M. Jonker

IROS Conference 2015 Conference Paper

On the need for a coordination mechanism to guarantee task completion in a cooperative team

• Chris Rozemuller
• Koen V. Hindriks
• Mark A. Neerincx

To design good cooperative team members in robotics it is important to know what coordination mechanisms are required. Our approach to explore the need for a coordination mechanism is based on a systematic methodology to identify team coordination requirements. We show that a team combined of robots that each individually can solve a task not always is able to guarantee task completion as a team. In these cases some mechanism for coordination is required and we formally identify various problem classes that impose different requirements. We introduce a formal task model and distinguish between no, implicit and explicit coordination mechanisms. This model is used to study which mechanisms guarantee task completion. It allows us to prove some empirical findings reported in the literature such as that a simple foraging task does not require coordination.

EUMAS Conference 2014 Conference Paper

Auction-Based Dynamic Task Allocation for Foraging with a Cooperative Robot Team

• Changyun Wei
• Koen V. Hindriks
• Catholijn M. Jonker

Abstract Many application domains require search and retrieval, which is also known in the robotic domain as foraging. An example domain is search and rescue where a disaster area needs to be explored and transportation of survivors to a safe area needs to be arranged. Performing these tasks by more than one robot increases performance if tasks are allocated and executed efficiently. In this paper, we study the Multi-Robot Task Allocation (MRTA) problem in the foraging domain. We assume that a team of robots is cooperatively searching for targets of interest in an environment which need to be retrieved and brought back to a home base. We look at a more general foraging problem than is typically studied where coordination also requires to take temporal constraints into account. As usual, robots have no prior knowledge about the location of targets, but in addition need to deliver targets to the home base in a specific order. This significantly increases the complexity of a foraging problem. We use a graph-based model to analyse the problem and the dynamics of allocating exploration and retrieval tasks. Our main contribution is an extension of auction-based approaches to deal with dynamic foraging task allocation where not all tasks are initially known. We use the Blocks World for Teams (BW4T) simulator to evaluate the proposed approach.

IROS Conference 2014 Conference Paper

Effects of bodily mood expression of a robotic teacher on students

• Junchao Xu
• Joost Broekens
• Koen V. Hindriks
• Mark A. Neerincx

This paper reports our investigation into the effects of bodily mood expression of a humanoid robot in a scenario close to real life. To this end, we used the NAO robot to perform as a lecturer in a university class. To display either a positive or a negative mood, we modulated 41 co-verbal gestures by adjusting behavior parameters that control spatial extent and motion dynamics, without modifying gesture function. Unique in this study is that (a) the robot gave an actual lecture to real students, (b) the interaction is one-to-many and relatively long (30 min), and (c) mood modulation was applied to a large set of behaviors. The robot presented the same lecture either in a positive or a negative mood to two audiences (between subjects). Although statistical analysis does not show that participants consciously recognized the robot mood, the results do show that participants in the positive mood condition rated their own arousal significantly higher than in the negative condition. Further, video annotation showed increased valence and arousal of the audience in the positive condition. Finally, participants' ratings of the lecturing quality and the gesture quality of the robot are higher in the positive condition, demonstrating the importance of robot mood expression in a one-to-many interaction setting.

ECAI Conference 2014 Conference Paper

The Significance of Bidding, Accepting and Opponent Modeling in Automated Negotiation

• Tim Baarslag
• Alexander Dirkzwager
• Koen V. Hindriks
• Catholijn M. Jonker

Given the growing interest in automated negotiation, the search for effective strategies has produced a variety of different negotiation agents. Despite their diversity, there is a common structure to their design. A negotiation agent comprises three key components: the bidding strategy, the opponent model and the acceptance criteria. We show that this three-component view of a negotiating architecture not only provides a useful basis for developing such agents but also provides a useful analytical tool. By combining these components in varying ways, we are able to demonstrate the contribution of each component to the overall negotiation result, and thus determine the key contributing components. Moreover, we study the interaction between components and present detailed interaction effects. Furthermore, we find that the bidding strategy in particular is of critical importance to the negotiator's success and far exceeds the importance of opponent preference modeling techniques. Our results contribute to the shaping of a research agenda for negotiating agent design by providing guidelines on how agent developers can spend their time most effectively.

IROS Conference 2013 Conference Paper

Robot learning and use of affordances in goal-directed tasks

• Chang Wang 0005
• Koen V. Hindriks
• Robert Babuska

An affordance is a relation between an object, an action, and the effect of that action in a given environmental context. One key benefit of the concept of affordance is that it provides information about the consequence of an action which can be stored and reused in a range of tasks that a robot needs to learn and perform. In this paper, we address the challenge of the on-line learning and use of affordances simultaneously while performing goal-directed tasks. This requires efficient online performance to ensure the robot is able to achieve its goal fast. By providing conceptual knowledge of action possibilities and desired effects, we show that a humanoid robot NAO can learn and use affordances in two different task settings. We demonstrate the effectiveness of this approach by integrating affordances into an Extended Classifier System for learning general rules in a reinforcement learning framework. Our experimental results show significant speedups in learning how a robot solves a given task.

AAMAS Conference 2011 Conference Paper

Taming the Complexity of Linear Time BDI Logics

• Nils Bulling
• Koen V. Hindriks

Reasoning about the mental states of agents is important in various settings, and has been recognized as vital for teamwork. But the complexity of some of the more well-known agent logics that facilitate reasoning about mental states prohibits the use of these logics in practice. An alternative is to investigate fragments of these logics that have a lower complexity but are still expressive enough for reasoning about the mental states of (other) agents. We explore this alternative and take as our starting point the linear time variant of BDI logic (BDI _ LTL ). We summarize some of the relevant known complexity results for e. g. LTL, KD45 _n, and BDI _ LTL itself. We present a tableau-based method for establishing complexity bounds, and provide a map of the complexity of (various fragments of) BDI _ LTL. Finally, we identify a few fragments that may be usefully applied for reasoning about mental states.

AAMAS Conference 2009 Conference Paper

Agent Programming with Temporally Extended Goals

• Koen V. Hindriks
• Wiebe van der Hoek
• M. Birna van Riemsdijk

In planning as well as in other areas, temporal logic has been used to specify so-called temporally extended goals. Temporally extended goals refer to desirable sequences of states instead of a set of desirable final states as the traditional notion of achievement goal does, and provide for more variety in the types of goals allowed. In this paper, we show how temporally extended goals can be integrated into the agent programming language GOAL. The result is that GOAL agents may now have both beliefs about the future as well as have temporally extended goals. We propose a new decision making mechanism that takes temporally extended goals into account, and investigate properties of this framework.

JELIA Conference 2008 Conference Paper

GOAL Agents Instantiate Intention Logic

• Koen V. Hindriks
• Wiebe van der Hoek

Abstract It is commonly believed there is a big gap between agent logics and computational agent frameworks. In this paper, we show that this gap is not as big as believed by showing that GOAL agents instantiate Intention Logic of Cohen and Levesque. That is, we show that GOAL agent programs can be formally related to Intention Logic. We do so by proving that the GOAL Verification Logic can be embedded into Intention Logic. It follows that (a fragment of) Intention Logic can be used to prove properties of GOAL agents. The work reported is an important step towards the application of standard tools from modal logic for e. g. model checking agent programs. Our results also prove useful for extending the expressiveness of the GOAL agent language. This is illustrated by incorporating temporally extended goals into GOAL agents.

ECAI Conference 2000 Conference Paper

An Embedding of ConGolog in 3APL

• Koen V. Hindriks
• Yves Lespérance
• Hector J. Levesque

ECAI Conference 2000 Conference Paper

Architecture for Agent Programming Languages

• Koen V. Hindriks
• Mark d'Inverno
• Michael Luck