Author name cluster

Pieter Simoens

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

AAMAS Conference 2026 Conference Paper

Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment

Elias Malomgré
Pieter Simoens

AIalignmentisgrowinginimportance, yetmanycurrentapproaches learn safety behavior by directly modifying policy parameters, entangling normative constraints with the underlying policy. This oftenyieldsopaque, difficult-to-editalignmentartifactsandreduces their reuse across models or deployments, a failure mode we term Alignment Waste. We propose Interactionless Inverse Reinforcement Learning, a framework for learning inspectable, editable, and reusable reward artifacts separately from policy optimization. We further introduce the Alignment Flywheel, a human-in-the-loop lifecycle for iteratively auditing, patching, and hardening these artifacts through automated evaluation and refinement. Together, these ideas recast alignment from a disposable training expense into a durable, verifiable engineering asset.

PDF

ICRA Conference 2025 Conference Paper

Learning Task Specifications from Demonstrations as Probabilistic Automata

Mattijs Baert
Sam Leroux
Pieter Simoens

Specifying tasks for robotic systems traditionally requires coding expertise, deep domain knowledge, and significant time investment. While learning from demonstration offers a promising alternative, existing methods often struggle with tasks of longer horizons. To address this limitation, we introduce a computationally efficient approach for learning probabilistic deterministic finite automata (PDFA) that capture task structures and expert preferences directly from demonstrations. Our approach infers sub-goals and their temporal dependencies, producing an interpretable task specification that domain experts can easily understand and adjust. We validate our method through experiments involving object manipulation tasks, showcasing how our method enables a robot arm to effectively replicate diverse expert strategies while adapting to changing conditions.

Details

NeSy Conference 2023 Conference Paper

Learning Logic Constraints From Demonstration

Mattijs Baert
Sam Leroux
Pieter Simoens

Autonomous agents operating in real-world settings are often required to efficiently accomplish a task while adhering to certain environmental constraints. For instance, a self-driving car must transport its passengers to their intended destination as fast as possible while complying with traffic regulations. Inverse Constrained Reinforcement Learning (ICRL) is a technique that enables the learning of a policy from demonstrations of expert agents. When these expert agents adhere to the environmental constraints, ICRL thus allows for compliant policies to be learned without the need to define constraints beforehand. However, this approach provides no insight into the constraints themselves although this is desired for safety-critical applications such as autonomous driving. In such settings, it is important to verify what is learned from the given demonstrations. In this work, we propose a novel approach for learning logic rules that represent the environmental constraints given demonstrations of agents that comply with them, thus providing an interpretable representation of the environmental constraints.

Details

AAMAS Conference 2021 Conference Paper

No More Hand-Tuning Rewards: Masked Constrained Policy Optimization for Safe Reinforcement Learning

Stef Van Havermaet
Yara Khaluf
Pieter Simoens

In safe Reinforcement Learning (RL), the agent attempts to find policies which maximize the expectation of accumulated rewards and guarantee its safety to remain above a given threshold. Hence, it is straightforward to formalize safe RL problems by both a reward function and a safety constraint. We define safety as the probability of survival in environments where taking risky actions could lead to early termination of the task. Although the optimization problem is already constrained by a safety threshold, reward signals related to unsafe terminal states influence the original maximization objective of the task. Selecting the appropriate value of these signals is often a time consuming and challenging reward engineering task, which requires expert knowledge of the domain. This paper presents a safe RL algorithm, called Masked Constrained Policy Optimization (MCPO), in which the learning process is constrained by safety and excludes the unsafe reward signals. We develop MCPO as an extension of gradient-based policy search methods, in which the updates of the policy and the expected reward models are masked. Our method benefits from having a high probability of satisfying the given constraints for every policy in the learning process. We validate the proposed algorithm in two continuous tasks. Our findings prove the proposed algorithm is able to neglect unsafe reward signals, and thereby resolving the desired safety-performance trade-off without having the need for hand-tuning rewards.

PDF

AAAI Conference 2020 Short Paper

Action Graphs for Goal Recognition Problems with Inaccurate Initial States (Student Abstract)

Helen Harman
Pieter Simoens

Goal recognisers attempt to infer an agent’s intentions from a sequence of observations. Approaches that adapt classical planning techniques to goal recognition have previously been proposed but, generally, they assume the initial world state is accurately deﬁned. In this paper, a state is inaccurate if any ﬂuent’s value is unknown or incorrect. To cope with this, a cyclic Action Graph, which models the order constraints between actions, is traversed to label each node with their distance from each hypothesis goal. These distances are used to calculate the posterior goal probabilities. Our experimental results, for 15 different domains, demonstrate that our approach is unaffected by an inaccurately deﬁned initial state.

PDF Details

ECAI Conference 2020 Conference Paper

Enhanced Foraging in Robot Swarms Using Collective Lévy Walks

Johannes Nauta
Stef Van Havermaet
Pieter Simoens
Yara Khaluf

A key aspect of foraging in robot swarms is optimizing the search efficiency when both the environment and target density are unknown. Hence, designing optimal exploration strategies is desirable. This paper proposes a novel approach that extends the individual Lévy walk to a collective one. To achieve this, we adjust the individual motion through applying an artificial potential field method originating from local communication. We demonstrate the effectiveness of the enhanced foraging by confirming that the collective trajectory follows a heavy-tailed distribution over a wide range of swarm sizes. Additionally, we study target search efficiency of the proposed algorithm in comparison with the individual Lévy walk for two different types of target distributions: homogeneous and heterogeneous. Our results highlight the advantages of the proposed approach for both target distributions, while increasing the scalability to large swarm sizes. Finally, we further extend the individual exploration algorithm by adapting the Lévy walk parameter α, altering the motion pattern based on a local estimation of the target density. This adaptive behavior is particularly useful when targets are distributed in patches.

Details

IROS Conference 2019 Conference Paper

Learning to Grasp Arbitrary Household Objects from a Single Demonstration

Elias De Coninck
Tim Verbelen
Pieter Van Molle
Pieter Simoens
Bart Dhoedt

Upon the advent of Industry 4. 0, collaborative robotics and intelligent automation gain more and more traction for enterprises to improve their production processes. In order to adapt to this trend, new programming, learning and collaborative techniques are investigated. Program-bydemonstration is one of the techniques that aim to reduce the burden of manually programming collaborative robots. However, this is often limited to teaching to grasp at a certain position, rather than grasping a certain object. In this paper, we propose a method that learns to grasp an arbitrary object from visual input. While other learning-based approaches for robotic grasping require collecting a large dataset, manually or automatically labeled in a real or simulated world, our approach requires a single demonstration. We present results on grasping various objects with the Franka Panda collaborative robot after capturing a single image from a wrist mounted RGB camera. From this image we learn a robot controller with a convolutional neural network to adapt to changes in the object’s position and rotation with less than 5 minutes of training time on a NVIDIA Titan X GPU, achieving over 90% grasp success rate.

Details

IROS Conference 2017 Conference Paper

Sensor fusion for robot control through deep reinforcement learning

Steven Bohez
Tim Verbelen
Elias De Coninck
Bert Vankeirsbilck
Pieter Simoens
Bart Dhoedt

Deep reinforcement learning is becoming increasingly popular for robot control algorithms, with the aim for a robot to self-learn useful feature representations from unstructured sensory input leading to the optimal actuation policy. In addition to sensors mounted on the robot, sensors might also be deployed in the environment, although these might need to be accessed via an unreliable wireless connection. In this paper, we demonstrate deep neural network architectures that are able to fuse information generated by multiple sensors and are robust to sensor failures at runtime. We evaluate our method on a search and pick task for a robot both in simulation and the real world.

Details