Author name cluster

Matteo Leonetti

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

2 author rows

AAMAS Conference 2026 Conference Paper

Towards Generalisable Imitation Learning Through Conditioned Transition Estimation and Online Behaviour Alignment

Nathan Gavenski
Matteo Leonetti
Odinaldo Rodrigues

State-of-the-art imitation learning from observation methods (ILfO) have recently made significant progress, but they still have some limitations: they need action-based supervised optimisation, assume that states have a single optimal action, and tend to apply teacher actions without full consideration of the actual environment state. While the truth may be out there in observed trajectories, existing methods struggle to extract it without supervision. In this work, we propose Unsupervised Imitation Learning from Observation (UfO) that addresses all of these limitations. UfO learns a policy through a two-stage process, in which the agent first obtains an approximation of the teacher’s true actions in the observed state transitions, and then refines the learned policy further by adjusting agent trajectories to closely align them with the teacher’s. Experiments we conducted in five widely used environments show that UfO not only outperforms the teacher and all other ILfO methods but also displays the smallest standard deviation. This reduction in standard deviation indicates better generalisation in unseen scenarios.

PDF

AAMAS Conference 2026 Conference Paper

Translating Latent State World Model Representations into Natural Language

Matthew Barker
Matteo Leonetti

Recent successes in model-based reinforcement learning have stemmed from models that learn a latent representation of the world. However, these latent representations are unintelligible, meaning that we cannot interpret the agent’s internal world representation, noranyplansmadeinthislatentspace. InthisworkwepresentSomniloquy, an algorithm that learns to translate latent state plans into a natural language description that captures what the plan means in terms of the agent’s expected interaction with the world. We demonstrate that latent plan translations can be learned in tandem with the latent representations, whilst giving the latent representations an additional learning signal to be translatable, and that Somniloquy enables a deep model-based reinforcement learning agent to verbalise its latent plan in natural language prior to acting. In addition to interpretability, we show that Somniloquy enables defining desired behaviour in language by rewarding latent states whose translation matches the requested behaviour. Importantly, this requires no reward signal at any point from the environment. Wedemonstrateexperimentallythat, indeterministicenvironments, Somniloquy’s plan translation accurately describes the plan’s execution, and that in the stochastic setting the translations of multiple latent plan rollouts can approximate the true environment dynamics. Finally, we demonstrate that our translation reward function approach successfully trains policies to achieve goals specified in naturallanguage, andachieveson-parperformancewithtrainingan agent that has access to each natural language goal’s unobservable extrinsic reward function.

PDF

EWRL Workshop 2025 Workshop Paper

Exploiting Model Errors for Exploration in Model-Based Reinforcement Learning

Jared Swift
Matteo Leonetti

We address the problem of exploration in model-based reinforcement learning (MBRL). We present Model-Corrective eXploration (MCX) a novel approach to exploration in MBRL that is both agnostic to the model representation and scalable to complex environments. MCX learns to generalise model prediction errors in order to make hypotheses about how the model might else be wrong, and uses such hypotheses for performing planning to facilitate exploration. We demonstrate the efficacy of our method in visual control tasks with the state-of-the-art MBRL algorithm, DreamerV3.

PDF

IROS Conference 2024 Conference Paper

Learning Social Cost Functions for Human-Aware Path Planning

Andrea Eirale
Matteo Leonetti
Marcello Chiaberge

Achieving social acceptance is one of the main goals of Social Robotic Navigation. Despite this topic has received increasing interest in recent years, most of the research has focused on driving the robotic agent along obstacle-free trajectories, planning around estimates of future human motion to respect personal distances and optimize navigation. However, social interactions in everyday life are also dictated by norms that do not strictly depend on movement, such as when standing at the end of a queue rather than cutting it. In this paper, we propose a novel method to recognize common social scenarios and modify a traditional planner’s cost function to adapt to them. This solution enables the robot to carry out different social navigation behaviors that would not arise otherwise, maintaining the robustness of traditional navigation. Our approach allows the robot to learn different social norms with a single learned model, rather than having different modules for each task. As a proof of concept, we consider the tasks of queuing and respect interaction spaces of groups of people talking to one another, but the method can be extended to other human activities that do not involve motion.

Details

IROS Conference 2024 Conference Paper

Probabilistic Inference of Human Capabilities from Passive Observations

Peter Tisnikar
Gerard Canal
Matteo Leonetti

Modern robots need to adapt to diverse human partners with whom they collaborate. To this end, learning a representation of human capabilities enables the robot to personalize their behaviour to their collaborators across multiple tasks. We propose CApability Modeling from Observations (CAMO), a model-based estimation algorithm, in which human capabilities that parameterize a given model are inferred from observations of the human behaviour on known collaborative tasks. We apply the method to joint limit learning in order to predict future trajectories of a 7-DOF manipulator arm. Furthermore, we show that CAMO can be used as a sub-task assignment routine in a simulated human–robot collaboration scenario, allowing the robot to adapt its task allocation to perform tasks that the person is not able to do.

Details

ICRA Conference 2023 Conference Paper

Goal-Conditioned Action Space Reduction for Deformable Object Manipulation

Shengyin Wang
Rafael Papallas
Matteo Leonetti
Mehmet Remzi Dogar

Planning for deformable object manipulation has been a challenge for a long time in robotics due to its high computational cost. In this work, we propose to reduce this cost by reducing the number of pick points on a deformable object in the action space. We do this by identifying a small number of key particles that are sufficient as pick points to reach a given goal state. We find these key particles through a geometric model simplification process, which finds the minimal geometric model that still enables a good approximation of the original model at the goal state. We present an implementation of this general approach for 1-D linear deformable objects (e. g. , ropes) that uses a piece-wise line fitted model, and for 2-D flat deformable objects (e. g. , cloth) that uses a mesh simplified model. We conducted simulation experiments on ropes and cloths, which demonstrate the effectiveness of the proposed method. Finally, the planned paths are executed in a real-world setting for two cloth folding tasks.

Details

IROS Conference 2023 Conference Paper

Online Human Capability Estimation Through Reinforcement Learning and Interaction

Chengke Sun
Anthony G. Cohn 0001
Matteo Leonetti

Service robots are expected to assist users in a constantly growing range of environments and tasks. People may be unique in many ways, and online adaptation of robots is central to personalized assistance. We focus on collaborative tasks in which the human collaborator may not be fully ablebodied, with the aim for the robot to automatically determine the best level of support. We propose a methodology for online adaptation based on Reinforcement Learning and Bayesian inference. As the Reinforcement Learning process continuously adjusts the robot's behavior, the actions that become part of the improved policy are used by the Bayesian inference module as local evidence of human capability, which can be generalized across the state space. The estimated capabilities are then used as pre-conditions to collaborative actions, so that the robot can quickly disable actions that the person seems unable to perform. We demonstrate and validate our approach on two simulated tasks and one real-world collaborative task across a range of motion and sensing capabilities.

Details

ICAPS Conference 2021 Conference Paper

Meta Reinforcement Learning for Heuristic Planing

Ricardo Luna Gutierrez 0001
Matteo Leonetti

Heuristic planning has a central role in classical planning applications and competitions. Thanks to this success, there has been an increasing interest in using Deep Learning to create high-quality heuristics in a supervised fashion, learning from optimal solutions of previously solved planning problems. Meta-Reinforcement learning is a fast growing research area concerned with learning, from many tasks, behaviours that can quickly generalize to new tasks from the same distribution of the training ones. We make a connection between meta-reinforcement learning and heuristic planning, showing that heuristic functions meta-learned from planning problems, in a given domain, can outperform both popular domain-independent heuristics, and heuristics learned by supervised learning. Furthermore, while most supervised learning algorithms rely on ad-hoc encodings of the state representation, our method uses as input a general PDDL 3. 1 description. We evaluated our heuristic with an A* planner on six domains from the International Planning Competition and the FF Domain Collection, showing that the meta-learned heuristic leads to the expansion, on average, of fewer states than three popular heuristics used by the FastDownward planner, and a supervised-learned heuristic.

Details

IROS Conference 2021 Conference Paper

Occlusion-Aware Search for Object Retrieval in Clutter

Wissam Bejjani
Wisdom C. Agboh
Mehmet Remzi Dogar
Matteo Leonetti

We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop. The hybrid planner explores likely locations of the occluded target object as predicted by a learned distribution from the observation stream. The search is guided by a heuristic trained with reinforcement learning to act on observations with occlusions. We evaluate our approach in different simulation and real-world settings (video available on https://youtu.be/dY7YQ3LUVQg).The results validate that our approach can search and retrieve a target object in near real time in the real world while only being trained in simulation.

Details

JMLR Journal 2020 Journal Article

Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey

Sanmit Narvekar
Bei Peng
Matteo Leonetti
Jivko Sinapov
Matthew E. Taylor
Peter Stone

Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. Despite many advances over the past three decades, learning in many domains still requires a large amount of interaction with the environment, which can be prohibitively expensive in realistic scenarios. To address this problem, transfer learning has been applied to reinforcement learning such that experience gained in one task can be leveraged when starting to learn the next, harder task. More recently, several lines of research have explored how tasks, or data samples themselves, can be sequenced into a curriculum for the purpose of learning a problem that may otherwise be too difficult to learn from scratch. In this article, we present a framework for curriculum learning (CL) in reinforcement learning, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals. Finally, we use our framework to find open problems and suggest directions for future RL curriculum learning research. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

PDF Details

ICRA Conference 2020 Conference Paper

Human-like Planning for Reaching in Cluttered Environments

Mohamed Hasan
Matthew Warburton
Wisdom C. Agboh
Mehmet Remzi Dogar
Matteo Leonetti
He Wang 0002
Faisal Mushtaq
Mark Mon-Williams

Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling of configuration space- which becomes excessively high-dimensional with large number of objects. Consequently, most planners often fail to efficiently find object manipulation plans in such environments. We addressed this problem by identifying high-level manipulation plans in humans, and transferring these skills to robot planners. We used virtual reality to capture human participants reaching for a target object on a tabletop cluttered with obstacles. From this, we devised a qualitative representation of the task space to abstract the decision making, irrespective of the number of obstacles. Based on this representation, human demonstrations were segmented and used to train decision classifiers. Using these classifiers, our planner produced a list of waypoints in task space. These waypoints provided a high-level plan, which could be transferred to an arbitrary robot model and used to initialise a local trajectory optimiser. We evaluated this approach through testing on unseen human VR data, a physics-based robot simulation, and a real robot (dataset and code are publicly available 1 ). We found that the human-like planner outperformed a state-of-the-art standard trajectory optimisation algorithm, and was able to generate effective strategies for rapid planning- irrespective of the number of obstacles in the environment.

Details

NeurIPS Conference 2020 Conference Paper

Information-theoretic Task Selection for Meta-Reinforcement Learning

Ricardo Luna Gutierrez
Matteo Leonetti

In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of target tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better performance in the target tasks), if the training tasks are appropriately selected. We propose a task selection algorithm based on information theory, which optimizes the set of tasks used for training in meta-RL, irrespectively of how they are generated. The algorithm establishes which training tasks are both sufficiently relevant for the target tasks, and different enough from one another. We reproduce different meta-RL experiments from the literature and show that our task selection algorithm improves the final performance in all of them.

PDF Details

IJCAI Conference 2019 Conference Paper

Curriculum Learning for Cumulative Return Maximization

Francesco Foglino
Christiano Coletto Christakou
Ricardo Luna Gutierrez
Matteo Leonetti

Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape exploration through transfer. We propose a task sequencing algorithm maximizing the cumulative return, that is, the return obtained by the agent across all the learning episodes. By maximizing the cumulative return, the agent not only aims at achieving high rewards as fast as possible, but also at doing so while limiting suboptimal actions. We experimentally compare our task sequencing algorithm to several popular metaheuristic algorithms for combinatorial optimization, and show that it achieves significantly better performance on the problem of cumulative return maximization. Furthermore, we validate our algorithm on a critical task, optimizing a home controller for a micro energy grid.

PDF Details

IROS Conference 2019 Conference Paper

Learning Physics-Based Manipulation in Clutter: Combining Image-Based Generalization and Look-Ahead Planning

Wissam Bejjani
Mehmet Remzi Dogar
Matteo Leonetti

Physics-based manipulation in clutter involves complex interaction between multiple objects. In this paper, we consider the problem of learning, from interaction in a physics simulator, manipulation skills to solve this multi-step sequential decision making problem in the real world. Our approach has two key properties: (i) the ability to generalize and transfer manipulation skills (over the type, shape, and number of objects in the scene) using an abstract image-based representation that enables a neural network to learn useful features; and (ii) the ability to perform look-ahead planning in the image space using a physics simulator, which is essential for such multi-step problems. We show, in sets of simulated and real-world experiments (video available on https://youtu.be/EmkUQfyvwkY), that by learning to evaluate actions in an abstract image-based representation of the real world, the robot can generalize and adapt to the object shapes in challenging real-world environments.

Details

AAAI Conference 2017 Conference Paper

Automatic Curriculum Graph Generation for Reinforcement Learning Agents

Maxwell Svetlik
Matteo Leonetti
Jivko Sinapov
Rishi Shah
Nick Walker
Peter Stone

In recent years, research has shown that transfer learning methods can be leveraged to construct curricula that sequence a series of simpler tasks such that performance on a ﬁnal target task is improved. A major limitation of existing approaches is that such curricula are handcrafted by humans that are typically domain experts. To address this limitation, we introduce a method to generate a curriculum based on task descriptors and a novel metric of transfer potential. Our method automatically generates a curriculum as a directed acyclic graph (as opposed to a linear sequence as done in existing work). Experiments in both discrete and continuous domains show that our method produces curricula that improve the agent’s learning performance when compared to the baseline condition of learning on the target task from scratch.

PDF Details

AIJ Journal 2016 Journal Article

A synthesis of automated planning and reinforcement learning for efficient, robust decision-making

Matteo Leonetti
Luca Iocchi
Peter Stone

Automated planning and reinforcement learning are characterized by complementary views on decision making: the former relies on previous knowledge and computation, while the latter on interaction with the world, and experience. Planning allows robots to carry out different tasks in the same domain, without the need to acquire knowledge about each one of them, but relies strongly on the accuracy of the model. Reinforcement learning, on the other hand, does not require previous knowledge, and allows robots to robustly adapt to the environment, but often necessitates an infeasible amount of experience. We present Domain Approximation for Reinforcement LearnING (DARLING), a method that takes advantage of planning to constrain the behavior of the agent to reasonable choices, and of reinforcement learning to adapt to the environment, and increase the reliability of the decision making process. We demonstrate the effectiveness of the proposed method on a service robot, carrying out a variety of tasks in an office building. We find that when the robot makes decisions by planning alone on a given model it often fails, and when it makes decisions by reinforcement learning alone it often cannot complete its tasks in a reasonable amount of time. When employing DARLING, even when seeded with the same model that was used for planning alone, however, the robot can quickly learn a behavior to carry out all the tasks, improves over time, and adapts to the environment as it changes.

Details DOI

AAMAS Conference 2016 Conference Paper

Source Task Creation for Curriculum Learning

Sanmit Narvekar
Jivko Sinapov
Matteo Leonetti
Peter Stone

Transfer learning in reinforcement learning has been an active area of research over the past decade. In transfer learning, training on a source task is leveraged to speed up or otherwise improve learning on a target task. This paper presents the more ambitious problem of curriculum learning in reinforcement learning, in which the goal is to design a sequence of source tasks for an agent to train on, such that final performance or learning speed is improved. We take the position that each stage of such a curriculum should be tailored to the current ability of the agent in order to promote learning new behaviors. Thus, as a first step towards creating a curriculum, the trainer must be able to create novel, agent-specific source tasks. We explore how such a space of useful tasks can be created using a parameterized model of the domain and observed trajectories on the target task. We experimentally show that these methods can be used to form components of a curriculum and that such a curriculum can be used successfully for transfer learning in 2 challenging multiagent reinforcement learning domains.

PDF

ICRA Conference 2014 Conference Paper

Online discovery of AUV control policies to overcome thruster failures

Seyed Reza Ahmadzadeh
Matteo Leonetti
Arnau Carrera
Marc Carreras
Petar Kormushev
Darwin G. Caldwell

We investigate methods to improve fault-tolerance of Autonomous Underwater Vehicles (AUVs) to increase their reliability and persistent autonomy. We propose a learning-based approach that is able to discover new control policies to overcome thruster failures as they happen. The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the AUV. The model is adapted to a new condition when a fault is detected and isolated. Since the approach generates an optimal trajectory, the learned fault-tolerant policy is able to navigate the AUV towards a specified target with minimum cost. Finally, the learned policy is executed on the real robot in a closed-loop using the state feedback of the AUV. Unlike most existing methods which rely on the redundancy of thrusters, our approach is also applicable when the AUV becomes under-actuated in the presence of a fault. To validate the feasibility and efficiency of the presented approach, we evaluate it with three learning algorithms and three policy representations with increasing complexity. The proposed method is tested on a real AUV, Girona500.

Details

ICAPS Conference 2014 Conference Paper

Planning in Action Language BC while Learning Action Costs for Mobile Robots

Piyush Khandelwal
Fangkai Yang
Matteo Leonetti
Vladimir Lifschitz
Peter Stone 0001

The action language BC provides an elegant way of formalizing dynamic domains which involve indirect effects of actions and recursively defined fluents. In complex robot task planning domains, it may be necessary for robots to plan with incomplete information, and reason about indirect or recursive action effects. In this paper, we demonstrate how BC can be used for robot task planning to solve these issues. Additionally, action costs are incorporated with planning to produce optimal plans, and we estimate these costs from experience making planning adaptive. This paper presents the first application of BC on a real robot in a realistic domain, which involves human-robot interaction for knowledge acquisition, optimal plan generation to minimize navigation time, and learning for adaptive planning.

Details

IROS Conference 2013 Conference Paper

On-line identification of autonomous underwater vehicles through global derivative-free optimization

George C. Karras
Charalampos P. Bechlioulis
Matteo Leonetti
Narcís Palomeras
Petar Kormushev
Kostas J. Kyriakopoulos
Darwin G. Caldwell

We describe the design and implementation of an on-line identification scheme for Autonomous Underwater Vehicles (AUVs). The proposed method estimates the dynamic parameters of the vehicle based on a global derivative-free optimization algorithm. It is not sensitive to initial conditions, unlike other on-line identification schemes, and does not depend on the differentiability of the model with respect to the parameters. The identification scheme consists of three distinct modules: a) System Excitation, b) Metric Calculator and c) Optimization Algorithm. The System Excitation module sends excitation inputs to the vehicle. The Optimization Algorithm module calculates a candidate parameter vector, which is fed to the Metric Calculator module. The Metric Calculator module evaluates the candidate parameter vector, using a metric based on the residual of the actual and the predicted commands. The predicted commands are calculated utilizing the candidate parameter vector and the vehicle state vector, which is available via a complete navigation module. Then, the metric is directly fed back to the Optimization Algorithm module, and it is used to correct the estimated parameter vector. The procedure continues iteratively until the convergence properties are met. The proposed method is generic, demonstrates quick convergence and does not require a linear formulation of the model with respect to the parameter vector. The applicability and performance of the proposed algorithm is experimentally verified using the AUV Girona 500.

Details

AAMAS Conference 2012 Conference Paper

Induction and Learning of Finite-State controllers from Simulation

Matteo Leonetti
Luca Iocchi
Subramanian Ramamoorthy

We propose a method to generate agent controllers, represented as state machines, to act in partially observable environments. Such controllers are used to constrain the search space, applying techniques from Hierarchical Reinforcement Learning. We define a multi-step process, in which a simulator is employed to generate possible traces of execution. Those traces are then utilized to induce a non-deterministic state machine, that represents all reasonable behaviors, given the approximate models and planners used in simulation. The state machine will have multiple possible choices in some of its states. Those states are choice points, and we defer the learning of those choices to the deployment of the agent in the actual environment. The controller obtained can therefore adapt to the actual environment, limiting the search space in a sensible way.

PDF

IROS Conference 2010 Conference Paper

A probabilistic action duration model for plan selection and monitoring

Vittorio A. Ziparo
Luca Iocchi
Matteo Leonetti
Daniele Nardi

The execution of tasks for a robotic agent embedded in a dynamic environment brings about several challenges, due to unpredictable (or unobservable) events, and to inaccurate perception. Moreover, the agent can perform multiple tasks and each task can be achieved by applying different plans, therefore the decision about which strategy is the most convenient, given the current situation of the world, is important for assessing an intelligent overall behavior of the agent. This paper tackles the problem of on-line execution monitoring in a novel way with respect to previous work, since: (1) it considers uncertainty in the duration of actions with a probabilistic model of action duration; (2) it evaluates the cost of each possible plan at run-time in terms of probability of successful termination within a desired expected time. The approach has been evaluated both in a robotic soccer and a surveillance scenario.

Details

AAMAS Conference 2010 Conference Paper

Improving the Performance of Complex Agent Plans Through Reinforcement Learning

Matteo Leonetti
Luca Iocchi

Agent programming in complex, partially observable andstochastic domains usually requires a great deal of understanding of both the domain and the task, in order to provide the agent with the knowledge necessary to act effectively. While symbolic methods allow the designer to specify declarative knowledge about the domain, the resultingplan can be brittle since it is difficult to supply a symbolicmodel that is accurate enough to foresee all possible eventsin complex environments, especially in the case of partialobservability. Reinforcement Learning (RL) techniques, onthe other hand, can learn a policy and make use of a learnedmodel, but it is difficult to reduce and shape the scope of thelearning algorithm by exploiting a priori information. Wepropose a methodology for writing complex agent programsthat can be effectively improved through experience. Weshow how to derive a stochastic process from a partial specification of the plan, so that the latter's perfomance can beimproved solving a RL problem much smaller than classicalRL formulations. Finally, we demonstrate our approach inthe context of Keepaway Soccer, a common RL benchmarkbased on a RoboCup Soccer 2D simulator.

PDF

AAMAS Conference 2010 Conference Paper

On-line robot execution monitoring using probabilistic action duration

Vittorio Ziparo
Luca Iocchi
Matteo Leonetti
Daniele Nardi

The execution of tasks for a mobile robot embedded in a dynamicenvironment brings about several challenges, due to the dynamicchanges of the environment and the inaccurate perception of therobot. This paper tackles the problem of on-line execution monitoring when the agent has different tasks and several plan to accomplish them, as in the BDI framework. Our method considersuncertainty in the duration of actions with a probabilistic modelof action duration, and evaluates the cost of each possible plan atrun-time in terms of probability of successful termination within adesired expected time.

PDF