Paulo E. Santos

ICRA Conference 2025 Conference Paper

Control Reallocation Using Deep Reinforcement Learning for Actuator Fault Recovery of an Autonomous Underwater Vehicle

• Katell Lagattu
• Eva Artusi
• Paulo E. Santos
• Karl Sammut
• Gilles Le Chenadec
• Benoit Clement

Actuator faults in dynamic systems pose significant challenges, particularly for robotic systems operating in hostile environments such as Autonomous Underwater Vehicles (AUVs), risking loss of stability and performance degradation. Fault Tolerant Control (FTC) strategies, including Control Reallocation (CR), have been developed to mitigate such risks. However, these strategies extensively depend on explicit fault diagnosis, which may present challenges regarding computational demands and efficiency, particularly when dealing with unknown faults. This paper presents a novel method that performs CR with Deep Reinforcement Learning (DRL) for actuator fault recovery without explicit fault diagnosis. The approach is implemented on a BlueROV2 underwater vehicle and demonstrates improved performance for fault recovery compared to a standard Proportional-Integral-Derivative (PID) controller and a variable gain PID controller, both in simulation and in real-world conditions. The DRL-based CR method demonstrates generalisability by successfully handling faults not encountered during training, highlighting its adaptability to unforeseen circumstances.

KR Conference 2020 Conference Paper

Spatial Reasoning about String Loops and Holes in Temporal ASP

• Pedro Cabalar
• Paulo E. Santos

This paper introduces a new formalism for the automated solution of spatial scenarios involving strings and holed objects. In particular, we revisit a previous formalisation that allows string loops to be treated as holes, but make a substantial modification by removing a previous limitation that prevented a string to cross its own loops. The formalisation introduced in the present paper relies on string segments as basic entities and achieves a greater degree of elaboration tolerance by using inertia to describe those parts of the physical scenario that are unaffected by a given action. As a representation language, we have used Temporal Answer Set Programming since it provides a simple and natural way to deal with time and inertia while, at the same time, it is accompanied by the automated tool 'telingo' that allows a systematic testing of the effects of any sequence of actions. As an illustrative example, we have studied the African Ring puzzle, a problem involving loops crossed by a unique string, and provided the first formalisation of its solution, to the best of our knowledge.

AIJ Journal 2016 Journal Article

A qualitative spatial representation of string loops as holes

• Pedro Cabalar
• Paulo E. Santos

This research note contains an extension of a previous work by Cabalar and Santos (2011) that formalised several spatial puzzles formed by strings and holes. That approach explicitly ignored some configurations and actions that were irrelevant for the studied puzzles but are physically possible and may become crucial for other spatial reasoning problems. In particular, the previous work did not consider the formation of string loops or the situations where a holed object is partially crossed by another holed object. In this paper, we remove these limitations by treating string loops as dynamic holes that can be created or destroyed by a pair of elementary actions, respectively picking or pulling from strings. We explain how string loops can be recognised in a data structure representing the domain states and define a notation to represent crossings through string loops. The resulting formalism is dual in the sense that it also allows understanding any hole as a kind of (sometimes rigid) closed string loop.

AIJ Journal 2015 Journal Article

Transferring knowledge as heuristics in reinforcement learning: A case-based approach

• Reinaldo A.C. Bianchi
• Luiz A. Celiberto
• Paulo E. Santos
• Jackson P. Matsuura
• Ramon Lopez de Mantaras

The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms.

ICAPS Conference 2013 Conference Paper

An Investigation of Actions, Change and Space

• Paulo E. Santos
• Pedro Cabalar

This work investigates the spatial knowledge and automated solution of a domain composed of non-trivial objects such as strings and holed objects.

AIJ Journal 2011 Journal Article

Formalising the Fisherman's Folly puzzle

• Pedro Cabalar
• Paulo E. Santos

This paper investigates the challenging problem of encoding the common sense knowledge involved in the manipulation of spatial objects from a reasoning about actions and change perspective. In particular, we propose a formal solution to a puzzle composed of non-trivial objects (such as holes and strings) assuming a version of the Situation Calculus written over first-order Equilibrium Logic, whose models generalise the stable model semantics.

AIIM Journal 2010 Journal Article

Exploring the knowledge contained in neuroimages: Statistical discriminant analysis and automatic segmentation of the most significant changes

• Paulo E. Santos
• Carlos E. Thomaz
• Danilo dos Santos
• Rodolpho Freire
• João R. Sato
• Mario Louzã
• Paulo Sallet
• Geraldo Busatto

Objective The aim of this article is to propose an integrated framework for extracting and describing patterns of disorders from medical images using a combination of linear discriminant analysis and active contour models. Methods A multivariate statistical methodology was first used to identify the most discriminating hyperplane separating two groups of images (from healthy controls and patients with schizophrenia) contained in the input data. After this, the present work makes explicit the differences found by the multivariate statistical method by subtracting the discriminant models of controls and patients, weighted by the pooled variance between the two groups. A variational level-set technique was used to segment clusters of these differences. We obtain a label of each anatomical change using the Talairach atlas. Results In this work all the data was analysed simultaneously rather than assuming a priori regions of interest. As a consequence of this, by using active contour models, we were able to obtain regions of interest that were emergent from the data. The results were evaluated using, as gold standard, well-known facts about the neuroanatomical changes related to schizophrenia. Most of the items in the gold standard was covered in our result set. Conclusions We argue that such investigation provides a suitable framework for characterising the high complexity of magnetic resonance images in schizophrenia as the results obtained indicate a high sensitivity rate with respect to the gold standard.

ECAI Conference 2010 Conference Paper

Knowledge-based adaptive thresholding from shadows

• Paulo E. Santos
• Hannah M. Dee
• Valquiria Fenelon

This paper presents results of a mobile robot qualitative self-localisation experiment using information from cast shadows. We present results of self-localisation using two methods for obtaining the threshold automatically: in one method the images are segmented according to their grey-scale histograms, in the other the threshold is set according to a prediction about the robot's location, given a shadow-based map defined upon a qualitative spatial reasoning theory. To the best of our knowledge this is the first work that uses qualitative spatial representations both to perform egolocation and to calibrate a robot's interpretation of its perceptual input.

ICRA Conference 2009 Conference Paper

Qualitative robot localisation using information from cast shadows

• Paulo E. Santos
• Hannah M. Dee
• Valquiria Fenelon

Recently, cognitive psychologists and others have turned their attention to the formerly neglected study of shadows, and the information they purvey. These studies show that the human perceptual system values information from shadows very highly, particularly in the perception of depth, even to the detriment of other cues. However with a few notable exceptions, computer vision systems have treated shadows not as signal but as noise. This paper makes a step towards redressing this imbalance by considering the formal representation of shadows. We take one particular aspect of reasoning about shadows, developing the idea that shadows carry information about a fragment of the viewpoint of the light source. We start from the observation that the region on which the shadow is cast is occluded by the caster with respect to the light source and build a qualitative theory about shadows using a region-based spatial formalism about occlusion. Using this spatial formalism and a machine vision system we are able to draw simple conclusions about domain objects and egolocation for a mobile robot.

ECAI Conference 2008 Conference Paper

Reasoning about Dynamic Depth Profiles

• Mikhail Soutchanski
• Paulo E. Santos

Reasoning about perception of depth and about spatial relations between moving physical objects is a challenging problem. We investigate the representation of depth and motion by means of depth profiles whereby each object in the world is represented as a single peak. We propose a logical theory, formulated in the situation calculus (SC), that is used for reasoning about object motion (including motion of the observer). The theory proposed here is comprehensive enough to accommodate reasoning about both sensor data and actions in the world. We show that reasoning about depth profiles is sound and complete with respect to actual motion in the world. This shows that in the conceptual neighbourhood diagram (CND) of all possible depth perceptions, the transitions between perceptions are logical consequences of the proposed theory of depth and motion.

AIJ Journal 2005 Journal Article

Protocols from perceptual observations

• Chris J. Needham
• Paulo E. Santos
• Derek R. Magee
• Vincent Devin
• David C. Hogg
• Anthony G. Cohn

This paper presents a cognitive vision system capable of autonomously learning protocols from perceptual observations of dynamic scenes. The work is motivated by the aim of creating a synthetic agent that can observe a scene containing interactions between unknown objects and agents, and learn models of these sufficient to act in accordance with the implicit protocols present in the scene. Discrete concepts (utterances and object properties), and temporal protocols involving these concepts, are learned in an unsupervised manner from continuous sensor input alone. Crucial to this learning process are methods for spatio-temporal attention applied to the audio and visual sensor data. These identify subsets of the sensor data relating to discrete concepts. Clustering within continuous feature spaces is used to learn object property and utterance models from processed sensor data, forming a symbolic description. The progol Inductive Logic Programming system is subsequently used to learn symbolic models of the temporal protocols presented in the presence of noise and over-representation in the symbolic data input to it. The models learned are used to drive a synthetic agent that can interact with the world in a semi-natural way. The system has been evaluated in the domain of table-top game playing and has been shown to be successful at learning protocol behaviours in such real-world audio-visual environments.

ECAI Conference 2004 Conference Paper

Combining Multiple Answers for Learning Mathematical Structures from Visual Observation

• Paulo E. Santos
• Derek R. Magee
• Anthony G. Cohn 0001
• David Crossland Hogg

ECAI Conference 2002 Conference Paper

Hypothesising Object Relations from Image Transitions

• Paulo E. Santos
• Murray Shanahan