Author name cluster

Marc Hanheide

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

23 papers

2 author rows

ICRA Conference 2021 Conference Paper

Efficient and Robust Orientation Estimation of Strawberries for Fruit Picking Applications

Nikolaus Wagner
Raymond Kirk
Marc Hanheide
Grzegorz Cielniak

Recent developments in agriculture have high-lighted the potential of as well as the need for the use of robotics. Various processes in this field can benefit from the proper use of state of the art technology [1], in terms of efficiency as well as quality. One of these areas is the harvesting of ripe fruit. In order to be able to automate this process, a robotic harvester needs to be aware of the full poses of the crop/fruit to be collected in order to perform proper path- and collision-planning. The current state of the art mainly considers problems of detection and segmentation of fruit with localisation limited to the 3D position only. The reliable and real-time estimation of the respective orientations remains a mostly unaddressed problem. In this paper, we present a compact and efficient network architecture for estimating the orientation of soft fruit such as strawberries from colour and, optionally, depth images. The proposed system can be automatically trained in a realistic simulation environment. We evaluate the system’s performance on simulated datasets and validate its operation on publicly available images of strawberries to demonstrate its practical use. Depending on the amount of training data used, coverage of state space, as well as the availability of RGB-D or RGB data only, mean errors of as low as 11° could be achieved.

ICRA Conference 2020 Conference Paper

Context Dependant Iterative Parameter Optimisation for Robust Robot Navigation

Adam Binch
Gautham P. Das
Jaime Pulido Fentanes
Marc Hanheide

Progress in autonomous mobile robotics has seen significant advances in the development of many algorithms for motion control and path planning. However, robust performance from these algorithms can often only be expected if the parameters controlling them are tuned specifically for the respective robot model, and optimised for specific scenarios in the environment the robot is working in. Such parameter tuning can, depending on the underlying algorithm, amount to a substantial combinatorial challenge, often rendering extensive manual tuning of these parameters intractable. In this paper, we present a framework that permits the use of different navigation actions and/or parameters depending on the spatial context of the navigation task. We consider the respective navigation algorithms themselves mostly as a "black box", and find suitable parameters by means of an iterative optimisation, improving for performance metrics in simulated environments. We present a genetic algorithm incorporated into the framework, and empirically show that the resulting parameter sets lead to substantial performance improvements in both simulated and real-world environments in the domain of agricultural robots.

ICRA Conference 2020 Conference Paper

Enhancing Grasp Pose Computation in Gripper Workspace Spheres

Mohamed Sorour
Khaled Elgeneidy
Marc Hanheide
M. Abdalmjed
A. Srinivasan
Gerhard Neumann

In this paper, enhancement to the novel grasp planning algorithm based on gripper workspace spheres is presented. Our development requires a registered point cloud of the target from different views, assuming no prior knowledge of the object, nor any of its properties. This work features a new set of metrics for grasp pose candidates evaluation, as well as exploring the impact of high object sampling on grasp success rates. In addition to gripper position sampling, we now perform orientation sampling about the x, y, and z-axes, hence the grasping algorithm no longer require object orientation estimation. Successful experiments have been conducted on a simple jaw gripper (Franka Panda gripper) as well as a complex, high Degree of Freedom (DoF) hand (Allegro hand) as a proof of its versatility. Higher grasp success rates of 76% and 85. 5% respectively has been reported by real world experiments.

IROS Conference 2020 Conference Paper

Incorporating Spatial Constraints into a Bayesian Tracking Framework for Improved Localisation in Agricultural Environments

Muhammad Waqas Khan
Gautham P. Das
Marc Hanheide
Grzegorz Cielniak

Global navigation satellite system (GNSS) has been considered as a panacea for positioning and tracking since the last decade. However, it suffers from severe limitations in terms of accuracy, particularly in highly cluttered and indoor environments. Though real-time kinematics (RTK) supported GNSS promises extremely accurate localisation, employing such services are expensive, fail in occluded environments and are unavailable in areas where cellular base stations are not accessible. It is, therefore, necessary that the GNSS data is to be filtered if high accuracy is required. Thus, this article presents a GNSS-based particle filter that exploits the spatial constraints imposed by the environment. In the proposed setup, the state prediction of the sample set follows a restricted motion according to the topological map of the environment. This results in the transition of the samples getting confined between specific discrete points, called the topological nodes, defined by a topological map. This is followed by a refinement stage where the full set of predicted samples goes through weighting and resampling, where the weight is proportional to the predicted particle's proximity with the GNSS measurement. Thus, a discrete space continuous-time Bayesian filter is proposed, called the Topological Particle Filter (TPF). The proposed TPF is put to test by localising and tracking fruit pickers inside polytunnels. Fruit pickers inside polytunnels can only follow specific paths according to the topology of the tunnel. These paths are defined in the topological map of the polytunnels and are fed to TPF to tracks fruit pickers. Extensive datasets are collected to demonstrate the improved discrete tracking of strawberry pickers inside polytunnels thanks to the exploitation of the environmental constraints.

IROS Conference 2020 Conference Paper

Interactive Movement Primitives: Planning to Push Occluding Pieces for Fruit Picking

Sariah Mghames
Marc Hanheide
Amir M. Ghalamzan E.

Robotic technology is increasingly considered the major mean for fruit picking. However, picking fruits in a dense cluster imposes a challenging research question in terms of motion/path planning as conventional planning approaches may not find collision-free movements for the robot to reach-and-pick a ripe fruit within a dense cluster. In such cases, the robot needs to safely push unripe fruits to reach a ripe one. Nonetheless, existing approaches to planning pushing movements in cluttered environments either are computationally expensive or only deal with 2-D cases and are not suitable for fruit picking, where it needs to compute 3-D pushing movements in a short time. In this work, we present a path planning algorithm for pushing occluding fruits to reach-and-pick a ripe one. Our proposed approach, called Interactive Probabilistic Movement Primitives (I-ProMP), is not computationally expensive (its computation time is in the order of 100 milliseconds) and is readily used for 3-D problems. We demonstrate the efficiency of our approach with pushing unripe strawberries in a simulated polytunnel. Our experimental results confirm I-ProMP successfully pushes table top grown strawberries and reaches a ripe one.

IROS Conference 2019 Conference Paper

Grasping Unknown Objects Based on Gripper Workspace Spheres

Mohamed Sorour
Khaled Elgeneidy
Aravinda Srinivasan
Marc Hanheide
Gerhard Neumann

In this paper, we present a novel grasp planning algorithm for unknown objects given a registered point cloud of the target from different views. The proposed methodology requires no prior knowledge of the object, nor offline learning. In our approach, the gripper kinematic model is used to generate a point cloud of each finger workspace, which is then filled with spheres. At run-time, first the object is segmented, its major axis is computed, in a plane perpendicular to which, the main grasping action is constrained. The object is then uniformly sampled and scanned for various gripper poses that assure at least one object point is located in the workspace of each finger. In addition, collision checks with the object or the table are performed using computationally inexpensive gripper shape approximation. Our methodology is both time efficient (consumes less than 1. 5 seconds in average) and versatile. Successful experiments have been conducted on a simple jaw gripper (Franka Panda gripper) as well as a complex, high Degree of Freedom (DoF) hand (Allegro hand).

ICRA Conference 2018 Conference Paper

3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data

Li Sun 0005
Zhi Yan 0001
Sergi Molina Mellado
Marc Hanheide
Tom Duckett

This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i. e. 2D position plus 1D rotation within the world coordinate system). Our approach, T-Pose-LSTM (Temporal 3DOF-Pose Long-Short-Term Memory), is trained using long-term data from real-world robot deployments and aims to learn context-dependent (environment- and time-specific) human activities. Our approach incorporates long-term temporal information (i. e. date and time) with short-term pose observations as input. A sequence-to-sequence LSTM encoder-decoder is trained, which encodes observations into LSTM and then decodes the resulting predictions. On deployment, the approach can perform on-the-fly prediction in real-time. Instead of using manually annotated data, we rely on a robust human detection, tracking and SLAM system, providing us with examples in a global coordinate system. We validate the approach using more than 15 km of pedestrian trajectories recorded in a care home environment over a period of three months. The experiments show that the proposed T-Pose-LSTM model outperforms the state-of-the-art 2D-based method for human trajectory prediction in long-term mobile robot deployments.

AIJ Journal 2017 Journal Article

Robot task planning and explanation in open and uncertain worlds

Marc Hanheide
Moritz Göbelbecker
Graham S. Horn
Andrzej Pronobis
Kristoffer Sjöö
Alper Aydemir
Patric Jensfelt
Charles Gretton

A long-standing goal of AI is to enable robots to plan in the face of uncertain and incomplete information, and to handle task failure intelligently. This paper shows how to achieve this. There are two central ideas. The first idea is to organize the robot's knowledge into three layers: instance knowledge at the bottom, commonsense knowledge above that, and diagnostic knowledge on top. Knowledge in a layer above can be used to modify knowledge in the layer(s) below. The second idea is that the robot should represent not just how its actions change the world, but also what it knows or believes. There are two types of knowledge effects the robot's actions can have: epistemic effects (I believe X because I saw it) and assumptions (I'll assume X to be true). By combining the knowledge layers with the models of knowledge effects, we can simultaneously solve several problems in robotics: (i) task planning and execution under uncertainty; (ii) task planning and execution in open worlds; (iii) explaining task failure; (iv) verifying those explanations. The paper describes how the ideas are implemented in a three-layer architecture on a mobile robot platform. The robot implementation was evaluated in five different experiments on object search, mapping, and room categorization.

IROS Conference 2016 Conference Paper

Persistent localization and life-long mapping in changing environments using the Frequency Map Enhancement

Tomás Krajník
Jaime Pulido Fentanes
Marc Hanheide
Tom Duckett

We present a lifelong mapping and localisation system for long-term autonomous operation of mobile robots in changing environments. The core of the system is a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future. During navigation, our robot builds temporally local maps and integrates then into the global spatio-temporal grid. Through re-observation of the same locations, the spatio-temporal grid learns the long-term environment dynamics and gains the ability to predict the future environment states. This predictive ability allows to generate time-specific 2d maps used by the robot's localisation and planning modules. By analysing data from a long-term deployment of the robot in a human-populated environment, we show that the proposed representation improves localisation accuracy and the efficiency of path planning. We also show how to integrate the method into the ROS navigation stack for use by other roboticists.

IROS Conference 2016 Conference Paper

Towards automated system and experiment reproduction in robotics

Florian Lier
Marc Hanheide
Lorenzo Natale
Simon Schulz
Jonathan Weisz
Sven Wachsmuth
Sebastian Wrede 0001

Even though research on autonomous robots and human-robot interaction accomplished great progress in recent years, and reusable soft- and hardware components are available, many of the reported findings are only hardly reproducible by fellow scientists. Usually, reproducibility is impeded because required information, such as the specification of software versions and their configuration, required data sets, and experiment protocols are not mentioned or referenced in most publications. In order to address these issues, we recently introduced an integrated tool chain and its underlying development process to facilitate reproducibility in robotics. In this contribution we instantiate the complete tool chain in a unique user study in order to assess its applicability and usability. To this end, we chose three different robotic systems from independent institutions and modeled them in our tool chain, including three exemplary experiments. Subsequently, we asked twelve researchers to reproduce one of the formerly unknown systems and the associated experiment. We show that all twelve scientists were able to replicate a formerly unknown robotics experiment using our tool chain.

ICRA Conference 2015 Conference Paper

Now or later? Predicting and maximising success of navigation actions from long-term experience

Jaime Pulido Fentanes
Bruno Lacerda
Tomás Krajník
Nick Hawes
Marc Hanheide

In planning for deliberation or navigation in real-world robotic systems, one of the big challenges is to cope with change. It lies in the nature of planning that it has to make assumptions about the future state of the world, and the robot's chances of successively accomplishing actions in this future. Hence, a robot's plan can only be as good as its predictions about the world. In this paper, we present a novel approach to specifically represent changes that stem from periodic events in the environment (e. g. a door being opened or closed), which impact on the success probability of planned actions. We show that our approach to model the probability of action success as a set of superimposed periodic processes allows the robot to predict action outcomes in a long-term data obtained in two real-life offices better than a static model. We furthermore discuss and showcase how this knowledge gathered can be successfully employed in a probabilistic planning framework to devise better navigation plans. The key contributions of this paper are (i) the formation of the spectral model of action outcomes from non-uniform sampling, the (ii) analysis of its predictive power using two long-term datasets, and (iii) the application of the predicted outcomes in an MDP-based planning framework.

IROS Conference 2014 Conference Paper

Long-term topological localisation for service robots in dynamic environments using spectral maps

Tomás Krajník
Jaime Pulido Fentanes
Óscar Martínez Mozos
Tom Duckett
Johan Ekekrantz
Marc Hanheide

This paper presents a new approach for topological localisation of service robots in dynamic indoor environments. In contrast to typical localisation approaches that rely mainly on static parts of the environment, our approach makes explicit use of information about changes by learning and modelling the spatio-temporal dynamics of the environment where the robot is acting. The proposed spatio-temporal world model is able to predict environmental changes in time, allowing the robot to improve its localisation capabilities during long-term operations in populated environments. To investigate the proposed approach, we have enabled a mobile robot to autonomously patrol a populated environment over a period of one week while building the proposed model representation. We demonstrate that the experience learned during one week is applicable for topological localization even after a hiatus of three months by showing that the localization error rate is significantly lower compared to static environment representations.

ICRA Conference 2013 Conference Paper

Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection

Christian Lang 0002
Sven Wachsmuth
Marc Hanheide
Heiko Wersing

Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.

IROS Conference 2011 Conference Paper

A system for interactive learning in dialogue with a tutor

Danijel Skocaj
Matej Kristan
Alen Vrecko
Marko Mahnic
Miroslav Janícek
Geert-Jan M. Kruijff
Marc Hanheide
Nick Hawes

In this paper we present representations and mechanisms that facilitate continuous learning of visual concepts in dialogue with a tutor and show the implemented robot system. We present how beliefs about the world are created by processing visual and linguistic information and show how they are used for planning system behaviour with the aim at satisfying its internal drive - to extend its knowledge. The system facilitates different kinds of learning initiated by the human tutor or by the system itself. We demonstrate these principles in the case of learning about object colours and basic shapes.

IJCAI Conference 2011 Conference Paper

Exploiting Probabilistic Knowledge under Uncertain Sensing for Efficient Robot Behaviour

Marc Hanheide
Charles Gretton
R. Dearden
Nick Hawes
Jeremy Wyatt
Andrzej Pronobis
Alper Aydemir
Moritz G

Robots must perform tasks efficiently and reliably while acting underuncertainty. One way to achieve efficiency is to give the robot common-sense knowledge about the structure of the world. Reliable robot behaviour can be achieved by modelling the uncertaintyin the world probabilistically. We present a robot system that combines these two approaches and demonstrate the improvements in efficiency and reliability that result. Our first contribution is a probabilistic relational model integrating common-sense knowledge about the world in general, with observations of a particular environment. Our second contribution is a continual planning system which is able to plan in the large problems posed by that model, by automatically switching between decision-theoretic and classical procedures. We evaluate our system on object search tasks in two different real-world indoor environments. By reasoning about the trade-offs between possible courses of action with different informational effects, and exploiting the cues and general structures of those environments, our robot is able to consistently demonstrate efficient and reliable goal-directed behaviour.

PDF Details DOI

ICRA Conference 2011 Conference Paper

Home alone: Autonomous extension and correction of spatial representations

Nick Hawes
Marc Hanheide
Jack Hargreaves
Ben Page
Hendrik Zender
Patric Jensfelt

In this paper we present an account of the problems faced by a mobile robot given an incomplete tour of an unknown environment, and introduce a collection of techniques which can generate successful behaviour even in the presence of such problems. Underlying our approach is the principle that an autonomous system must be motivated to act to gather new knowledge, and to validate and correct existing knowledge. This principle is embodied in Dora, a mobile robot which features the aforementioned techniques: shared representations, non-monotonic reasoning, and goal generation and management. To demonstrate how well this collection of techniques work in real-world situations we present a comprehensive analysis of the Dora system's performance over multiple tours in an in door environment. In this analysis Dora successfully completed 18 of 21 attempted runs, with all but 3 of these successes requiring one or more of the integrated techniques to recover from problems.

IROS Conference 2011 Conference Paper

Online data-driven fault detection for robotic systems

Raphael Golombek
Sebastian Wrede 0001
Marc Hanheide
Martin Heckmann

In this paper we demonstrate the online applicability of the fault detection and diagnosis approach which we previously developed and published in [1]. In our former work we showed that a purely data driven fault detection approach can be successfully built based on monitored inter-component communication data of a robotic system and used for a-posteriori fault detection. Here we propose an extension to this approach which is capable of online learning of the fault model as well as for online fault detection. We evaluate the application of our approach in the context of a RoboCup task executed by our service robot BIRON in corporation with an expert user.

AAMAS Conference 2010 Conference Paper

Dora The Explorer: A Motivated Robot

Nick Hawes
Marc Hanheide
Kristoffer Sj
ouml;
Alper Ayedemir
Patric Jensfelt
Moritz G
ouml; belbecker

Dora the Explorer is a mobile robot with a sense of curiosity and a drive to explore its world. Given an incompletetour of an indoor environment, Dora is driven by internalmotivations to probe the gaps in her spatial knowledge. Sheactively explores regions of space which she hasn't previouslyvisited but which she expects will lead her to further unexplored space. She will also attempt to determine the categories of rooms through active visual search for functionallyimportant objects, and through ontology-driven inference onthe results of this search.

IROS Conference 2010 Conference Paper

Dynamic path planning adopting human navigation strategies for a domestic mobile robot

Fang Yuan
Lukas Twardon
Marc Hanheide

Mobile robots that are employed in people's homes need to safely navigate their environment. And natural human-inhabited environments still pose significant challenges for robots despite the impressive progress that has been achieved in the field of path planning and obstacle avoidance. These challenges mostly arise from the fact that (i) the perceptual abilities of a robot are limited, thus sometimes impeding its ability to see relevant obstacles (e. g. transparent objects), and (ii) the environment is highly dynamic being populated by humans. In this contribution we are making a case for an integrated solution to these challenges that builds upon the analysis and use of implicit human knowledge in path planning and a cascade of replanning approaches. We combine state of the art path planning and obstacle avoidance algorithms with the knowledge about how humans navigate in their very own environment. The approach results in a more robust and predictable navigation ability for domestic robots as is demonstrated in a number of experimental runs.

IROS Conference 2010 Conference Paper

Learning a probabilistic self-awareness model for robotic systems

Raphael Golombek
Sebastian Wrede 0001
Marc Hanheide
Martin Heckmann

In order to address the problem of failure detection in the robotics domain, we present in this contribution a so-called self-awareness model, based on the system's internal data exchange and the inherent dynamics of inter-component communication. The model is strongly data driven and provides an anomaly detector for robotics systems both applicable in-situ at runtime as well as a-posteriori in post-mortem analysis. Current architectures or methods for failure detection in autonomous robots are either implementations of watch dog concepts or are based on excessive amounts of domain-specific error detection code. The approach presented in this contribution provides an avenue for the detection of more subtle anomalies originating from external sources such as the environment itself or system failures such as resource starvation. Additionally, developers are alleviated from explicitly modeling and foreseeing every exceptional situation, instead training the presented probabilistic model with the known normal modes within the specification of the robot system. As we developed and evaluated the self-awareness model on a mobile robot platform featuring an event-driven software architecture, the presented method can easily be applied in other current robotics software architectures.

ICRA Conference 2009 Conference Paper

Laser-based navigation enhanced with 3D time-of-flight data

Fang Yuan
Agnes Swadzba
Roland Philippsen
Orhan Engin
Marc Hanheide
Sven Wachsmuth

Navigation and obstacle avoidance in robotics using planar laser scans has matured over the last decades. They basically enable robots to penetrate highly dynamic and populated spaces, such as people's home, and move around smoothly. However, in an unconstrained environment the two-dimensional perceptual space of a fixed mounted laser is not sufficient to ensure safe navigation. In this paper, we present an approach that pools a fast and reliable motion generation approach with modern 3D capturing techniques using a time-of-flight camera. Instead of attempting to implement full 3D motion control, which is computationally more expensive and simply not needed for the targeted scenario of a domestic robot, we introduce a ldquovirtual laserrdquo. For the originally solely laser-based motion generation the technique of fusing real laser measurements and 3D point clouds into a continuous data stream is 100% compatible and transparent. The paper covers the general concept, the necessary extrinsic calibration of two very different types of sensors, and exemplarily illustrates the benefit which is to avoid obstacles not being perceivable in the original laser scan.

ICRA Conference 2009 Conference Paper

Mixed-initiative in human augmented mapping

Julia Peltason
Frederic H. K. Siepmann
Thorsten Spexard
Britta Wrede
Marc Hanheide
Elin Anna Topp

In scenarios that require a close collaboration and knowledge transfer between inexperienced users and robots, the ldquolearning by interactingrdquo paradigm goes hand in hand with appropriate representations and learning methods. In this paper we discuss a mixed initiative strategy for robotic learning by interacting with a user in a joint map acquisition process. We propose the integration of an environment representation approach into our interactive learning framework. The environment representation and mapping system supports both user driven and data driven strategies for the acquisition of spatial information, so that a mixed initiative strategy for the learning process is realised. We evaluate our system with test runs according to the scenario of a guided tour, extending the area of operation from structured laboratory environment to less predictable domestic settings.

ICRA Conference 2008 Conference Paper

Who am I talking with? A face memory for social robots

Marc Hanheide
Sebastian Wrede 0001
Christian Lang 0002
Gerhard Sagerer

In order to provide personalized services and to develop human-like interaction capabilities robots need to recognize their human partner. Face recognition has been studied in the past decade exhaustively in the context of security systems and with significant progress on huge datasets. However, these capabilities are not in focus when it comes to social interaction situations. Humans are able to remember people seen for a short moment in time and apply this knowledge directly in their engagement in conversation. In order to equip a robot with capabilities to recall human interlocutors and to provide user- aware services, we adopt human-human interaction schemes to propose a face memory on the basis of active appearance models integrated with the active memory architecture. This paper presents the concept of the interactive face memory, the applied recognition algorithms, and their embedding into the robot's system architecture. Performance measures are discussed for general face databases as well as scenario-specific datasets.