Cyrill Stachniss

AAAI Conference 2026 Conference Paper

Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Clouds (Abstract Reprint)

• Matthias Zeller
• Vardeep Singh Sandhu
• Benedikt Mersch
• Jens Behley
• Michael Heidingsfeld
• Cyrill Stachniss

The perception of moving objects is crucial for autonomous robots performing collision avoidance in dynamic environments. LiDARs and cameras tremendously enhance scene interpretation but do not provide direct motion information and face limitations under adverse weather. Radar sensors overcome these limitations and provide Doppler velocities, delivering direct information on dynamic objects. In this article, we address the problem of moving instance segmentation in radar point clouds to enhance scene interpretation for safety-critical tasks. Our radar instance transformer enriches the current radar scan with temporal information without passing aggregated scans through a neural network. We propose a full-resolution backbone to prevent information loss in sparse point cloud processing. Our instance transformer head incorporates essential information to enhance segmentation but also enables reliable, class-agnostic instance assignments. In sum, our approach shows superior performance on the new moving instance segmentation benchmarks, including diverse environments, and provides model-agnostic modules to enhance scene interpretation.

IROS Conference 2025 Conference Paper

3D Hierarchical Panoptic Segmentation in Real Orchard Environments Across Different Sensors

• Matteo Sodano
• Federico Magistri
• Elias Marks
• Fares Hosn
• Aibek Zurbayev
• Rodrigo Marcuzzi
• Meher V. R. Malladi
• Jens Behley

Crop yield estimation is a relevant problem in agriculture, because an accurate yield estimate can support farmers’ decisions on harvesting or precision intervention. Robots can help to automate this process. To do so, they need to be able to perceive the surrounding environment to identify target objects such as trees and plants. In this paper, we introduce a novel approach to address the problem of hierarchical panoptic segmentation of apple orchards on 3D data from different sensors. Our approach is able to simultaneously provide semantic segmentation, instance segmentation of trunks and fruits, and instance segmentation of trees (a trunk with its fruits). This allows us to identify relevant information such as individual plants, fruits, and trunks, and capture the relationship among them, such as precisely estimate the number of fruits associated to each tree in an orchard. To efficiently evaluate our approach for hierarchical panoptic segmentation, we provide a dataset designed specifically for this task. Our dataset is recorded in Bonn, Germany, in a real apple orchard with a variety of sensors, spanning from a terrestrial laser scanner to a RGB-D camera mounted on different robots platforms. The experiments show that our approach surpasses state-of-the-art approaches in 3D panoptic segmentation in the agricultural domain, while also providing full hierarchical panoptic segmentation. Our dataset is publicly available at https://www.ipb.uni-bonn.de/data/hops/.The open-source implementation of our approach is available at https://github.com/PRBonn/hapt3D.

ICRA Conference 2025 Conference Paper

A Dataset and Benchmark for Shape Completion of Fruits for Agricultural Robotics

• Federico Magistri
• Thomas Läbe
• Elias Marks
• Sumanth Nagulavancha
• Yue Pan 0009
• Claus Smitt
• Lasse Klingbeil
• Michael Halstead

As the world population is expected to reach 10 billion by 2050, our agricultural production system needs to double its productivity despite a decline of human workforce in the agricultural sector. Autonomous robotic systems are one promising pathway to increase productivity by taking over labor-intensive manual tasks like fruit picking. To be effective, such systems need to monitor and interact with plants and fruits precisely, which is challenging due to the cluttered nature of agricultural environments causing, for example, strong occlusions. Thus, being able to estimate the complete 3D shapes of objects in presence of occlusions is crucial for automating operations such as fruit harvesting. In this paper, we propose the first publicly available 3D shape completion dataset for agricultural vision systems. We provide an RGB-D dataset for estimating the 3D shape of fruits. Specifically, our dataset contains RGB-D frames of single sweet peppers in lab conditions but also in a commercial greenhouse. For each fruit, we additionally collected high-precision point clouds that we use as ground truth. For acquiring the ground truth shape, we developed a measuring process that allows us to record data of real sweet pepper plants, both in the lab and in the greenhouse with high precision, and determine the shape of the sensed fruits. We release our dataset, consisting of almost 7, 000 RGB-D frames belonging to more than 100 different fruits. We provide segmented RGB-D frames, with camera intrinsics to easily obtain colored point clouds, together with the corresponding high-precision, occlusion-free point clouds obtained with a high-precision laser scanner. We additionally enable evaluation of shape completion approaches on a hidden test set through a public challenge on a benchmark server.

ICRA Conference 2025 Conference Paper

Adaptive Thresholding for Sequence-Based Place Recognition

• Olga Vysotska
• Igor Bogoslavskyi
• Marco Hutter 0001
• Cyrill Stachniss

Robots need to know where they are in the world to operate effectively without human support. One common first step for precise robot localization is visual place recognition. It is a challenging problem, especially when the output is required in an online fashion, and the current state-of-the-art approaches that tackle it usually require either large amounts of labeled training data or rely on parameters that need to be tuned manually, often per dataset. One such parameter often used for sequence-based place recognition is the image similarity threshold that allows to differentiate between pairs of images that represent the same place even in the presence of severe environmental and structural changes, and those that represent different places even if they share a similar appearance. Currently, selecting this threshold is a manual procedure and requires human expertise. We propose an automatic similarity threshold selection technique and integrate it into a complete sequence-based place recognition system. The experiments on a broad range of real-world and simulated data show that our approach is capable of matching image sequences under various illumination, viewpoint and underlying structural changes, runs online, and requires no manual parameter tuning while yielding performance comparable to a manual, dataset-specific parameter tuning. Thus, this paper substantially increases the ease of use of visual place recognition in real-world settings.

IROS Conference 2025 Conference Paper

Benchmark for Evaluating Long-Term Localization in Indoor Environments under Substantial Static and Dynamic Scene Changes

• Niklas Trekel
• Tiziano Guadagnino
• Thomas Läbe
• Louis Wiesmann
• Perrine Aguiar
• Jens Behley
• Cyrill Stachniss

Accurate localization is crucial for the autonomous operation of mobile robots. Specifically for indoor scenarios, localization algorithms typically rely on a previously generated map. However, many real-world sites like warehouses or healthcare environments violate the underlying assumption that the robot’s surroundings are mainly static. In this paper, we introduce a new dataset plus a benchmark that enables evaluating and comparing indoor localization methods in complex and changing real-world scenarios. While several datasets for indoor scenes exist, only a few combine the long-term localization aspect of repeatedly revisiting the same environment under varying conditions with precise ground truth over multiple rooms. Our dataset comprises various sequences recorded with a wheeled robot covering an office environment. We provide data from two 2D LiDARs, multiple consumer-grade RGB-D cameras, and the robot’s wheel odometry. By densely placing fiducial markers on every room ceiling, we can also provide accurate pose information within a single global frame for the whole environment, estimated through an additional upward-facing camera. We evaluate existing localization algorithms on our data and make the dataset together with a server-based benchmark evaluation publicly available. This facilitates an unbiased evaluation of localization approaches and enables further research on their application in challenging indoor scenarios.

IROS Conference 2025 Conference Paper

Coherent Online Road Topology Estimation and Reasoning with Standard-Definition Maps

• Khanh Son Pham
• Christian Witte
• Jens Behley
• Johannes Betz
• Cyrill Stachniss

Most autonomous cars rely on the availability of high-definition (HD) maps. Current research aims to address this constraint by directly predicting HD map elements from onboard sensors and reasoning about the relationships between the predicted map and traffic elements. Despite recent advancements, the coherent online construction of HD maps remains a challenging endeavor, as it necessitates modeling the high complexity of road topologies in a unified and consistent manner. To address this challenge, we propose a coherent approach to predict lane segments and their corresponding topology, as well as road boundaries, all by leveraging prior map information represented by commonly available standard-definition (SD) maps. We propose a network architecture, which leverages hybrid lane segment encodings comprising prior information and denoising techniques to enhance training stability and performance. Furthermore, we facilitate past frames for temporal consistency. Our experimental evaluation demonstrates that our approach outperforms previous methods by a large margin, highlighting the benefits of our modeling scheme.

ICRA Conference 2025 Conference Paper

Digiforests: a Longitudinal Lidar Dataset for Forestry Robotics

• Meher V. R. Malladi
• Nived Chebrolu
• Irene Scacchetti
• Luca Lobefaro
• Tiziano Guadagnino
• Benoît Casseau
• Haedam Oh
• Leonard Freißmuth

Forests are vital to our ecosystems, acting as carbon sinks, climate stabilizers, biodiversity centers, and wood sources. Due to their scale, monitoring and managing forests takes a lot of work. Forestry robotics offers the potential for enabling efficient and sustainable foresting practices through automation. Despite increasing interest in this field, the scarcity of robotics datasets and benchmarks in forest environments is hampering progress in this domain. In this paper, we present a real-world, longitudinal dataset for forestry robotics that enables the development and comparison of approaches for various relevant applications, ranging from semantic interpretation to estimating traits relevant to forestry management. The dataset consists of multiple recordings of the same plots in a forest in Switzerland during three different growth periods. We recorded the data with a mobile 3D LiDAR scanning setup. Additionally, we provide semantic annotations of trees, shrubs, and ground, instance-level annotations of trees, as well as more fine-grained annotations of tree stems and crowns. Furthermore, we provide reference field measurements of traits relevant to forestry management for a subset of the trees. Together with the data, we also provide open-source baseline panoptic segmentation and tree trait estimation approaches to enable the community to bootstrap further research and simplify comparisons in this domain.

ICRA Conference 2025 Conference Paper

Ground-Aware Automotive Radar Odometry

• Daniel Casado Herraez
• Franz Kaschner
• Matthias Zeller
• Dominik Muhle
• Jens Behley
• Michael Heidingsfeld
• Daniel Cremers
• Cyrill Stachniss

Odometry is crucial for the navigation of autonomous vehicles in unknown environments. While cameras and LiDARs are commonly used to estimate the ego-motion of a vehicle, these sensors face limitations under bad lighting and severe weather conditions. Automotive radars overcome these challenges, but radar point clouds are generally sparse and noisy, making it difficult to identify useful features within a radar scan. In this paper, we address the problem of ego-motion estimation using a single automotive radar sensor. We propose a simple, yet effective, heuristic-based method to extract the ground plane from single radar scans and perform ground plane matching between consecutive scans. Additionally, we perform a windowed factor-graph optimization of the poses together with the ground plane, improving the accuracy of the pose estimation. We put our work to the test using the 4DRadarDataset. Our findings illustrate the state-of-the-art performance of our odometry approach compared to existing alternatives that use radar point clouds.

ICRA Conference 2025 Conference Paper

Improving Indoor Localization Accuracy by Using an Efficient Implicit Neural Map Representation

• Haofei Kuang
• Yue Pan 0009
• Xingguang Zhong
• Louis Wiesmann
• Jens Behley
• Cyrill Stachniss

Globally localizing a mobile robot in a known map is often a foundation for enabling robots to navigate and operate autonomously. In indoor environments, traditional Monte Carlo localization based on occupancy grid maps is considered the gold standard, but its accuracy is limited by the representation capabilities of the occupancy grid map. In this paper, we address the problem of building an effective map representation that allows to accurately perform probabilistic global localization. To this end, we propose an implicit neural map representation that is able to capture positional and directional geometric features from 2D LiDAR scans to efficiently represent the environment and learn a neural network that is able to predict both, the non-projective signed distance and a direction-aware projective distance for an arbitrary point in the mapped environment. This combination of neural map representation with a lightweight neural network allows us to design an efficient observation model within a conventional Monte Carlo localization framework for pose estimation of a robot in real time. We evaluated our approach to indoor localization on a publicly available dataset for global localization and the experimental results indicate that our approach is able to more accurately localize a mobile robot than other localization approaches employing occupancy or existing neural map representations. In contrast to other approaches employing an implicit neural map representation for 2D LiDAR localization, our approach allows to perform real-time pose tracking after convergence and near real-time global localization. The code of our approach is available at: https://github.com/PRBonn/enm-mcl.

ICRA Conference 2025 Conference Paper

Kinematic-ICP: Enhancing LiDAR Odometry with Kinematic Constraints for Wheeled Mobile Robots Moving on Planar Surfaces

• Tiziano Guadagnino
• Benedikt Mersch
• Ignacio Vizzo
• Saurabh Gupta
• Meher V. R. Malladi
• Luca Lobefaro
• Guillaume Doisy
• Cyrill Stachniss

LiDAR odometry is essential for many robotics applications, including 3D mapping, navigation, and simultaneous localization and mapping. LiDAR odometry systems are usually based on some form of point cloud registration to compute the ego-motion of a mobile robot. Yet, few of today's LiDAR odometry systems consider domain-specific knowledge or the kinematic model of the mobile platform during the point cloud alignment. In this paper, we present Kinematic-ICP, a LiDAR odometry system that focuses on wheeled mobile robots equipped with a 3D LiDAR and moving on a planar surface, which is a common assumption for warehouses, offices, hospitals, etc. Our approach introduces kinematic constraints within the optimization of a traditional point-to-point iterative closest point scheme. In this way, the resulting motion follows the kinematic constraints of the platform, effectively exploiting the robot's wheel odometry and the 3D LiDAR observations. We dynamically adjust the influence of LiDAR measurements and wheel odometry in our optimization scheme, allowing the system to handle degenerate scenarios such as feature-poor corridors. We evaluate our approach on robots operating in large-scale warehouse environments, but also outdoors. The experiments show that our approach achieves top performances and is more accurate than wheel odometry and common LiDAR odometry systems. Kinematic-ICP has been recently deployed in the Dexory fleet of robots operating in warehouses worldwide at their customers' sites, showing that our method can run in the real world alongside a complete navigation stack.

IROS Conference 2025 Conference Paper

KISS-SLAM: A Simple, Robust, and Accurate 3D LiDAR SLAM System With Enhanced Generalization Capabilities

• Tiziano Guadagnino
• Benedikt Mersch
• Saurabh Gupta
• Ignacio Vizzo
• Giorgio Grisetti
• Cyrill Stachniss

Robust and accurate localization and mapping of an environment using laser scanners, so-called LiDAR SLAM, is essential to many robotic applications. Early 3D LiDAR SLAM methods often exploited additional information from IMU or GNSS sensors to enhance localization accuracy and mitigate drift. Later, advanced systems further improved the estimation at the cost of a higher runtime and complexity. This paper explores the limits of what can be achieved with a LiDAR-only SLAM approach while following the "Keep It Small and Simple" (KISS) principle. By leveraging this minimalist design principle, our system, KISS-SLAM, achieves state-of-the-art performance in pose accuracy while requiring little to no parameter tuning for deployment across diverse environments, sensors, and motion profiles. We follow best practices in graph-based SLAM and build upon LiDAR odometry to compute the relative motion between scans and construct local maps of the environment. To correct drift, we match local maps and optimize the trajectory in a pose graph optimization step. The experimental results demonstrate that this design achieves competitive performance while reducing complexity and reliance on additional sensor modalities. By prioritizing simplicity, this work provides a new strong baseline for LiDAR-only SLAM and a high-performing starting point for future research. Furthermore, our pipeline builds consistent maps that can be used directly for downstream tasks like navigation. Our open-source system operates faster than the sensor frame rate in all presented datasets and is designed for real-world scenarios.

IROS Conference 2025 Conference Paper

Zero-Shot Semantic Segmentation for Robots in Agriculture

• Yue Linn Chong
• Lucas Nunes
• Federico Magistri
• Xingguang Zhong
• Jens Behley
• Cyrill Stachniss

Conventional crop production, which is essential for providing food, feed, fuel, and fiber for our society, relies heavily on harmful herbicides to control weeds. Instead, agricultural robots could remove weeds more sustainably. However, these robots require a generalizable perception system that can locate weeds, enabling automatic removal of weeds. Specifically, they need to perform crop-weed semantic segmentation, which locates and distinguishes between the crop and the weed plants with pixel-level resolution. However, most existing crop-weed semantic segmentation methods are fully supervised and require expensive and labor-intensive pixel-wise labeling of the training data. To avoid the costly labeling process, we address the problem of unsupervised crop-weed segmentation in this paper. Unlike previous approaches, we leverage the idea that weeds are "weird" plants that occur less frequently and are highly variable in appearance, and reframe the problem as an anomaly segmentation problem. We propose an approach to segment weeds as anomalous plants by categorizing plants in the feature space of a pretrained foundation model. Our approach curates a bag-of-features representation of crop features and models the manifold of crop plants as hyperspheres. During inference, it classifies vegetation segments of the image with features within this manifold as crop plants and all other plants as weeds. Our experiments show that our zero-shot anomaly segmentation method can perform crop-weed segmentation on several datasets from real crop fields.

IROS Conference 2024 Conference Paper

BonnBeetClouds3D: A Dataset Towards Point Cloud-Based Organ-Level Phenotyping of Sugar Beet Plants Under Real Field Conditions

• Elias Marks
• Jonas Bömer
• Federico Magistri
• Anurag Sag
• Jens Behley
• Cyrill Stachniss

Agricultural production is facing challenges in the next decades induced by climate change and the need for more sustainability by reducing its impact on the environment. Advances in field management through robotic intervention, monitoring of crops by autonomous unmanned aerial vehicles (UAVs) supporting breeding of novel and more resilient crop varieties can help to address these challenges. The analysis of plant traits is called phenotyping and is an essential activity in plant breeding; it however involves a great amount of manual labor. With this paper, we provide means to better tackle the problems of instance segmentation to support robotic intervention and automatic fine-grained, organ-level geometric analysis needed for precision phenotyping. As the availability of real-world data in this domain is relatively scarce, we provide a novel dataset that was acquired using UAVs capturing high-resolution images of real breeding trials containing 48 plant varieties and therefore covering a relevant morphological and appearance spectrum. This enables the development of approaches for instance segmentation and autonomous phenotyping that generalize well to different plant varieties. Based on overlapping high-resolution images taken from multiple viewing angles, we provide photogrammetric dense point clouds and provide detailed and accurate point-wise labels for plants, leaves, and salient points as the tip and the base in 3D. Additionally, we include measurements of phenotypic traits performed by experts from the German Federal Plant Variety Office on the real plants, allowing the evaluation of new approaches not only on segmentation and keypoint detection but also directly on actual traits. The provided labeled point clouds enable finegrained plant analysis and support further progress in the development of automatic phenotyping approaches, but also enable further research in surface reconstruction, point cloud completion, and semantic interpretation of point clouds.

ICRA Conference 2024 Conference Paper

Effectively Detecting Loop Closures using Point Cloud Density Maps

• Saurabh Gupta
• Tiziano Guadagnino
• Benedikt Mersch
• Ignacio Vizzo
• Cyrill Stachniss

The ability to detect loop closures plays an essential role in any SLAM system. Loop closures allow correcting the drifting pose estimates from a sensor odometry pipeline. In this paper, we address the problem of effectively detecting loop closures in LiDAR SLAM systems in various environments with longer lengths of sequences and agnostic of the scanning pattern of the sensor. While many approaches for loop closures using 3D LiDAR sensors rely on individual scans, we propose the usage of local maps generated from locally consistent odometry estimates. Several recent approaches compute the maximum elevation map on a bird’s eye view projection of point clouds to compute feature descriptors. In contrast, we use a density image bird’s eye view representation, which is robust to viewpoint changes. The utilization of dense local maps allows us to reduce the complexity of features describing these maps, as well as the size of the database required to store these features over a long sequence. This yields a real-time application of our approach for a typical robotic 3D LiDAR sensor. We perform extensive experiments to evaluate our approach against other state-of-the-art approaches and show the benefits of our proposed approach.

ICRA Conference 2024 Conference Paper

Efficient and Accurate Transformer-Based 3D Shape Completion and Reconstruction of Fruits for Agricultural Robots

• Federico Magistri
• Rodrigo Marcuzzi
• Elias Marks
• Matteo Sodano
• Jens Behley
• Cyrill Stachniss

Robots that operate in agricultural environments need a robust perception system that can deal with occlusions, which are naturally present in agricultural scenarios. In this paper, we address the problem of estimating 3D shapes of fruits when only partial observations are available. Generally speaking, such a shape completion can be realized by exploiting prior knowledge about the geometry of the fruit. This is typically done by template matching using traditional optimization algorithms, which are slow but accurate, or by encoding such knowledge into the weights of a neural network, leading to faster but often less accurate estimates. Our approach combines the best of both worlds. It exploits the benefit of having a template representing our object of interest with the advantages of using a neural network to learn how to deform a template. Our experimental evaluation demonstrates that our approach yields accurate estimation at a competitively low inference time in challenging greenhouse environments.

IROS Conference 2024 Conference Paper

Exploiting Priors from 3D Diffusion Models for RGB-Based One-Shot View Planning

• Sicong Pan
• Liren Jin
• Xuying Huang
• Cyrill Stachniss
• Marija Popovic
• Maren Bennewitz

Object reconstruction is relevant for many autonomous robotic tasks that require interaction with the environment. A key challenge in such scenarios is planning view configurations to collect informative measurements for reconstructing an initially unknown object. One-shot view planning enables efficient data collection by predicting view configurations and planning the globally shortest path connecting all views at once. However, prior knowledge about the object is required to conduct one-shot view planning. In this work, we propose a novel one-shot view planning approach that utilizes the powerful 3D generation capabilities of diffusion models as priors. By incorporating such geometric priors into our pipeline, we achieve effective one-shot view planning starting with only a single RGB image of the object to be reconstructed. Our planning experiments in simulation and real-world setups indicate that our approach balances well between object reconstruction quality and movement cost.

IROS Conference 2024 Conference Paper

Fast Global Point Cloud Registration using Semantic NDT

• Robert Schirmer
• Narunas Vaskevicius
• Peter Biber
• Cyrill Stachniss

Robust and accurate point cloud registration is an essential part of many robotic tasks such as SLAM or object pose retrieval. In this paper, we address the problem of global 3D point cloud registration, i. e. , the task of estimating the 3D rigid body transform between a source and a target point cloud without any initial guess. Typically, the problem is solved by extracting and matching features to find a data association and then computing a transform that minimizes the squared distance between points. Our approach combines the normal distributions transform and oriented point pair framework and introduces the NDT distance histogram to quickly generate and test candidate transforms. Our method further exploits semantic information if available for greater speed. We implement our algorithm in C++ and compare it to other state-of-the-art approaches on a diverse set of environments. Our evaluation shows that our method outperforms the other approaches, especially concerning run-time and compute efficiency.

IROS Conference 2024 Conference Paper

HeLiMOS: A Dataset for Moving Object Segmentation in 3D Point Clouds From Heterogeneous LiDAR Sensors

• Hyungtae Lim
• Seoyeon Jang
• Benedikt Mersch
• Jens Behley
• Hyun Myung
• Cyrill Stachniss

Moving object segmentation (MOS) using a 3D light detection and ranging (LiDAR) sensor is crucial for scene understanding and identification of moving objects. Despite the availability of various types of 3D LiDAR sensors in the market, MOS research still predominantly focuses on 3D point clouds from mechanically spinning omnidirectional LiDAR sensors. Thus, we are, for example, lacking a dataset with MOS labels for point clouds from solid-state LiDAR sensors which have irregular scanning patterns. In this paper, we present a labeled dataset, called HeLiMOS, that enables to test MOS approaches on four heterogeneous LiDAR sensors, including two solid-state LiDAR sensors. Furthermore, we introduce a novel automatic labeling method to substantially reduce the labeling effort required from human annotators. To this end, our framework exploits an instance-aware static map building approach and tracking-based false label filtering. Finally, we provide experimental results regarding the performance of commonly used state-of-the-art MOS approaches on HeLiMOS that suggest a new direction for a sensor-agnostic MOS, which generally works regardless of the type of LiDAR sensors used to capture 3D point clouds. Our dataset is available at https://sites.google.com/view/helimos.

IROS Conference 2024 Conference Paper

Leveraging GNSS and Onboard Visual Data from Consumer Vehicles for Robust Road Network Estimation

• Balázs Opra
• Betty Le Dem
• Jeffrey M. Walls
• Dimitar Lukarski
• Cyrill Stachniss

Maps are essential for diverse applications, such as vehicle navigation and autonomous robotics. Both require spatial models for effective route planning and localization. This paper addresses the challenge of road graph construction for autonomous vehicles. Despite recent advances, creating a road graph remains labor-intensive and has yet to achieve full automation. The goal of this paper is to generate such graphs automatically and accurately. Modern cars are equipped with onboard sensors used for today's advanced driver assistance systems like lane keeping. We propose using global navigation satellite system (GNSS) traces and basic image data acquired from these standard sensors in consumer vehicles to estimate road-level maps with minimal effort. We exploit the spatial information in the data by framing the problem as a road centerline semantic segmentation task using a convolutional neural network. We also utilize the data’s time series nature to refine the neural network’s output by using map matching. We implemented and evaluated our method using a fleet of real consumer vehicles, only using the deployed onboard sensors. Our evaluation demonstrates that our approach not only matches existing methods on simpler road configurations but also significantly outperforms them on more complex road geometries and topologies. This work received the 2023 Woven by Toyota Invention Award.

ICRA Conference 2024 Conference Paper

LIO-EKF: High Frequency LiDAR-Inertial Odometry using Extended Kalman Filters

• Yibin Wu
• Tiziano Guadagnino
• Louis Wiesmann
• Lasse Klingbeil
• Cyrill Stachniss
• Heiner Kuhlmann

Odometry estimation is crucial for every autonomous system requiring navigation in an unknown environment. In modern mobile robots, 3D LiDAR-inertial systems are often used for this task. By fusing LiDAR scans and IMU measurements, these systems can reduce the accumulated drift caused by sequentially registering individual LiDAR scans and provide a robust pose estimate. Although effective, LiDAR-inertial odometry systems require proper parameter tuning to be deployed. In this paper, we propose LIO-EKF, a tightly-coupled LiDAR-inertial odometry system based on point-to-point registration and the classical extended Kalman filter scheme. We propose an adaptive data association that considers the relative pose uncertainty, the map discretization errors, and the LiDAR noise. In this way, we can substantially reduce the parameters to tune for a given type of environment. The experimental evaluation suggests that the proposed system performs on par with the state-of-the-art LiDAR-inertial odometry pipelines but is significantly faster in computing the odometry. The source code of our implementation is publicly available (https://github.com/YibinWu/LIO-EKF).

ICRA Conference 2024 Conference Paper

Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds

• Matthias Zeller
• Daniel Casado Herraez
• Jens Behley
• Michael Heidingsfeld
• Cyrill Stachniss

Robots and autonomous vehicles should be aware of what happens in their surroundings. The segmentation and tracking of moving objects are essential for reliable path planning, including collision avoidance. We investigate this estimation task for vehicles using radar sensing. We address moving instance tracking in sparse radar point clouds to enhance scene interpretation. We propose a learning-based radar tracker incorporating temporal offset predictions to enable direct center-based association and enhance segmentation performance by including additional motion cues. We implement attention-based tracking for sparse radar scans to include appearance features and enhance performance. The final association combines geometric and appearance features to overcome the limitations of center-based tracking to associate instances reliably. Our approach shows an improved performance on the moving instance tracking benchmark of the RadarScenes dataset compared to the current state of the art.

ICRA Conference 2024 Conference Paper

Radar-Only Odometry and Mapping for Autonomous Vehicles

• Daniel Casado Herraez
• Matthias Zeller
• Le Chang
• Ignacio Vizzo
• Michael Heidingsfeld
• Cyrill Stachniss

Odometry and mapping play a pivotal role in the navigation of autonomous vehicles. In this paper, we address the problem of pose estimation and map creation using only radar sensors. We focus on two odometry estimation approaches followed by a mapping step. The first one is a new point-to-point ICP approach that leverages the velocity information provided by 3D radar sensors. The second one is advantageous for 2D radars with a low number of samples, and particularly useful for scenarios where the sensor is being blocked by large dynamic obstacles. It exploits a constant velocity filter and the measured Doppler velocities to estimate the vehicle’s ego-motion. We enrich this with a filtering step to improve the accuracy of the points in the resulting map. We put our work to the test using the View of Delft and NuScenes datasets, which involve 3D and 2D radar sensors. Our findings illustrate state-of-the-art performance of our odometry techniques in terms of accuracy when compared to existing alternatives. Moreover, we demonstrate that our map filtering methodology achieves higher similarity rates than the raw unfiltered map when benchmarked against a corresponding LiDAR map.

IROS Conference 2024 Conference Paper

Spatio-Temporal Consistent Mapping of Growing Plants for Agricultural Robots in the Wild

• Luca Lobefaro
• Meher V. R. Malladi
• Tiziano Guadagnino
• Cyrill Stachniss

Tracking changes in growing plants is important for automating phenotyping and robots managing crops. In this paper, we propose a system that uses a 3D model of plants along crop rows to enable a robotic platform to localize itself even in the presence of heavy changes and deforming the model to adapt the scene description to the new measurements. In particular, we focus on consumer RGB-D cameras due to their cost-effectiveness and ease of deployment on real platforms. Our approach exploits modern deep-learning-based feature descriptors and geometric information to obtain matches between 3D points corresponding to temporally distant sessions. We then use the associations in a non-rigid registration pipeline to obtain the final result, an updated representation of the 3D model that reflects plant changes. Using a standard RGB-D sensor, we validate our approach on a real-world dataset recorded in a glasshouse. We obtain accurate 4D models of the plants and track the plant traits’ evolution over time. We show, through experiments, that our method is applicable to interpolate plant organs’ evolution, a helpful result for phenotypic trait measurement. We see our approach as a relevant step toward 4D reconstruction for robotic agriculture in the wild.

IROS Conference 2024 Conference Paper

STAIR: Semantic-Targeted Active Implicit Reconstruction

• Liren Jin
• Haofei Kuang
• Yue Pan 0009
• Cyrill Stachniss
• Marija Popovic

Many autonomous robotic applications require object-level understanding when deployed. Actively reconstructing objects of interest, i. e. objects with specific semantic meanings, is therefore relevant for a robot to perform downstream tasks in an initially unknown environment. In this work, we propose a novel framework for semantic-targeted active reconstruction using posed RGB-D measurements and 2D semantic labels as input. The key components of our framework are a semantic implicit neural representation and a compatible planning utility function based on semantic rendering and uncertainty estimation, enabling adaptive view planning to target objects of interest. Our planning approach achieves better reconstruction performance in terms of mesh and novel view rendering quality compared to implicit reconstruction baselines that do not consider semantics for view planning. Our framework further outperforms a state-of-the-art semantic-targeted active reconstruction pipeline based on explicit maps, justifying our choice of utilising implicit neural representations to tackle semantic-targeted active reconstruction problems.

ICRA Conference 2024 Conference Paper

Tree Instance Segmentation and Traits Estimation for Forestry Environments Exploiting LiDAR Data Collected by Mobile Robots

• Meher V. R. Malladi
• Tiziano Guadagnino
• Luca Lobefaro
• Matías Mattamala
• Holger Griess
• Janine Schweier
• Nived Chebrolu
• Maurice F. Fallon

Forests play a crucial role in our ecosystems, functioning as carbon sinks, climate stabilizers, biodiversity hubs, and sources of wood. By the very nature of their scale, monitoring and maintaining forests is a challenging task. Robotics in forestry can have the potential for substantial automation toward efficient and sustainable foresting practices. In this paper, we address the problem of automatically producing a forest inventory by exploiting LiDAR data collected by a mobile platform. To construct an inventory, we first extract tree instances from point clouds. Then, we process each instance to extract forestry inventory information. Our approach provides the per-tree geometric trait of "diameter at breast height" together with the individual tree locations in a plot. We validate our results against manual measurements collected by foresters during field trials. Our experiments show strong segmentation and tree trait estimation performance, underlining the potential for automating forestry services. Results furthermore show a superior performance compared to the popular baseline methods used in this domain.

IROS Conference 2023 Conference Paper

Constructing Metric-Semantic Maps Using Floor Plan Priors for Long-Term Indoor Localization

• Nicky Zimmerman
• Matteo Sodano
• Elias Marks
• Jens Behley
• Cyrill Stachniss

Object-based maps are relevant for scene under-standing since they integrate geometric and semantic information of the environment, allowing autonomous robots to robustly localize and interact with on objects. In this paper, we address the task of constructing a metric-semantic map for the purpose of long-term object-based localization. We exploit 3D object detections from monocular RGB frames for both, the object-based map construction, and for globally localizing in the constructed map. To tailor the approach to a target environment, we propose an efficient way of generating 3D annotations to finetune the 3D object detection model. We evaluate our map construction in an office building, and test our long-term localization approach on challenging sequences recorded in the same environment over nine months. The experiments suggest that our approach is suitable for constructing metric-semantic maps, and that our localization approach is robust to long-term changes. Both, the mapping algorithm and the localization pipeline can run online on an onboard computer. We release an open-source C++/ros implementation of our approach.

IROS Conference 2023 Conference Paper

Estimating 4D Data Associations Towards Spatial-Temporal Mapping of Growing Plants for Agricultural Robots

• Luca Lobefaro
• Meher V. R. Malladi
• Olga Vysotska
• Tiziano Guadagnino
• Cyrill Stachniss

Our world is non-static, and robots should be able to track its changing geometry. For tracking changes, data asso-ciations between 3D points over time are key. In this paper, we investigate the problem of associating 3D points on plant organs from different mapping runs over time while the plants grow. We achieve a high spatial-temporal matching performance by combining 3D RGB-D SLAM, visual place recognition, and 2D/3D matching exploiting background knowledge. We showcase our approach in a real agricultural glasshouse used to grow sweet peppers, using RGB-D observations from a mobile robot traversing the environment. Our experiments suggest that with our approach, we can robustly make data associations in highly repetitive scenes and under changing geometries caused by plant growth. We see our approach as an important step towards spatial-temporal data association for robotic agriculture.

ICRA Conference 2023 Conference Paper

Fruit Tracking Over Time Using High-Precision Point Clouds

• Alessandro Riccardi
• Shane Kelly
• Elias Marks
• Federico Magistri
• Tiziano Guadagnino
• Jens Behley
• Maren Bennewitz
• Cyrill Stachniss

Monitoring the traits of plants and fruits is a fundamental task in horticulture. With accurate measurements, farmers can predict the yield of their crops and use this information for making informed management decisions, and breeders can use it for variety selection. Agricultural robotic applications promise to automate this monitoring task. In this paper, we address the problem of monitoring fruit growth and investigate the matching of fruits recorded in commercial greenhouses at different growth stages based on data recorded from terrestrial laser scanners. This is challenging as fruits appear highly similar, change over time, and are subject to severe occlusions. We first propose a fruit descriptor, which captures the topology of the fruit surroundings to facilitate the matching between different points in time. We capture and describe the relationship between a fruit and its neighbors such that our descriptors are less affected by the growth over time. Furthermore, we define a matching cost function and use an optimal assignment algorithm to match the fruit observations taken in different weeks. The experiments show that our descriptor achieves a high spatio-temporal matching accuracy, which is superior to the commonly used geometric point cloud descriptors.

ICRA Conference 2023 Conference Paper

Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain

• Gianmarco Roggiolani
• Matteo Sodano
• Tiziano Guadagnino
• Federico Magistri
• Jens Behley
• Cyrill Stachniss

Plant phenotyping is a central task in agriculture, as it describes plants' growth stage, development, and other relevant quantities. Robots can help automate this process by accurately estimating plant traits such as the number of leaves, leaf area, and the plant size. In this paper, we address the problem of joint semantic, plant instance, and leaf instance segmentation of crop fields from RGB data. We propose a single convolutional neural network that addresses the three tasks simultaneously, exploiting their underlying hierarchical structure. We introduce task-specific skip connections, which our experimental evaluation proves to be more beneficial than the usual schemes. We also propose a novel automatic post-processing, which explicitly addresses the problem of spatially close instances, common in the agricultural domain because of overlapping leaves. Our architecture simultaneously tackles these problems jointly in the agricultural context. Previous works either focus on plant or leaf segmentation, or do not optimise for semantic segmentation. Results show that our system has superior performance compared to state-of-the-art approaches, while having a reduced number of parameters and is operating at camera frame rate.

ICRA Conference 2023 Conference Paper

Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors

• Hao Dong 0011
• Xieyuanli Chen
• Mihai Dusmanu
• Viktor Larsson
• Marc Pollefeys
• Cyrill Stachniss

A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks, such as image matching, image retrieval, and visual localization. State-of-the-art descriptors, from hand-crafted descriptors such as SIFT to learned ones such as HardNet, are usually high-dimensional; 128 dimensions or even more. The higher the dimensionality, the larger the memory consumption and computational time for approaches using such descriptors. In this paper, we investigate multi-layer perceptrons (MLPs) to extract low-dimensional but high-quality descriptors. We thoroughly analyze our method in unsuper-vised, self-supervised, and supervised settings, and evaluate the dimensionality reduction results on four representative descriptors. We consider different applications, including visual localization, patch verification, image matching and retrieval. The experiments show that our lightweight MLPs trained using supervised method achieve better dimensionality reduction than PCA. The lower-dimensional descriptors generated by our approach outperform the original higher-dimensional descriptors in downstream tasks, especially for the hand-crafted ones. The code is available at https://github.com/PRBonn/descriptor-dr.

ICRA Conference 2023 Conference Paper

On Domain-Specific Pre- Training for Effective Semantic Perception in Agricultural Robotics

• Gianmarco Roggiolani
• Federico Magistri
• Tiziano Guadagnino
• Jan Weyler
• Giorgio Grisetti
• Cyrill Stachniss
• Jens Behley

Agricultural robots have the prospect to enable more efficient and sustainable agricultural production of food, feed, and fiber. Perception of crops and weeds is a central component of agricultural robots that aim to monitor fields and assess the plants as well as their growth stage in an automatic manner. Semantic perception mostly relies on deep learning using supervised approaches, which require time and qualified workers to label fairly large amounts of data. In this paper, we look into the problem of reducing the amount of labels without compromising the final segmentation performance. For robots operating in the field, pre-training networks in a supervised way is already a popular method to reduce the number of required labeled images. We investigate the possibility of pre-training in a self-supervised fashion using data from the target domain. To better exploit this data, we propose a set of domain-specific augmentation strategies. We evaluate our pre-training on semantic segmentation and leaf instance segmentation, two important tasks in our domain. The experimental results suggest that pre-training with domain-specific data paired with our data augmentation strategy leads to superior performance compared to commonly used pre-trainings. Furthermore, the pre-trained networks obtain similar performance to the fully supervised with less labeled data.

IROS Conference 2023 Conference Paper

Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots

• Yue Pan 0009
• Federico Magistri
• Thomas Läbe
• Elias Marks
• Claus Smitt
• Chris McCool
• Jens Behley
• Cyrill Stachniss

Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. Obtaining such 3D information is non-trivial as agricultural environments are often repetitive and cluttered, and one has to account for the partial observability of fruit and plants. In this paper, we address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot. To this end, we propose an online multi-resolution panoptic mapping system where regions of interest are represented with a higher resolution. We exploit data to learn a general fruit shape representation that we use at inference time together with an occlusion-aware differentiable rendering pipeline to complete partial fruit observations and estimate the 7 DoF pose of each fruit in the map. The experiments presented in this paper, evaluated both in the controlled environment and in a commercial greenhouse, show that our novel algorithm yields higher completion and pose estimation accuracy than existing methods, with an improvement of 41 % in completion accuracy and 52 % in pose estimation accuracy while keeping a low inference time of 0. 6 s in average.

ICRA Conference 2023 Conference Paper

Radar Velocity Transformer: Single-scan Moving Object Segmentation in Noisy Radar Point Clouds

• Matthias Zeller
• Vardeep S. Sandhu
• Benedikt Mersch
• Jens Behley
• Michael Heidingsfeld
• Cyrill Stachniss

The awareness about moving objects in the surroundings of a self-driving vehicle is essential for safe and reliable autonomous navigation. The interpretation of LiDAR and camera data achieves exceptional results but typically requires to accumulate and process temporal sequences of data in order to extract motion information. In contrast, radar sensors, which are already installed in most recent vehicles, can overcome this limitation as they directly provide the Doppler velocity of the detections and, hence incorporate instantaneous motion information within a single measurement. In this paper, we tackle the problem of moving object segmentation in noisy radar point clouds. We also consider differentiating parked from moving cars, to enhance scene understanding. Instead of exploiting temporal dependencies to identify moving objects, we develop a novel transformer-based approach to perform single-scan moving object segmentation in sparse radar scans accurately. The key to our Radar Velocity Transformer is to incorporate the valuable velocity information throughout each module of the network, thereby enabling the precise segmentation of moving and non-moving objects. Additionally, we propose a transformer-based upsampling, which enhances the performance by adaptively combining information and over-coming the limitation of interpolation of sparse point clouds. Finally, we create a new radar moving object segmentation benchmark based on the RadarScenes dataset and compare our approach to other state-of-the-art methods. Our network runs faster than the frame rate of the sensor and shows superior segmentation results using only single-scan radar data.

ICRA Conference 2023 Conference Paper

Robust Double-Encoder Network for RGB-D Panoptic Segmentation

• Matteo Sodano
• Federico Magistri
• Tiziano Guadagnino
• Jens Behley
• Cyrill Stachniss

Perception is crucial for robots that act in real-world environments, as autonomous systems need to see and understand the world around them to act properly. Panoptic segmentation provides an interpretation of the scene by computing a pixelwise semantic label together with instance IDs. In this paper, we address panoptic segmentation using RGB-D data of indoor scenes. We propose a novel encoder-decoder neural network that processes RGB and depth separately through two encoders. The features of the individual encoders are progressively merged at different resolutions, such that the RGB features are enhanced using complementary depth information. We propose a novel merging approach called ResidualExcite, which reweighs each entry of the feature map according to its importance. With our double-encoder architecture, we are robust to missing cues. In particular, the same model can train and infer on RGB-D, RGB-only, and depth-only input data, without the need to train specialized models. We evaluate our method on publicly available datasets and show that our approach achieves superior results compared to other common approaches for panoptic segmentation.

IROS Conference 2023 Conference Paper

Semantically Informed MPC for Context-Aware Robot Exploration

• Yash Goel
• Narunas Vaskevicius
• Luigi Palmieri
• Nived Chebrolu
• Kai O. Arras
• Cyrill Stachniss

We investigate the task of object goal navigation in unknown environments where a target object is given as a semantic label (e. g. find a couch). This task is challenging as it requires the robot to consider the semantic context in diverse settings (e. g. TVs are often nearby couches). Most of the prior work tackles this problem under the assumption of a discrete action policy whereas we present an approach with continuous control which brings it closer to real world applications. In this paper, we use information-theoretic model predictive control on dense cost maps to bring object goal navigation closer to real robots with kinodynamic constraints. We propose a deep neural network framework to learn cost maps that encode semantic context and guide the robot towards the target object. We also present a novel way of fusing mid-level visual representations in our architecture to provide additional semantic cues for cost map prediction. The experiments show that our method leads to more efficient and accurate goal navigation with higher quality paths than the reported baselines. The results also indicate the importance of mid-level representations for navigation by improving the success rate by 8 percentage points.

ICRA Conference 2023 Conference Paper

SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations

• Xingguang Zhong
• Yue Pan 0009
• Jens Behley
• Cyrill Stachniss

Accurate mapping of large-scale environments is an essential building block of most outdoor autonomous systems. Challenges of traditional mapping methods include the balance between memory consumption and mapping accuracy. This paper addresses the problem of achieving large-scale 3D reconstruction using implicit representations built from 3D LiDAR measurements. We learn and store implicit features through an octree-based, hierarchical structure, which is sparse and extensible. The implicit features can be turned into signed distance values through a shallow neural network. We leverage binary cross entropy loss to optimize the local features with the 3D measurements as supervision. Based on our implicit representation, we design an incremental mapping system with regularization to tackle the issue of forgetting in continual learning. Our experiments show that our 3D reconstructions are more accurate, complete, and memory-efficient than current state-of-the-art 3D mapping methods.

ICRA Conference 2023 Conference Paper

Target-Aware Implicit Mapping for Agricultural Crop Inspection

• Shane Kelly
• Alessandro Riccardi
• Elias Marks
• Federico Magistri
• Tiziano Guadagnino
• Margarita Chli
• Cyrill Stachniss

Crop inspection is a critical part of modern agricultural practices that helps farmers assess the current status of a field and then make crop management decisions. Current crop inspection methods are labour-intensive tasks, which makes them rather slow and expensive to apply. In this paper, we exploit recent advancements in implicit mapping to tackle the challenging context of agricultural environments to create dense maps of crop rows with high enough fidelity to be useful for automated crop inspection. Specifically, we map strawberry and sweet pepper crop rows using RGB images captured by a wheeled mobile field robot inside a greenhouse and then use this data to build 3D maps to document the development of plants and fruits. Our Target-Aware Implicit Mapping system (TAIM) uses a SLAM-based pose initialization strategy for robust pose convergence, an efficient information-guided training sample selection framework for faster loss reduction, and focuses on exploiting training samples for fruit regions of the scene, which are critical for crop inspection tasks, to create more accurate maps in less time.

IROS Conference 2022 Conference Paper

ICK-Track: A Category-Level 6-DoF Pose Tracker Using Inter-Frame Consistent Keypoints for Aerial Manipulation

• Jingtao Sun
• Yaonan Wang 0001
• Mingtao Feng
• Danwei Wang
• Jiawen Zhao
• Cyrill Stachniss
• Xieyuanli Chen

Robots that are supposed to interact with or manipulate objects in the world must be able to track the poses of objects in their sensor data. Thus, Detecting and tracking the 6-DoF poses of targeted objects is important for aerial manipulation and is still in the early stage due to the high dynamics and limited onboard capacity of such systems. In this paper, we propose ICK-Track, a novel method for onboard category-level object 6-DoF pose tracking that can be applied to aerial manipulation without using any pre-defined object CAD models. It first utilizes a semi-supervised video segmentation to detect objects in the eye-in-hand RGB-D camera stream to segment the 3D points of objects. Then, canonical keypoints are extracted using iterative farthest point sampling. We propose a novel inter-frame consistent keypoints generation network to generate the corresponding keypoint pairs, which are used together with ICP to estimate the pose changes of objects for tracking. Experimental results show that our method is more robust to viewpoint changes and runs faster than the state-of-the-art methods on category-level pose tracking. We further test our proposed method on a real aerial manipulator. A demo video showing the use of our method on a real aerial manipulator and the implementation of our method are available at: https://github.com/S-JingTao/ICK-Track.

IROS Conference 2022 Conference Paper

Informative Path Planning for Active Learning in Aerial Semantic Mapping

• Julius Rückin
• Liren Jin
• Federico Magistri
• Cyrill Stachniss
• Marija Popovic

Semantic segmentation of aerial imagery is an important tool for mapping and earth observation. However, supervised deep learning models for segmentation rely on large amounts of high-quality labelled data, which is labour-intensive and time-consuming to generate. To address this, we propose a new approach for using unmanned aerial vehicles (UAVs) to autonomously collect useful data for model training. We exploit a Bayesian approach to estimate model uncertainty in semantic segmentation. During a mission, the semantic predictions and model uncertainty are used as input for terrain mapping. A key aspect of our pipeline is to link the mapped model uncertainty to a robotic planning objective based on active learning. This enables us to adaptively guide a UAV to gather the most informative terrain images to be labelled by a human for model training. Our experimental evaluation on real-world data shows the benefit of using our informative planning approach in comparison to static coverage paths in terms of maximising model performance and reducing labelling efforts.

IROS Conference 2022 Conference Paper

MD-SLAM: Multi-cue Direct SLAM

• Luca Di Giammarino
• Leonardo Brizi
• Tiziano Guadagnino
• Cyrill Stachniss
• Giorgio Grisetti

Simultaneous Localization and Mapping (SLAM) systems are fundamental building blocks for any autonomous robot navigating in unknown environments. The SLAM implementation heavily depends on the sensor modality employed on the mobile platform. For this reason, assumptions on the scene's structure are often made to maximize estimation accuracy. This paper presents a novel direct 3D SLAM pipeline that works independently for RGB-D and LiDAR sensors. Building upon prior work on multi-cue photometric frame-to-frame alignment [4], our proposed approach provides an easy-to-extend and generic SLAM system. Our pipeline requires only minor adaptations within the projection model to handle different sensor modalities. We couple a position tracking system with an appearance-based relocalization mechanism that handles large loop closures. Loop closures are validated by the same direct registration algorithm used for odometry estimation. We present comparative experiments with state-of-the-art approaches on publicly available benchmarks using RGB-D cameras and 3D LiDARs. Our system performs well in heterogeneous datasets compared to other sensor-specific methods while making no assumptions about the environment. Finally, we release an open-source C++ implementation of our system.

ICRA Conference 2022 Conference Paper

Precise 3D Reconstruction of Plants from UAV Imagery Combining Bundle Adjustment and Template Matching

• Elias Marks
• Federico Magistri
• Cyrill Stachniss

Monitoring individual plants and computing precise 3D reconstructions is highly relevant for crop breeding. In the conventional breeding approach, humans measure phenotypic traits by hand, requiring substantial manual labor. This paper addresses precise 3D plant reconstructions in a crop field or breeding plot based on UAV imagery. We explicitly address the challenges resulting from the thin structures of leaves and naturally occurring self-occlusions. We combine photogrammetric bundle adjustment with a template-based matching approach and produce accurate 3D models that allow us to derive common, geometric traits used by breeders to phenotype plants. We provide a thorough experimental evaluation on commercially used sugar beet breeding plots to illustrate the capabilities of our method as well as its real world applicability.

ICRA Conference 2022 Conference Paper

Retriever: Point Cloud Retrieval in Compressed 3D Maps

• Louis Wiesmann
• Rodrigo Marcuzzi
• Cyrill Stachniss
• Jens Behley

Most autonomous driving and robotic applications require retrieving map data around the vehicle's current location. Those maps can cover large areas and are often stored in a compressed form to save memory and allow for efficient transmission. In this paper, we address the problem of place recognition in a compressed point cloud map. To this end, we propose a novel deep neural network architecture that directly operates on a compressed feature representation produced by a compression encoder. This enables us to bypass compute-heavy decompression of the map and exploits the compact as well as descriptive nature of the compressed features. Additionally, we propose an alternative to the commonly used NetVLAD layer to aggregate local descriptors. Here, we utilize an attention mechanism between local features and a latent code. Our experiments suggest that this produces a more descriptive feature representation of the point clouds for place recognition. We experimentally validate all architectural choices we made by our ablation studies and compare our performance to other state-of-the-art baselines on two commonly used datasets.

IROS Conference 2022 Conference Paper

Robust Onboard Localization in Changing Environments Exploiting Text Spotting

• Nicky Zimmerman
• Louis Wiesmann
• Tiziano Guadagnino
• Thomas Läbe
• Jens Behley
• Cyrill Stachniss

Robust localization in a given map is a crucial component of most autonomous robots. In this paper, we address the problem of localizing in an indoor environment that changes and where prominent structures have no correspondence in the map built at a different point in time. To overcome the discrepancy between the map and the observed environment caused by such changes, we exploit human-readable localization cues to assist localization. These cues are readily available in most facilities and can be detected using RGB camera images by utilizing text spotting. We integrate these cues into a Monte Carlo localization framework using a particle filter that operates on 2D LiDAR scans and camera data. By this, we provide a robust localization solution for environments with structural changes and dynamics by humans walking. We evaluate our localization framework on multiple challenging indoor scenarios in an office environment. The experiments suggest that our approach is robust to structural changes and can run on an onboard computer. We release an open source implementation of our approach 1 1 https://github.com/PRBonn/tmcl, which uses off-the-shelf text spotting, written in C++ with a ROS wrapper.

IROS Conference 2022 Conference Paper

Voxfield: Non-Projective Signed Distance Fields for Online Planning and 3D Reconstruction

• Yue Pan 0009
• Yves Kompis
• Luca Bartolomei 0002
• Ruben Mascaro
• Cyrill Stachniss
• Margarita Chli

Creating accurate maps of complex, unknown environments is of utmost importance for truly autonomous navigation robot. However, building these maps online is far from trivial, especially when dealing with large amounts of raw sensor readings on a computation and energy constrained mobile system, such as a small drone. While numerous approaches tackling this problem have emerged in recent years, the mapping accuracy is often sacrificed as systematic approximation errors are tolerated for efficiency's sake. Motivated by these challenges, we propose Voxfield, a mapping framework that can generate maps online with higher accuracy and lower computational burden than the state of the art. Built upon the novel formulation of non-projective truncated signed distance fields (TSDFs), our approach produces more accurate and complete maps, suitable for surface reconstruction. Additionally, it enables efficient generation of Euclidean signed distance fields (ESDFs), useful e. g. , for path planning, that does not suffer from typical approximation errors. Through a series of experiments with public datasets, both real-world and synthetic, we demonstrate that our method beats the state of the art in map coverage, accuracy and computational time. Moreover, we show that Voxfield can be utilized as a back-end in recent multi-resolution mapping frameworks, producing high quality maps even in large-scale experiments. Finally, we validate our method by running it onboard a quadrotor, showing it can generate accurate ESDF maps usable for real-time path planning and obstacle avoidance.

ICRA Conference 2021 Conference Paper

A Benchmark for LiDAR-based Panoptic Segmentation based on KITTI

• Jens Behley
• Andres Milioto
• Cyrill Stachniss

Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly [18]. In this paper, we present an extension of SemanticKITTI [1], a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark [10]. This extension enables training and evaluation of LiDAR-based panoptic segmentation. We provide the data and discuss the processing steps needed to enrich a given semantic annotation with temporally consistent instance information, i. e. , instance information that supplements the semantic labels and identifies the same instance over sequences of LiDAR point clouds. Additionally, we present two strong baselines that combine state-of-the-art LiDAR-based semantic segmentation approaches with a state-of-the-art detector enriching the segmentation with instance information and that allow other researchers to compare their approaches against. We believe that our extension of SemanticKITTI with strong baselines enables the creation of novel algorithms for LiDAR-based panoptic segmentation as much as it has for the original semantic segmentation and semantic scene completion tasks. Data, code, and an online evaluation service using a hidden test set are publicly available at http://semantic-kitti.org.

IROS Conference 2021 Conference Paper

Efficient Localisation Using Images and OpenStreetMaps

• Mengjie Zhou
• Xieyuanli Chen
• Noe Samano
• Cyrill Stachniss
• Andrew Calway

The ability to localise is key for robot navigation. We describe an efficient method for vision-based localisation, which combines sequential Monte Carlo tracking with matching ground-level images to 2-D cartographic maps such as OpenStreetMaps. The matching is based on a learned embedded space representation linking images and map tiles, encoding the common semantic information present in both and providing potential for invariance to changing conditions. Moreover, the compactness of 2-D maps supports scalability. This contrasts with the majority of previous approaches based on matching with single-shot geo-referenced images or 3-D reconstructions. We present experiments using the StreetLearn and Oxford RobotCar datasets and demonstrate that the method is highly effective, giving high accuracy and fast convergence.

IROS Conference 2021 Conference Paper

Improving Monocular Depth Estimation by Semantic Pre-training

• Peter Rottmann
• Thorbjörn Posewsky
• Andres Milioto
• Cyrill Stachniss
• Jens Behley

Knowing the distance to nearby objects is crucial for autonomous cars to navigate safely in everyday traffic. In this paper, we investigate monocular depth estimation, which advanced substantially within the last years and is providing increasingly more accurate results while only requiring a single camera image as input. In line with recent work, we use an encoder-decoder structure with so-called packing layers to estimate depth values in a self-supervised fashion. We propose integrating a joint pre-training of semantic segmentation plus depth estimation on a dataset providing semantic labels. By using a separate semantic decoder that is only needed for pre-training, we can keep the network comparatively small. Our extensive experimental evaluation shows that the addition of such pre-training improves the depth estimation performance substantially. Finally, we show that we achieve competitive performance on the KITTI dataset despite using a much smaller and more efficient network.

IROS Conference 2021 Conference Paper

Maneuver-based Trajectory Prediction for Self-driving Cars Using Spatio-temporal Convolutional Networks

• Benedikt Mersch
• Thomas Höllen
• Kun Zhao
• Cyrill Stachniss
• Ribana Roscher

The ability to predict the future movements of other vehicles is a subconscious and effortless skill for humans and key to safe autonomous driving. Therefore, trajectory prediction for autonomous cars has gained a lot of attention in recent years. It is, however, still a hard task to achieve human-level performance. Interdependencies between vehicle behaviors and the multimodal nature of future intentions in a dynamic and complex driving environment render trajectory prediction a challenging problem. In this work, we propose a new, datadriven approach for predicting the motion of vehicles in a road environment. The model allows for inferring future intentions from the past interaction among vehicles in highway driving scenarios. Using our neighborhood-based data representation, the proposed system jointly exploits correlations in the spatial and temporal domain using convolutional neural networks. Our system considers multiple possible maneuver intentions and their corresponding motion and predicts the trajectory for five seconds into the future. We implemented our approach and evaluated it on two highway datasets taken in different countries and are able to achieve a competitive prediction performance.

ICRA Conference 2021 Conference Paper

Poisson Surface Reconstruction for LiDAR Odometry and Mapping

• Ignacio Vizzo
• Xieyuanli Chen
• Nived Chebrolu
• Jens Behley
• Cyrill Stachniss

Accurately localizing in and mapping an environment are essential building blocks of most autonomous systems. In this paper, we present a novel approach for LiDAR odometry and mapping, focusing on improving the mapping quality and at the same time estimating the pose of the vehicle. Our approach performs frame-to-mesh ICP, but in contrast to other SLAM approaches, we represent the map as a triangle mesh computed via Poisson surface reconstruction. We perform the surface reconstruction in a sliding window fashion over a sequence of past scans. In this way, we obtain accurate local maps that are well suited for registration and can also be combined into a global map. This enables us to build a 3D map showing more geometric details than common mapping approaches relying on a truncated signed distance function or surfels. Our experimental evaluation shows quantitatively and qualitatively that our maps offer higher geometric accuracies than these other map representations. We also show that our maps are compact and can be used for LiDAR-based odometry estimation with a novel ray-casting-based data association.

ICRA Conference 2021 Conference Paper

Range Image-based LiDAR Localization for Autonomous Vehicles

• Xieyuanli Chen
• Ignacio Vizzo
• Thomas Läbe
• Jens Behley
• Cyrill Stachniss

Robust and accurate, map-based localization is crucial for autonomous mobile systems. In this paper, we exploit range images generated from 3D LiDAR scans to address the problem of localizing mobile robots or autonomous cars in a map of a large-scale outdoor environment represented by a triangular mesh. We use the Poisson surface reconstruction to generate the mesh-based map representation. Based on the range images generated from the current LiDAR scan and the synthetic rendered views from the mesh-based map, we propose a new observation model and integrate it into a Monte Carlo localization framework, which achieves better localization performance and generalizes well to different environments. We test the proposed localization approach on multiple datasets collected in different environments with different LiDAR scanners. The experimental results show that our method can reliably and accurately localize a mobile system in different environments and operate online at the LiDAR sensor frame rate to track the vehicle pose.

ICRA Conference 2021 Conference Paper

Simple But Effective Redundant Odometry for Autonomous Vehicles

• Andrzej Reinke
• Xieyuanli Chen
• Cyrill Stachniss

Robust and reliable ego-motion is a key component of most autonomous mobile systems. Many odometry estimation methods have been developed using different sensors such as cameras or LiDARs. In this work, we present a resilient approach that exploits the redundancy of multiple odometry algorithms using a 3D LiDAR scanner and a monocular camera to provide reliable state estimation for autonomous vehicles. Our system utilizes a stack of odometry algorithms that run in parallel. It chooses from them the most promising pose estimation considering sanity checks using dynamic and kinematic constraints of the vehicle as well as a score computed between the current LiDAR scan and a locally built point cloud map. In this way, our method can exploit the advantages of different existing ego-motion estimating approaches. We evaluate our method on the KITTI Odometry dataset. The experimental results suggest that our approach is resilient to failure cases and achieves an overall better performance than individual odometry methods employed by our system.

ICRA Conference 2021 Conference Paper

Towards In-Field Phenotyping Exploiting Differentiable Rendering with Self-Consistency Loss

• Federico Magistri
• Nived Chebrolu
• Jens Behley
• Cyrill Stachniss

In modern agriculture, measuring phenotypic traits helps breeders monitor plant growth, increase yield, and provide food, feed, and fiber. Traditional phenotyping requires intensive manual work, partially being intrusive. In this paper, we investigate the challenge of measuring phenotypic traits in an automated fashion through mobile robots operating in field environments. In particular, we want to measure plants from images acquired by mobile robots instead of using data from a static scanning environment. We propose to use a differentiable rendering approach to deform a generic 3D template of a plant to fit the observation recorded by a robot while ensuring a coherent deformation of the plant template. The experiments presented in this paper suggest that our approach allows for 3D reconstruction of different plant species at different growth stages using single images. From that model, we can compute important phenotypic traits, such as the leaf area index.

IROS Conference 2021 Conference Paper

Visual Place Recognition using LiDAR Intensity Information

• Luca Di Giammarino
• Irvin Aloise
• Cyrill Stachniss
• Giorgio Grisetti

Robots and autonomous systems need to know where they are within a map to navigate effectively. Thus, simultaneous localization and mapping or SLAM is a common building block of robot navigation systems. When building a map via a SLAM system, robots need to re-recognize places to find loop closure and reduce the odometry drift. Image-based place recognition received a lot of attention in computer vision, and in this work, we investigate how such approaches can be used for 3D LiDAR data. Recent LiDAR sensors produce high-resolution 3D scans in combination with comparably stable intensity measurements. Through a cylindrical projection, we can turn this information into a 360° panoramic range image. As a result, we can apply techniques from visual place recognition to LiDAR intensity data. The question of how well this approach works in practice has only partially been investigated. This paper provides an analysis of how such visual techniques can be with LiDAR data, and we provide an evaluation on different datasets. Our results suggest that this form of place recognition is possible and an effective means for determining loop closures.

ICRA Conference 2020 Conference Paper

Beyond Photometric Consistency: Gradient-based Dissimilarity for Improving Visual Odometry and Stereo Matching

• Jan Quenzel
• Radu Alexandru Rosu
• Thomas Läbe
• Cyrill Stachniss
• Sven Behnke

Pose estimation and map building are central ingredients of autonomous robots and typically rely on the registration of sensor data. In this paper, we investigate a new metric for registering images that builds upon on the idea of the photometric error. Our approach combines a gradient orientation-based metric with a magnitude-dependent scaling term. We integrate both into stereo estimation as well as visual odometry systems and show clear benefits for typical disparity and direct image registration tasks when using our proposed metric. Our experimental evaluation indicate that our metric leads to more robust and more accurate estimates of the scene depth as well as camera trajectory. Thus, the metric improves camera pose estimation and in turn the mapping capabilities of mobile robots. We believe that a series of existing visual odometry and visual SLAM systems can benefit from the findings reported in this paper.

IROS Conference 2020 Conference Paper

Domain Transfer for Semantic Segmentation of LiDAR Data using Deep Neural Networks

• Ferdinand Langer
• Andres Milioto
• Alexandre Haag
• Jens Behley
• Cyrill Stachniss

Inferring semantic information towards an understanding of the surrounding environment is crucial for autonomous vehicles to drive safely. Deep learning-based segmentation methods can infer semantic information directly from laser range data, even in the absence of other sensor modalities such as cameras. In this paper, we address improving the generalization capabilities of such deep learning models to range data that was captured using a different sensor and in situations where no labeled data is available for the new sensor setup. Our approach assists the domain transfer of a LiDAR-only semantic segmentation model to a different sensor and environment exploiting existing geometric mapping systems. To this end, we fuse sequential scans in the source dataset into a dense mesh and render semi-synthetic scans that match those of the target sensor setup. Unlike simulation, this approach provides a real-to-real transfer of geometric information and delivers additionally more accurate remission information. We implemented and thoroughly tested our approach by transferring semantic scans between two different real-world datasets with different sensor setups. Our experiments show that we can improve the segmentation performance substantially with zero manual re-labeling. This approach solves the number one feature request since we released our semantic segmentation library LiDAR-bonnetal [18].

ICRA Conference 2020 Conference Paper

Gradient and Log-based Active Learning for Semantic Segmentation of Crop and Weed for Agricultural Robots

• Rasha Sheikh
• Andres Milioto
• Philipp Lottes
• Cyrill Stachniss
• Maren Bennewitz
• Thomas Schultz 0001

Annotated datasets are essential for supervised learning. However, annotating large datasets is a tedious and time-intensive task. This paper addresses active learning in the context of semantic segmentation with the goal of reducing the human labeling effort. Our application is agricultural robotics and we focus on the task of distinguishing between crop and weed plants from image data. A key challenge in this application is the transfer of an existing semantic segmentation CNN to a new field, in which growth stage, weeds, soil, and weather conditions differ. We propose a novel approach that, given a trained model on one field together with rough foreground segmentation, refines the network on a substantially different field providing an effective method of selecting samples to annotate for supporting the transfer. We evaluated our approach on two challenging datasets from the agricultural robotics domain and show that we achieve a higher accuracy with a smaller number of samples compared to random sampling as well as entropy based sampling, which consequently reduces the required human labeling effort.

IROS Conference 2020 Conference Paper

Learning an Overlap-based Observation Model for 3D LiDAR Localization

• Xieyuanli Chen
• Thomas Läbe
• Lorenzo Nardi
• Jens Behley
• Cyrill Stachniss

Localization is a crucial capability for mobile robots and autonomous cars. In this paper, we address learning an observation model for Monte-Carlo localization using 3D LiDAR data. We propose a novel, neural network-based observation model that computes the expected overlap of two 3D LiDAR scans. The model predicts the overlap and yaw angle offset between the current sensor reading and virtual frames generated from a pre-built map. We integrate this observation model into a Monte-Carlo localization framework and tested it on urban datasets collected with a car in different seasons. The experiments presented in this paper illustrate that our method can reliably localize a vehicle in typical urban environments. We furthermore provide comparisons to a beam-endpoint and a histogram-based method indicating a superior global localization performance of our method with fewer particles.

IROS Conference 2020 Conference Paper

LiDAR Panoptic Segmentation for Autonomous Driving

• Andres Milioto
• Jens Behley
• Chris McCool
• Cyrill Stachniss

Truly autonomous driving without the need for human intervention can only be attained when self-driving cars fully understand their surroundings. Most of these vehicles rely on a suite of active and passive sensors. LiDAR sensors are a cornerstone in most of these hardware stacks, and leveraging them as a complement to other passive sensors such as RGB cameras is an enticing goal. Understanding the semantic class of each point in a LiDAR sweep is important, as well as knowing to which instance of that class it belongs to. To this end, we present a novel, single-stage, and real-time capable panoptic segmentation approach using a shared encoder with a semantic and instance decoder. We leverage the geometric information of the LiDAR scan to perform a novel, distance- aware tri-linear upsampling, which allows our approach to use larger output strides than using transpose convolutions leading to substantial savings in computation time. Our experimental evaluation and ablation studies for each module show that combining our geometric and semantic embeddings with our learned, variable instance thresholds, a category-specific loss, and the novel trilinear upsampling module leads to higher panoptic quality. We will release the code of our approach in our LiDAR processing library LiDAR-Bonnetal [27].

ICRA Conference 2020 Conference Paper

Long-Term Robot Navigation in Indoor Environments Estimating Patterns in Traversability Changes

• Lorenzo Nardi
• Cyrill Stachniss

Nowadays, mobile robots are deployed in many indoor environments such as offices or hospitals. These environments are subject to changes in the traversability that often happen following patterns. In this paper, we investigate the problem of navigating in such environments over extended periods of time by capturing and exploiting these patterns to make informed decisions for navigation. Our approach uses a probabilistic graphical model to incrementally estimate a model of the traversability changes from the robot's observations and to make predictions at currently unobserved locations. In the belief space defined by the predictions, we plan paths that trade off the risk to encounter obstacles and the information gain of visiting unknown locations. We implemented our approach and tested it in different indoor environments. The experiments suggest that, in the long run, our approach leads robots to navigate along shorter paths compared to following a greedy shortest path policy.

IROS Conference 2020 Conference Paper

Segmentation-Based 4D Registration of Plants Point Clouds for Phenotyping

• Federico Magistri
• Nived Chebrolu
• Cyrill Stachniss

Plant phenotyping, i. e. , the task of measuring plant traits to describe the anatomy and physiology of plants, is a central task in crop science and plant breeding. Standard methods often require intrusive or time-consuming operations involving a lot of manual labor. Cameras or range sensors, paired with 3D reconstructions methods, can support phenotyping but the task yields several challenges in practice such as plant growth over time. In this paper, we address the problem of finding correspondences between plants recorded at different points in time to track phenotypic traits in an automated fashion. Our approach makes use of semantic segmentation and unsupervised clustering to compute keypoints from plant point clouds. We extract a compact representation of the considered scan that encodes both, topology and semantic information. Through our approach, we are able to tackle the data association problem for 4D point cloud data of plants effectively. We tested our approach on different 3D plus time, i. e. , 4D, sequences of plant point clouds of different plant species. The experiments presented in this paper suggest that our 4D matching approach allows for non-rigid registration of the plants that change over time. Moreover, we show that our method allows for tracking different phenotyping traits at an organ level, forming a basis for automated temporal phenotyping.

ICRA Conference 2020 Conference Paper

Spatio-Temporal Non-Rigid Registration of 3D Point Clouds of Plants

• Nived Chebrolu
• Thomas Läbe
• Cyrill Stachniss

Analyzing sensor data of plants and monitoring plant performance is a central element in different agricultural robotics applications. In plant science, phenotyping refers to analyzing plant traits for monitoring growth, for describing plant properties, or characterizing the plant's overall performance. It plays a critical role in the agricultural tasks and in plant breeding. Recently, there is a rising interest in using 3D data obtained from laser scanners and 3D cameras to develop automated non-intrusive techniques for estimating plant traits. In this paper, we address the problem of registering 3D point clouds of the plants over time, which is a backbone of applications interested in tracking spatio-temporal traits of individual plants. Registering plants over time is challenging due to its changing topology, anisotropic growth, and non-rigid motion in between scans. We propose a novel approach that exploits the skeletal structure of the plant and determines correspondences over time and drives the registration process. Our approach explicitly accounts for the non-rigidity and the growth of the plant over time in the registration. We tested our approach on a challenging dataset acquired over the course of two weeks and successfully registered the 3D plant point clouds recorded with a laser scanner forming a basis for developing systems for automated temporal plant-trait analysis.

IROS Conference 2020 Conference Paper

Unsupervised Domain Adaptation for Transferring Plant Classification Systems to New Field Environments, Crops, and Robots

• Dario Gogoll
• Philipp Lottes
• Jan Weyler
• Nik Petrinic
• Cyrill Stachniss

Crops are an important source of food and other products. In conventional farming, tractors apply large amounts of agrochemicals uniformly across fields for weed control and plant protection. Autonomous farming robots have the potential to provide environment-friendly weed control on a per plant basis. A system that reliably distinguishes crops, weeds, and soil under varying environment conditions is the basis for plant-specific interventions such as spot applications. Such semantic segmentation systems, however, often show a performance decay when applied under new field conditions. In this paper, we therefore propose an effective approach to unsupervised domain adaptation for plant segmentation systems in agriculture and thus to adapt existing systems to new environments, different value crops, and other farm robots. Our system yields a high segmentation performance in the target domain by exploiting labels only from the source domain. It is based on CycleGANs and enforces a semantic consistency domain transfer by constraining the images to be pixel-wise classified in the same way before and after translation. We perform an extensive evaluation, which indicates that we can substantially improve the transfer of our semantic segmentation system to new field environments, different crops, and different sensors or robots.

ICRA Conference 2020 Conference Paper

Visual Servoing-based Navigation for Monitoring Row-Crop Fields

• Alireza Ahmadi
• Lorenzo Nardi
• Nived Chebrolu
• Cyrill Stachniss

Autonomous navigation is a pre-requisite for field robots to carry out precision agriculture tasks. Typically, a robot has to navigate along a crop field multiple times during a season for monitoring the plants, for applying agrochemicals, or for performing targeted interventions. In this paper, we propose a visual-based navigation framework tailored to row-crop fields that exploits the regular crop-row structure present in fields. Our approach uses only the images from on-board cameras without the need for performing explicit localization or maintaining a map of the field. Thus, it can operate without expensive RTK-GPS solutions often used in agricultural automation systems. Our navigation approach allows the robot to follow the crop rows accurately and handles the switch to the next row seamlessly within the same framework. We implemented our approach using C++ and ROS and thoroughly tested it in several simulated fields with different shapes and sizes. We also demonstrated the system running at frame-rate on an actual robot operating on a test row-crop field. The code and data have been published.

ICRA Conference 2019 Conference Paper

Accurate Direct Visual-Laser Odometry with Explicit Occlusion Handling and Plane Detection

• Kaihong Huang
• Junhao Xiao 0001
• Cyrill Stachniss

In this paper, we address the problem of combining 3D laser scanner and camera information to estimate the motion of a mobile platform. We propose a direct laser-visual odometry approach building upon photometric image alignment. Our approach is designed to maximize the information usage of both, the image and the laser scan, to compute an accurate frame-to-frame motion estimate. To deal with the sparsity of the range measurements, our approach identifies planar point sets within individual point clouds and subsequently extract their corresponding pixel patches from the camera image. The extracted planar image patches are used together with the non-planar pixels to estimate the frame-to-frame motion using a homography formulation capable of incorporating both types of pixel alignments. To achieve high estimation accuracy, we explicitly predict possible occlusions caused by observations taken from different locations. We evaluate our proposed approach using the KITTI dataset as well as data recorded with a Clearpath Husky platform. The experiments suggest that our approach can achieve competitive estimation accuracy and produce consistently registered, colored point clouds.

ICRA Conference 2019 Conference Paper

Actively Improving Robot Navigation On Different Terrains Using Gaussian Process Mixture Models

• Lorenzo Nardi
• Cyrill Stachniss

Robot navigation in outdoor environments is exposed to detrimental factors such as vibrations or power consumption due to the different terrains on which the robot navigates. In this paper, we address the problem of actively improving navigation by planning paths that aim at reducing over time phenomena such as vibrations during traversal. Our approach uses a Gaussian Process (GP) mixture model and an aerial image of the environment to learn and improve continuously a place-dependent model of such phenomena from the experiences of the robot. We use this model to plan paths that trade-off the exploration of unknown promising regions and the exploitation of known areas where the impact of the detrimental factors on navigation is low, leading to an improved navigation over time. We implemented our approach and thoroughly tested it using real-world data. Our experiments suggest that our approach with no initial information leads the robot, after few runs, to follow paths along which it experiences similar vibrations or energy consumption as if it was following the optimal path computed given the ground truth information.

ICRA Conference 2019 Conference Paper

Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs

• Andres Milioto
• Cyrill Stachniss

The ability to interpret a scene is an important capability for a robot that is supposed to interact with its environment. The knowledge of what is in front of the robot is, for example, relevant for navigation, manipulation, or planning. Semantic segmentation labels each pixel of an image with a class label and thus provides a detailed semantic annotation of the surroundings to the robot. Convolutional neural networks (CNNs) are popular methods for addressing this type of problem. The available software for training and the integration of CNNs for real robots, however, is quite fragmented and often difficult to use for non-experts, despite the availability of several high-quality open-source frameworks for neural network implementation and training. In this paper, we propose a tool called Bonnet, which addresses this fragmentation problem by building a higher abstraction that is specific for the semantic segmentation task. It provides a modular approach to simplify the training of a semantic segmentation CNN independently of the used dataset and the intended task. Furthermore, we also address the deployment on a real robotic platform. Thus, we do not propose a new CNN approach in this paper. Instead, we provide a stable and easy-to-use tool to make this technology more approachable in the context of autonomous systems. In this sense, we aim at closing a gap between computer vision research and its use in robotics research. We provide an open-source codebase for training and deployment. The training interface is implemented in Python using TensorFlow and the deployment interface provides C++ library that can be easily integrated in an existing robotics codebase, a ROS node, and two standalone applications for label prediction in images and videos.

ICRA Conference 2019 Conference Paper

Coverage Path Planning in Belief Space

• Robert Schirmer
• Peter Biber
• Cyrill Stachniss

For safety reasons, robotic lawn mowers and similar devices are required to stay within a predefined working area. Keeping the robot within its workspace is typically achieved by special safeguards such as a wire installed in the ground. In the case of robotic lawn mowers, this causes a certain customer reluctance. It is more desirable to fulfill those safety-critical tasks by safe navigation and path planning. In this paper, we tackle the problem of planning a coverage path composed of parallel lanes that maximizes robot safety under the constraints of cheap, low range sensors and thus substantial uncertainty in the robot's belief and ability to execute actions. Our approach uses a map of the environment to estimate localizability at all locations, and it uses these estimates to search for an uncertainty-aware coverage path while avoiding collisions. We implemented our approach using C++ and ROS and thoroughly tested it on real garden data. The experiment shows that our approach leads to safer meander patterns for the lawn mower and takes expected localizability information into account.

ICRA Conference 2019 Conference Paper

Fast Instance and Semantic Segmentation Exploiting Local Connectivity, Metric Learning, and One-Shot Detection for Robotics

• Andres Milioto
• Leonard P. Mandtler
• Cyrill Stachniss

Semantic scene understanding is important for autonomous robots that aim to navigate dynamic environments, manipulate objects, or interact with humans in a natural way. In this paper, we address the problem of jointly performing semantic segmentation as well as instance segmentation in an online fashion, so that autonomous robots can use this information on-the-go and without sacrificing accuracy. We achieve this by exploiting a local connectivity prior of objects in the real world and a multi-task convolutional neural network architecture. The network identifies the individual object instances and their classes without region proposals or pre-segmentation of the images into individual classes. We implemented and thoroughly evaluated our approach, and our experiments suggest that our method can be used to accurately segment instance masks of objects and identify their class in an online fashion.

ICRA Conference 2019 Conference Paper

Localization with Sliding Window Factor Graphs on Third-Party Maps for Automated Driving

• Daniel Wilbers
• Christian Merfels
• Cyrill Stachniss

Localizing a vehicle in a map is essential for automated driving and various other robotic applications. This paper addresses the problem of vehicle localization in urban environments. Our approach performs a graph-based sliding window optimization over a set of recent landmark and odometry measurements for fast and accurate vehicle localization on third-party maps. Our work incorporates landmark priors from third-party maps into the estimation problem and shows how to exploit the sliding window formulation for revising data associations. We describe how to construct our factor graph and derive its necessary factors to model the information from the map as a prior over the landmark detections. We implemented our approach on an automated car and thoroughly tested it on real-world data. The experiments suggest that the approach provides highly accurate pose estimates, is fast enough for automated driving applications, and outperforms localization using particle filters.

IROS Conference 2019 Conference Paper

RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation

• Andres Milioto
• Ignacio Vizzo
• Jens Behley
• Cyrill Stachniss

Perception in autonomous vehicles is often carried out through a suite of different sensing modalities. Given the massive amount of openly available labeled RGB data and the advent of high-quality deep learning algorithms for image-based recognition, high-level semantic perception tasks are pre-dominantly solved using high-resolution cameras. As a result of that, other sensor modalities potentially useful for this task are often ignored. In this paper, we push the state of the art in LiDAR-only semantic segmentation forward in order to provide another independent source of semantic information to the vehicle. Our approach can accurately perform full semantic segmentation of LiDAR point clouds at sensor frame rate. We exploit range images as an intermediate representation in combination with a Convolutional Neural Network (CNN) exploiting the rotating LiDAR sensor model. To obtain accurate results, we propose a novel post-processing algorithm that deals with problems arising from this intermediate representation such as discretization errors and blurry CNN outputs. We implemented and thoroughly evaluated our approach including several comparisons to the state of the art. Our experiments show that our approach outperforms state-of-the-art approaches, while still running online on a single embedded GPU. The code can be accessed at https://github.com/PRBonn/lidar-bonnetal.

IROS Conference 2019 Conference Paper

ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals

• Emanuele Palazzolo
• Jens Behley
• Philipp Lottes
• Philippe Giguère
• Cyrill Stachniss

Mapping and localization are essential capabilities of robotic systems. Although the majority of mapping systems focus on static environments, the deployment in real-world situations requires them to handle dynamic objects. In this paper, we propose an approach for an RGB-D sensor that is able to consistently map scenes containing multiple dynamic elements. For localization and mapping, we employ an efficient direct tracking on the truncated signed distance function (TSDF) and leverage color information encoded in the TSDF to estimate the pose of the sensor. The TSDF is efficiently represented using voxel hashing, with most computations parallelized on a GPU. For detecting dynamics, we exploit the residuals obtained after an initial registration, together with the explicit modeling of free space in the model. We evaluate our approach on existing datasets, and provide a new dataset showing highly dynamic scenes. These experiments show that our approach often surpass other state-of-the-art dense SLAM methods. We make available our dataset with the ground truth for both the trajectory of the RGB-D sensor obtained by a motion capture system and the model of the static environment using a high-precision terrestrial laser scanner. Finally, we release our approach as open source code.

ICRA Conference 2019 Conference Paper

Robot Localization Based on Aerial Images for Precision Agriculture Tasks in Crop Fields

• Nived Chebrolu
• Philipp Lottes
• Thomas Läbe
• Cyrill Stachniss

Localization is a pre-requisite for most autonomous robots. For example, to carry out precision agriculture tasks effectively, a robot must be able to localize itself accurately in crop fields. The crop field environment presents unique challenges such as the highly repetitive structure of the crops leading to visual aliasing as well as the continuously changing appearance of the field, which makes it difficult to localize over time. In this paper, we present a localization system, which uses an aerial map of the field and exploits the semantic information of the crops, weeds, and their stem positions to resolve the visual ambiguity problem and to enable robot localization over extended periods of time. We evaluate our approach on a real field over multiple sessions spanning several weeks. Experiments suggest that our approach provides the necessary accuracy required by precision agriculture applications and works in cases where current techniques using typical visual features tend to fail.

IROS Conference 2019 Conference Paper

SuMa++: Efficient LiDAR-based Semantic SLAM

• Xieyuanli Chen
• Andres Milioto
• Emanuele Palazzolo
• Philippe Giguère
• Jens Behley
• Cyrill Stachniss

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfel-based mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

ICRA Conference 2019 Conference Paper

Uncertainty-Aware Path Planning for Navigation on Road Networks Using Augmented MDPs

• Lorenzo Nardi
• Cyrill Stachniss

Although most robots use probabilistic algorithms to solve state estimation problems, path planning is often performed without considering the uncertainty about the robot's position. Uncertainty, however, matters in planning, but considering it often leads to computationally expensive algorithms. In this paper, we investigate the problem of path planning considering the uncertainty in the robot's belief about the world, in its perceptions and in its action execution. We propose the use of an uncertainty-augmented Markov Decision Process to approximate the underlying Partially Observable Markov Decision Process, and we employ a localization prior to estimate how the belief about the robot's position propagates through the environment. This yields to a planning approach that generates navigation policies able to make decisions according to the degree of uncertainty while being computationally tractable. We implemented our approach and thoroughly evaluated it on different navigation problems. Our experiments suggest that we are able to compute policies that are more effective than approaches that ignore the uncertainty, and that also outperform policies that always take the safest actions.

ICRA Conference 2018 Conference Paper

A General Framework for Flexible Multi-Cue Photometric Point Cloud Registration

• Bartolomeo Della Corte
• Igor Bogoslavskyi
• Cyrill Stachniss
• Giorgio Grisetti

The ability to build maps is a key functionality for the majority of mobile robots. A central ingredient to most mapping systems is the registration or alignment of the recorded sensor data. In this paper, we present a general methodology for photometric registration that can deal with multiple different cues. We provide examples for registering RGBD as well as 3D LIDAR data. In contrast to popular point cloud registration approaches such as ICP our method does not rely on explicit data association and exploits multiple modalities such as raw range and image data streams. Color, depth, and normal information are handled in an uniform manner and the registration is obtained by minimizing the pixel-wise difference between two multi-channel images. We developed a flexible and general framework and implemented our approach inside that framework. We also released our implementation as open source C++ code. The experiments show that our approach allows for an accurate registration of the sensor data without requiring an explicit data association or model-specific adaptations to datasets or sensors. Our approach exploits the different cues in a natural and consistent way and the registration can be done at framerate for a typical range or imaging sensor.

ICRA Conference 2018 Conference Paper

Fast Image-Based Geometric Change Detection Given a 3D Model

• Emanuele Palazzolo
• Cyrill Stachniss

3D models of the environment are used in numerous robotic applications and should reflect the current state of the world. In this paper, we address the problem of quickly finding structural changes between the current state of the world and a given 3D model using a small number of images. Our approach finds inconsistencies between pairs of images by re-projecting an image onto another one by passing through the given 3D model. This process leads to ambiguities, which we resolve by combining multiple images such that the 3D location of the change can be estimated. A focus of our approach is that it can be executed fast enough to allow the operation on a mobile system. We implemented our approach in C++ and released it as open source software. We tested it on existing datasets as well as on self-recorded image sequences and 3D models, which we publicly share. Our experiments show that our method quickly finds changes in the geometry of a scene.

IROS Conference 2018 Conference Paper

Joint Ego-motion Estimation Using a Laser Scanner and a Monocular Camera Through Relative Orientation Estimation and 1-DoF ICP

• Kaihong Huang
• Cyrill Stachniss

Pose estimation and mapping are key capabilities of most autonomous vehicles and thus a number of localization and SLAM algorithms have been developed in the past. Autonomous robots and cars are typically equipped with multiple sensors. Often, the sensor suite includes a camera and a laser range finder. In this paper, we consider the problem of incremental ego-motion estimation, using both, a monocular camera and a laser range finder jointly. We propose a new algorithm, that exploits the advantages of both sensors-the ability of cameras to determine orientations well and the ability of laser range finders to estimate the scale and to directly obtain 3D point clouds. Our approach estimates the 5 degrees of freedom relative orientation from image pairs through feature point correspondences and formulates the remaining scale estimation as a new variant of the iterative closest point problem with only one degree of freedom. We furthermore exploit the camera information in a new way to constrain the data association between laser point clouds. The experiments presented in this paper suggest that our approach is able to accurately estimate the ego-motion of a vehicle and that we obtain more accurate frame-to-frame alignments than with one sensor modality alone.

IROS Conference 2018 Conference Paper

Joint Stem Detection and Crop-Weed Classification for Plant-Specific Treatment in Precision Farming

• Philipp Lottes
• Jens Behley
• Nived Chebrolu
• Andres Milioto
• Cyrill Stachniss

Applying agrochemicals is the default procedure for conventional weed control in crop production, but has negative impacts on the environment. Robots have the potential to treat every plant in the field individually and thus can reduce the required use of such chemicals. To achieve that, robots need the ability to identify crops and weeds in the field and must additionally select effective treatments. While certain types of weed can be treated mechanically, other types need to be treated by (selective) spraying. In this paper, we present an approach that provides the necessary information for effective plant-specific treatment. It outputs the stem location for weeds, which allows for mechanical treatments, and the covered area of the weed for selective spraying. Our approach uses an end-to-end trainable fully convolutional network that simultaneously estimates stem positions as well as the covered area of crops and weeds. It jointly learns the class-wise stem detection and the pixel-wise semantic segmentation. Experimental evaluations on different real-world datasets show that our approach is able to reliably solve this problem. Compared to state-of-the-art approaches, our approach not only substantially improves the stem detection accuracy, i. e. , distinguishing crop and weed stems, but also provides an improvement in the semantic segmentation performance.

ICRA Conference 2018 Conference Paper

On Geometric Models and Their Accuracy for Extrinsic Sensor Calibration

• Kaihong Huang
• Cyrill Stachniss

Extrinsic sensor calibration is an important task in robotics. There are various ways to perform the calibration task, but it often remains unclear which methods are better than the others. In this paper, we provide a systematic study about the calibration accuracy of three types of calibration methods, each represented by an abstract geometric model based on the sensor configuration and the calibration setup. We discuss the advantages and disadvantages of each model and perform a rigorous study on their noise sensitivity from a geometric perspective. As a result, we can reveal and quantify the relative calibration accuracies of the three models, thus answering the question of “which model is better and why? ”. Beside our analytical analysis, we also provide numerical simulation experiments that validate our findings.

ICRA Conference 2018 Conference Paper

Real-Time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs

• Andres Milioto
• Philipp Lottes
• Cyrill Stachniss

Precision farming robots, which target to reduce the amount of herbicides that need to be brought out in the fields, must have the ability to identify crops and weeds in real time to trigger weeding actions. In this paper, we address the problem of CNN-based semantic segmentation of crop fields separating sugar beet plants, weeds, and background solely based on RGB data. We propose a CNN that exploits existing vegetation indexes and provides a classification in real time. Furthermore, it can be effectively re-trained to so far unseen fields with a comparably small amount of training data. We implemented and thoroughly evaluated our system on a real agricultural robot operating in different fields in Germany and Switzerland. The results show that our system generalizes well, can operate at around 20 Hz, and is suitable for online operation in the fields.

IROS Conference 2017 Conference Paper

Analyzing the quality of matched 3D point clouds of objects

• Igor Bogoslavskyi
• Cyrill Stachniss

3D laser scanners are frequently used sensors for mobile robots or autonomous cars and they are often used to perceive the static as well as dynamic aspects in the scene. In this context, matching 3D point clouds of objects is a crucial capability. Most matching methods such as numerous flavors of ICP provide little information about the quality of the match, i. e. how well do the matched objects correspond to each other, which goes beyond point-to-point or point-to-plane distances. In this paper, we propose a projective method that yields a probabilistic measure for the quality of matched scans. It not only considers the differences in the point locations but can also take free-space information into account. Our approach provides a probabilistic measure that is meaningful enough to evaluate scans and to cluster real-world data such as scans taken with Velodyne scanner in urban scenes in an unsupervised manner.

IROS Conference 2017 Conference Paper

Efficient path planning in belief space for safe navigation

• Robert Schirmer
• Peter Biber
• Cyrill Stachniss

Robotic lawn-mowers are required to stay within a predefined working area, otherwise they may drive into a pond or on the street. This turns navigation and path planning into safety critical components. If we consider using SLAM techniques in that context, we must be able to provide safety guarantees in the presence of sensor/actuator noise and featureless areas in the environment. In this paper, we tackle the problem of planning a path that maximizes robot safety while navigating inside the working area and under the constraints of limited computing resources and cheap sensors. Our approach uses a map of the environment to estimate localizability at all locations, and it uses these estimates to search for a path from start to goal in belief space using an extended heuristic search algorithm. We implemented our approach using C++ and ROS and thoroughly tested it on simulation data recorded on eight different gardens, as well as on a real robot. The experiments presented in this paper show that our approach leads to short computation times and short paths while maximizing robot safety under certain assumptions.

IROS Conference 2017 Conference Paper

Extrinsic multi-sensor calibration for mobile robots using the Gauss-Helmert model

• Kaihong Huang
• Cyrill Stachniss

Most state estimation procedures in mobile robotics require information about the locations of the individual sensors on the platform. In this paper, we study the motion-based multi-sensor extrinsic calibration problem and point out an overlooked defect of traditional least squares estimation in this context. We present a novel calibration approach based-on the Gauss-Helmert estimation paradigm, together with a formulation for multi-sensor motion constraint. Our approach estimates not only the extrinsic parameters but also the pose observation errors, thus recovering the underlying sensor movements that exactly fulfill the motion constraints. Compared to traditional least squares approaches that estimate only the parameters, our approach is statistically optimal, thus is more accurate and robust. We implemented our approach and tested it on real robot. The experiments show that our approach is able to accurately determine the extrinsic configuration of each sensor and can largely improve the accuracy when the noise level is high.

IROS Conference 2017 Conference Paper

Semi-supervised online visual crop and weed classification in precision farming exploiting plant arrangement

• Philipp Lottes
• Cyrill Stachniss

Precision farming robots offer a great potential for reducing the amount of agro-chemicals that is required in the fields through a targeted, per-plant intervention. To achieve this, robots must be able to reliably distinguish crops from weeds on different fields and across growth stages. In this paper, we tackle the problem of separating crops from weeds reliably while requiring only a minimal amount of training data through a semi-supervised approach. We exploit the fact that most crops are planted in rows with a similar spacing along the row, which in turn can be used to initialize a vision-based classifier requiring only minimal user efforts to adapt it to a new field. We implemented our approach using C++ and ROS and thoroughly tested it on real farm robots operating in different countries. The experiments presented in this paper show that with around 1 min of labeling time, we can achieve classification results with an accuracy of more than 95% in real sugar beet fields in Germany and Switzerland.

ICRA Conference 2017 Conference Paper

UAV-based crop and weed classification for smart farming

• Philipp Lottes
• Raghav Khanna
• Johannes Pfeifer
• Roland Siegwart
• Cyrill Stachniss

Unmanned aerial vehicles (UAVs) and other robots in smart farming applications offer the potential to monitor farm land on a per-plant basis, which in turn can reduce the amount of herbicides and pesticides that must be applied. A central information for the farmer as well as for autonomous agriculture robots is the knowledge about the type and distribution of the weeds in the field. In this regard, UAVs offer excellent survey capabilities at low cost. In this paper, we address the problem of detecting value crops such as sugar beets as well as typical weeds using a camera installed on a light-weight UAV. We propose a system that performs vegetation detection, plant-tailored feature extraction, and classification to obtain an estimate of the distribution of crops and weeds in the field. We implemented and evaluated our system using UAVs on two farms, one in Germany and one in Switzerland and demonstrate that our approach allows for analyzing the field and classifying individual plants.

ICRA Conference 2016 Conference Paper

An effective classification system for separating sugar beets and weeds for precision farming applications

• Philipp Lottes
• Markus Hoeferlin
• Slawomir Sander
• Matthias Muter
• Peter Schulze Lammers
• Cyrill Stachniss

Robots for precision farming have the potential to reduce the reliance on herbicides and pesticides through selectively spraying individual plants or through manual weed removal. To achieve this, the value crops and the weeds must be identified by the robot's perception system to trigger the actuators for spraying or removal. In this paper, we address the problem of detecting the sugar beet plants as well as weeds using a camera installed on a mobile robot operating on a field. We propose a system that performs vegetation detection, feature extraction, random forest classification, and smoothing through a Markov random field to obtain an accurate estimate of the crops and weeds. We implemented and thoroughly evaluated our system on a real farm robot on different sugar beet fields and illustrate that our approach allows for accurately identifying the weed on the field.

IROS Conference 2016 Conference Paper

Experience-based path planning for mobile robots exploiting user preferences

• Lorenzo Nardi
• Cyrill Stachniss

The demand for flexible industrial robotic solutions that are able to accomplish tasks at different locations in a factory is growing more and more. When deploying mobile robots in a factory environment, the predictability and reproducibility of their behaviors become important and are often requested. In this paper, we propose an easy-to-use motion planning scheme that can take into account user preferences for robot navigation. The preferences are extracted implicitly from the previous experiences or from demonstrations and are automatically considered in the subsequent planning steps. This leads to reproducible and thus better to predict navigation behaviors of the robot, without requiring experts to hard-coding control strategies or cost functions within a planner. Our system has been implemented and evaluated on a simulated KUKA mobile robot in different environments.

IROS Conference 2016 Conference Paper

Exploiting building information from publicly available maps in graph-based SLAM

• Olga Vysotska
• Cyrill Stachniss

Maps are an important component of most robotic navigation systems and building maps under uncertainty is often referred to as simultaneous localization and mapping or SLAM. Most SLAM approaches start from scratch and build a map only based on their own observations and odometry information. In this paper, we address the problem of how additional information can be exploited, for example from OpenStreetMap. We extend the standard graph-based SLAM formulation by relating the nodes of the pose-graph with an existing map. As this paper suggests, we can relate the newly built maps with information from publicly available maps with the laser range finder data from the robot and in this way improve the map quality. We implemented and evaluated our approach using real world data taken in urban environments. We illustrate that our extension to graph-based SLAM provides better aligned maps and adds only a marginal computational overhead.

ICRA Conference 2016 Conference Paper

Fast and effective online pose estimation and mapping for UAVs

• Johannes Schneider 0001
• Christian Eling
• Lasse Klingbeil
• Heiner Kuhlmann
• Wolfgang Förstner
• Cyrill Stachniss

Online pose estimation and mapping in unknown environments is essential for most mobile robots. Especially autonomous unmanned aerial vehicles require good pose estimates at comparably high frequencies. In this paper, we propose an effective system for online pose and simultaneous map estimation designed for light-weight UAVs. Our system consists of two components: (1) real-time pose estimation combining RTK-GPS and IMU at 100 Hz and (2) an effective SLAM solution running at 10 Hz using image data from an omnidirectional multi-fisheye-camera system. The SLAM procedure combines spatial resection computed based on the map that is incrementally refined through bundle adjustment and combines the image data with raw GPS observations and IMU data on keyframes. The overall system yields a real-time, georeferenced pose at 100 Hz in GPS-friendly situations. Additionally, we obtain a precise pose and feature map at 10 Hz even in cases where the GPS is not observable or underconstrained. Our system has been implemented and thoroughly tested on a 5 kg copter and yields accurate and reliable pose estimation at high frequencies. We compare the point cloud obtained by our method with a model generated from georeferenced terrestrial laser scanner.

IROS Conference 2016 Conference Paper

Fast range image-based segmentation of sparse 3D laser scans for online operation

• Igor Bogoslavskyi
• Cyrill Stachniss

Object segmentation from 3D range data is an important topic in mobile robotics. A robot navigating in a dynamic environment needs to be aware of objects that might change or move. A segmentation of the laser scans into individual objects is typically the first processing step before a further analysis is performed. In this paper, we present a fast method that segments 3D range data into different objects, runs online, and has small computational demands. Our approach avoids the explicit computation of the 3D point cloud and performs all computations directly on a 2D range image, which enables a fast segmentation for each scan. A further relevant aspect of our method is that we can segment objects even if the 3D data is sparse. This is important for scanners such as the new Velodyne Puck. We implemented our approach in C++ and ROS and thoroughly tested it using different 3D scanners. Our method can operate at over 100 Hz for the 64-beam Velodyne scanner on a single core of a mobile CPU while producing high quality segmentation results. In addition to this, we make the source code for the approach available.

IROS Conference 2016 Conference Paper

Pose fusion with chain pose graphs for automated driving

• Christian Merfels
• Cyrill Stachniss

Automated driving relies on fast, recent, accurate, and highly available pose estimates. A single localization system, however, can commonly ensure this only to some extent. In this paper, we propose a multi-sensor fusion approach that resolves this by combining multiple localization systems in a plug and play manner. We formulate our approach as a sliding window pose graph and enforce a particular graph structure which enables efficient optimization and a novel form of marginalization. Our pose fusion approach scales from a filtering-based to a batch solution by increasing the size of the sliding window. We evaluate our approach on simulated data as well as on real data gathered with a prototype vehicle and demonstrate that our solution runs comfortably at 20 Hz, provides timely estimates, is accurate, and yields a high availability.

ICRA Conference 2016 Conference Paper

Robust homing for autonomous robots

• Igor Bogoslavskyi
• Mladen Mazuran
• Cyrill Stachniss

In autonomous exploration tasks, robots usually rely on a SLAM system to build a map of the environment online and then use it for navigation purposes. Although there has been substantial progress in robustly building accurate maps, these systems cannot guarantee the consistency of the resulting environment model. In this paper, we address the problem of robustly guiding a robot back to its starting location after exploring an unknown environment-even if the mapping system fails to produce a consistent map. To tackle this problem, we propose a two-step procedure. First, we check if the current map is consistent using a statistical test. If the map is consistent, we navigate the robot back to its starting location using a standard navigation system. In case of an inconsistent map, however, we propose to rewind the trajectory from the current location to the start without relying on a map. We implemented the proposed system in ROS and showcase its effectiveness on an autonomous exploration robot in real underground and office environments.

ICRA Conference 2015 Conference Paper

Efficient and effective matching of image sequences under substantial appearance changes exploiting GPS priors

• Olga Vysotska
• Tayyab Naseer
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

The ability to localize a robot is an important capability and matching of observations under substantial changes is a prerequisite for robust long-term operation. This paper investigates the problem of efficiently coping with seasonal changes in image data. We present an extension of a recent approach [15] to visual image matching using sequence information. Our extension allows for exploiting GPS priors in the matching process to overcome the main computational bottleneck of the previous method and to handle loops within the image sequences. We present an experimental evaluation using real world data containing substantial seasonal changes and show that our approach outperforms the previous method in case a noisy GPS pose prior is available.

ICRA Conference 2015 Conference Paper

Predictive exploration considering previously mapped environments

• Daniel Perea Strom
• Fabrizio Nenci
• Cyrill Stachniss

The ability to explore an unknown environment is an important prerequisite for building truly autonomous robots. The central decision that a robot needs to make when exploring an unknown environment is to select the next view point(s) for gathering observations. In this paper, we consider the problem of how to select view points that support the underlying mapping process. We propose a novel approach that makes predictions about the structure of the environments in the unexplored areas by relying on maps acquired previously. Our approach seeks to find similarities between the current surroundings of the robot and previously acquired maps stored in a database in order to predict how the environment may expand in the unknown areas. This allows us to predict potential future loop closures early. This knowledge is used in the view point selection to actively close loops and in this way reduce the uncertainty in the robot's belief. We implemented and tested the proposed approach. The experiments indicate that our method improves the ability of a robot to explore challenging environments and improves the quality of the resulting maps.

ICRA Conference 2015 Conference Paper

Robot, organize my shelves! Tidying up objects by predicting user preferences

• Nichola Abdo
• Cyrill Stachniss
• Luciano Spinello
• Wolfram Burgard

As service robots become more and more capable of performing useful tasks for us, there is a growing need to teach robots how we expect them to carry out these tasks. However, learning our preferences is a nontrivial problem, as many of them stem from a variety of factors including personal taste, cultural background, or common sense. Obviously, such factors are hard to formulate or model a priori. In this paper, we present a solution for tidying up objects in containers, e. g. , shelves or boxes, by following user preferences. We learn the user preferences using collaborative filtering based on crowdsourced and mined data. First, we predict pairwise object preferences of the user. Then, we subdivide the objects in containers by modeling a spectral clustering problem. Our solution is easy to update, does not require complex modeling, and improves with the amount of user data. We evaluate our approach using crowdsoucing data from over 1, 200 users and demonstrate its effectiveness for two tidy-up scenarios. Additionally, we show that a real robot can reliably predict user preferences using our approach.

IROS Conference 2015 Conference Paper

Robust visual SLAM across seasons

• Tayyab Naseer
• Michael Ruhnke
• Cyrill Stachniss
• Luciano Spinello
• Wolfram Burgard

In this paper, we present an appearance-based visual SLAM approach that focuses on detecting loop closures across seasons. Given two image sequences, our method first extracts one descriptor per image for both sequences using a deep convolutional neural network. Then, we compute a similarity matrix by comparing each image of a query sequence with a database. Finally, based on the similarity matrix, we formulate a flow network problem and compute matching hypotheses between sequences. In this way, our approach can handle partially matching routes, loops in the trajectory and different speeds of the robot. With a matching hypothesis as loop closure information and the odometry information of the robot, we formulate a graph based SLAM problem and compute a joint maximum likelihood trajectory.

ICRA Conference 2015 Conference Paper

Where to park? minimizing the expected time to find a parking space

• Igor Bogoslavskyi
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Quickly finding a free parking spot that is close to a desired target location can be a difficult task. This holds for human drivers and autonomous cars alike. In this paper, we investigate the problem of predicting the occupancy of parking spaces and exploiting this information during route planning. We propose an MDP-based planner that considers route information as well as the occupancy probabilities of parking spaces to compute the path that minimizes the expected total time for finding an unoccupied parking space and for walking from the parking location to the target destination. We evaluated our system on real world data gathered over several days in a real parking lot. We furthermore compare our approach to three parking strategies and show that our method outperforms the alternative behaviors.

ICRA Conference 2014 Conference Paper

A statistical measure for map consistency in SLAM

• Mladen Mazuran
• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Map consistency is an important requirement for applications in which mobile robots need to effectively perform autonomous navigation tasks. While recent SLAM techniques provide an increased robustness even in the context of bad initializations or data association outliers, the question of how to determine whether or not the resulting map is consistent is still an open problem. In this paper, we introduce a novel measure for map consistency. We compute this measure by taking into account the discrepancies in the sensor data and leverage it to address two important problems in SLAM. First, we derive a statistical test for assessing whether a map is consistent or not. Second, we employ it to automatically set the free parameter of dynamic covariance scaling, a robust SLAM back-end. We present an evaluation of our approach on over 50 maps sourced from 16 publicly available datasets and illustrate its capability for the inconsistency detection and the tuning of the parameter of the back-end.

IROS Conference 2014 Conference Paper

Automatic channel selection and neural signal estimation across channels of neural probes

• Olga Vysotska
• Barbara Frank
• István Ulbert
• Oliver Paul
• Patrick Ruther
• Cyrill Stachniss
• Wolfram Burgard

High-resolution microprobes are used to record single neuron activity in the brain. This technology is envisaged to be a central component for brain-controlled computers and robots. Current neural probes, however, allow for recording only a small number of the densely spaced electrodes simultaneously. Therefore, we address the problem of autonomously choosing, for a given number, the subset of electrodes with the corresponding size so as to extract as much information as possible. We first present an approach for predicting neural spikes across different channels of the probe. Our method employs nonparametric sparse Gaussian process regression to predict the signal of a channel given the signals recorded at neighboring sites. Second, we utilize the signal predictions for efficiently seeking for the subset of electrodes that minimizes the overall prediction error. In experiments carried out using real neural data, we demonstrate that our selection procedure provides highly accurate results. Furthermore, the solutions found in our experiments are close to the optimal solution.

IROS Conference 2014 Conference Paper

Effective compression of range data streams for remote robot operations using H. 264

• Fabrizio Nenci
• Luciano Spinello
• Cyrill Stachniss

Most robots need the ability to communicate with a base station or with an operator during their mission. Teleoperated and semi-autonomous robots typically communicate continuously through a network connection with an operator. Transmitting raw sensor data over a low bandwidth network such as wireless or HSDPA, however, is problematic as the stream of sensor data is often large. In this paper, we present a method that exploits H. 264 compression to reduce the size of range data streams from sensors such as the Kinect camera or the Velodyne 3D laser scanner. We developed a practical and effective solution that exploits the state of the art in video compression to produce high-quality results. Our method is easy to implement and can have practical impact for researchers building robots for the real world. We implemented and thoroughly tested our approach using a large number of range data streams. Furthermore, we analyzed the impact of data compression on the accuracy and size of the transmitted data. We show that even a highly compressed stream of depth images can be used with dense mapping techniques such as KinFu for building environment models.

ICRA Conference 2014 Conference Paper

Experimental analysis of dynamic covariance scaling for robust map optimization under bad initial estimates

• Pratik Agarwal
• Giorgio Grisetti
• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Non-linear error minimization methods became widespread approaches for solving the simultaneous localization and mapping problem. If the initial guess is far away from the global minimum, converging to the correct solution and not to a local one can be challenging and sometimes even impossible. This paper presents an experimental analysis of dynamic covariance scaling, a recently proposed method for robust optimization of SLAM graphs, in the context of a poor initialization. Our evaluation shows that dynamic covariance scaling is able to mitigate the effects of poor initializations. In contrast to other methods that first aim at finding a good initial guess to seed the optimization, our method is more elegant because it does not require an additional method for initialization. Furthermore, it can robustly handle data association outliers. Experiments performed with real world and simulated datasets show that dynamic covariance scaling outperforms existing methods, both in the presence and absence of data association outliers.

ICRA Conference 2014 Conference Paper

Helmert's and Bowie's geodetic mapping methods and their relation to graph-based SLAM

• Pratik Agarwal
• Wolfram Burgard
• Cyrill Stachniss

The problem of simultaneously localization a robot and modeling the environment is a prerequisite for several robotic applications and a large variety of solutions have been proposed allowing robots to build maps and use them for navigation. Also the geodetic community addressed large-scale mapping for centuries, computing maps which span across continents. These mapping processes had to deal with several challenges that are similar to those of the robotics community. In this paper, we explain two key geodetic mapping methods that we believe are relevant for robotics. We also aim at providing a geodetic perspective on current state-of-the-art SLAM methods and at identifying similarities between the solutions proposed by both communities. The central goal of this paper is to bring both fields close together and to enable future synergies.

ICRA Conference 2014 Conference Paper

Inferring what to imitate in manipulation actions by using a recommender system

• Nichola Abdo
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Learning from demonstrations is an intuitive way for instructing robots by non-experts. One challenge in learning from demonstrations is to infer what to imitate, especially when the robot only observes the teacher and does not have further knowledge about the demonstrated actions. In this paper, we present a novel approach to the problem of inferring what to imitate to successfully reproduce a manipulation action based on a small number of demonstrations. Our method employs techniques from recommender systems to include expert knowledge. It models the demonstrated actions probabilistically and formulates the problem of inferring what to imitate via model selection. We select an appropriate model for the action each time the robot has to reproduce it given a new starting condition. We evaluate our approach using data acquired with a PR2 robot and demonstrate that our method achieves high success rates in different scenarios.

ICRA Conference 2014 Conference Paper

Learning to give route directions from human demonstrations

• Stefan Oßwald
• Henrik Kretzschmar
• Wolfram Burgard
• Cyrill Stachniss

For several applications, robots and other computer systems must provide route descriptions to humans. These descriptions should be natural and intuitive for the human users. In this paper, we present an algorithm that learns how to provide good route descriptions from a corpus of human-written directions. Using inverse reinforcement learning, our algorithm learns how to select the information for the description depending on the context of the route segment. The algorithm then uses the learned policy to generate directions that imitate the style of the descriptions provided by humans, thus taking into account personal as well as cultural preferences and special requirements of the particular user group providing the learning demonstrations. We evaluate our approach in a user study and show that the directions generated by our policy sound similar to human-given directions and substantially more natural than directions provided by commercial web services.

AAAI Conference 2014 Conference Paper

Robust Visual Robot Localization Across Seasons Using Network Flows

• Tayyab Naseer
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Image-based localization is an important problem in robotics and an integral part of visual mapping and navigation systems. An approach to robustly match images to previously recorded ones must be able to cope with seasonal changes especially when it is supposed to work reliably over long periods of time. In this paper, we present a novel approach to visual localization of mobile robots in outdoor environments, which is able to deal with substantial seasonal changes. We formulate image matching as a minimum cost flow problem in a data association graph to effectively exploit sequence information. This allows us to deal with non-matching image sequences that result from temporal occlusions or from visiting new places. We present extensive experimental evaluations under substantial seasonal changes. Our approach achieves accurate matching across seasons and outperforms existing state-of-the-art methods such as FABMAP2 and SeqSLAM.

ICRA Conference 2014 Conference Paper

W-RGB-D: Floor-plan-based indoor global localization using a depth camera and WiFi

• Seigo Ito
• Felix Endres
• Markus Kuderer
• Gian Diego Tipaldi
• Cyrill Stachniss
• Wolfram Burgard

Localization approaches typically rely on an already available map to identify the position of the sensor in the environment. Such maps are usually built beforehand and often require the user to record data from the same sensor used for localization. In this paper, we relax this assumption and present a localization approach based on architectural floor plans. In general, floor plans are readily available for most man-made buildings but only represent basic architectural structures. The incomplete knowledge leads to ambiguous pose estimates. To solve this problem, we present W-RGB-D, a new method for indoor global localization based on WiFi and an RGB-D camera. We introduce a sensor model for RGB-D cameras that is suitable to be used with abstract floor plans. To resolve ambiguities during global localization, we estimate a coarse initial distribution about the sensor position using the WiFi signal strength. We evaluate our W-RGB-D localization method in indoor environments and compare its performance with RGB-D-based Monte Carlo localization. Our results demonstrate that the use of WiFi information as proposed with our approach improves the localization in terms of convergence speed and quality of the solution.

ICRA Conference 2013 Conference Paper

A navigation system for robots operating in crowded urban environments

• Rainer Kümmerle
• Michael Ruhnke
• Bastian Steder
• Cyrill Stachniss
• Wolfram Burgard

Over the past years, there has been a tremendous progress in the area of robot navigation. Most of the systems developed thus far, however, are restricted to indoor scenarios, non-urban outdoor environments, or road usage with cars. Urban areas introduce numerous challenges to autonomous mobile robots as they are highly complex and in addition to that dynamic. In this paper, we present a navigation system for pedestrian-like autonomous navigation with mobile robots in city environments. We describe different components including a SLAM system for dealing with huge maps of city centers, a planning approach for inferring feasible paths taking also into account the traversability and type of terrain, and a method for accurate localization in dynamic environments. The navigation system has been implemented and tested in several large-scale field tests in which the robot Obelix managed to autonomously navigate from our university campus over a 3. 3 km long route to the city center of Freiburg.

ICRA Conference 2013 Conference Paper

Learning manipulation actions from a few demonstrations

• Nichola Abdo
• Henrik Kretzschmar
• Luciano Spinello
• Cyrill Stachniss

To efficiently plan complex manipulation tasks, robots need to reason on a high level. Symbolic planning, however, requires knowledge about the preconditions and effects of the individual actions. In this work, we present a practical approach to learn manipulation skills, including preconditions and effects, based on teacher demonstrations. We believe that requiring only a small number of demonstrations is essential for robots operating in the real world. Therefore, our main focus and contribution is the ability to infer the preconditions and effects of actions based on a small number of demonstrations. Our system furthermore expresses the acquired manipulation actions as planning operators and is therefore able to use symbolic planners to solve new tasks. We implemented our approach on a PR2 robot and present real world manipulation experiments that illustrate that our system allows non-experts to transfer knowledge to robots.

ICRA Conference 2013 Conference Paper

Robust map optimization using dynamic covariance scaling

• Pratik Agarwal
• Gian Diego Tipaldi
• Luciano Spinello
• Cyrill Stachniss
• Wolfram Burgard

Developing the perfect SLAM front-end that produces graphs which are free of outliers is generally impossible due to perceptual aliasing. Therefore, optimization back-ends need to be able to deal with outliers resulting from an imperfect front-end. In this paper, we introduce dynamic covariance scaling, a novel approach for effective optimization of constraint networks under the presence of outliers. The key idea is to use a robust function that generalizes classical gating and dynamically rejects outliers without compromising convergence speed. We implemented and thoroughly evaluated our method on publicly available datasets. Compared to recently published state-of-the-art methods, we obtain a substantial speed up without increasing the number of variables in the optimization process. Our method can be easily integrated in almost any SLAM back-end.

IROS Conference 2012 Conference Paper

On the position accuracy of mobile robot localization based on particle filters combined with scan matching

• Jörg Röwekämper
• Christoph Sprunk
• Gian Diego Tipaldi
• Cyrill Stachniss
• Patrick Pfaff
• Wolfram Burgard

Many applications in mobile robotics and especially industrial applications require that the robot has a precise estimate about its pose. In this paper, we analyze the accuracy of an integrated laser-based robot pose estimation and positioning system for mobile platforms. For our analysis, we used a highly accurate motion capture system to precisely determine the error in the robot's pose. We are able to show that by combining standard components such as Monte-Carlo localization, KLD sampling, and scan matching, an accuracy of a few millimeters at taught-in reference locations can be achieved. We believe that this is an important analysis for developers of robotic applications in which pose accuracy matters.

IROS Conference 2011 Conference Paper

Accurate human motion capture in large areas by combining IMU- and laser-based people tracking

• Jakob Ziegler
• Henrik Kretzschmar
• Cyrill Stachniss
• Giorgio Grisetti
• Wolfram Burgard

A large number of applications use motion capture systems to track the location and the body posture of people. For instance, the movie industry captures actors to animate virtual characters that perform stunts. Today's tracking systems either operate with statically mounted cameras and thus can be used in confined areas only or rely on inertial sensors that allow for free and large-scale motion but suffer from drift in the pose estimate. This paper presents a novel tracking approach that aims to provide globally aligned full body posture estimates by combining a mobile robot and an inertial motion capture system. In our approach, a mobile robot equipped with a laser scanner is used to anchor the pose estimates of a person given a map of the environment. It uses a particle filter to globally localize a person wearing a motion capture suit and to robustly track the person's position. To obtain a smooth and globally aligned trajectory of the person, we solve a least squares optimization problem formulated from the motion capture suite and tracking data. Our approach has been implemented on a real robot and exhaustively tested. As the experimental evaluation shows, our system is able to provide locally precise and globally aligned estimates of the person's full body posture.

IROS Conference 2011 Conference Paper

Efficient information-theoretic graph pruning for graph-based SLAM with laser range finders

• Henrik Kretzschmar
• Cyrill Stachniss
• Giorgio Grisetti

In graph-based SLAM, the pose graph encodes the poses of the robot during data acquisition as well as spatial constraints between them. The size of the pose graph has a substantial influence on the runtime and the memory requirements of a SLAM system, which hinders long-term mapping. In this paper, we address the problem of efficient information-theoretic compression of pose graphs. Our approach estimates the expected information gain of laser measurements with respect to the resulting occupancy grid map. It allows for restricting the size of the pose graph depending on the information that the robot acquires about the environment or based on a given memory limit, which results in an any-space SLAM system. When discarding laser scans, our approach marginalizes out the corresponding pose nodes from the graph. To avoid a densely connected pose graph, which would result from exact marginalization, we propose an approximation to marginalization that is based on local Chow-Liu trees and maintains a sparse graph. Real world experiments suggest that our approach effectively reduces the growth of the pose graph while minimizing the loss of information in the resulting grid map.

IROS Conference 2011 Conference Paper

Efficient motion planning for manipulation robots in environments with deformable objects

• Barbara Frank
• Cyrill Stachniss
• Nichola Abdo
• Wolfram Burgard

The ability to plan their own motions and to reliably execute them is an important precondition for autonomous robots. In this paper, we consider the problem of planning the motion of a mobile manipulation robot in the presence of deformable objects. Our approach combines probabilistic roadmap planning with a physical deformation simulation system. Since the physical deformation simulation is computationally demanding, we use efficient Gaussian process regression to estimate the deformation cost for individual objects based on training examples. We generate the training data by employing a simulation system in a preprocessing step. Consequently, no simulations are needed during runtime. We implemented and tested our approach on a mobile manipulation robot. Our experiments show that the robot is able to accurately predict and thus consider the deformation cost its manipulator introduces to the environment during motion planning. Simultaneously, the computation time is substantially reduced compared to a system that employs physical simulations online.

IROS Conference 2011 Conference Paper

Hierarchies of octrees for efficient 3D mapping

• Kai M. Wurm
• Daniel Hennes
• Dirk Holz
• Radu Bogdan Rusu
• Cyrill Stachniss
• Kurt Konolige
• Wolfram Burgard

The on-chip fabrication and manipulation of microstructures are expected to be applied for single cell analysis system such as cell manipulation and measurement tools. In this paper, we previously present a methodology for fabricating and assembling microstructures inside a microfluidic channel. By the illumination of patterned UV-ray through the mask under a microscope, microstructures with arbitrary shape are made of the photo-crosslinkable resin inside microfluidic device. The microstructures are fabricated at the desired place inside microfluidic channel and manipulated by optical tweezers. Based on the technique which can manipulate multiple points simultaneously by high-speed scanning of a single laser with galvanometer mirror, a rotational microstructure made of a microgear and a rotation axis is assembled and rotated. We also report two methods of solution replacement inside microfluidic channel which reduces viscosity of solvent in order to improve manipulation performance. By adjusting the concentration of photo-crosslinkable resin and replacing solution components, the viscosity of solvent inside channel can be changed. The manipulation speed of the rotational microstructure increases when the viscosity of solvent decreases, because the viscosity resistance for the movement of microstructure is weaker inside lower viscosity solvent. We fabricate rotational microstructures inside lower viscosity solvent and evaluate the movement efficiency compared with microstructures inside former high viscosity solvent.

ICRA Conference 2011 Conference Paper

Self-supervised obstacle detection for humanoid navigation using monocular vision and sparse laser data

• Daniel Maier 0001
• Maren Bennewitz
• Cyrill Stachniss

In this paper, we present an approach to obstacle detection for collision-free, efficient humanoid robot navigation based on monocular images and sparse laser range data. To detect arbitrary obstacles in the surroundings of the robot, we analyze 3D data points obtained from a 2D laser range finder installed in the robot's head. Relying only on this laser data, however, can be problematic. While walking, the floor close to the robot's feet is not observable by the laser sensor, which inherently increases the risk of collisions, especially in nonstatic scenes. Furthermore, it is time-consuming to frequently stop walking and tilting the head to obtain reliable information about close obstacles. We therefore present a technique to train obstacle detectors for images obtained from a monocular camera also located in the robot's head. The training is done online based on sparse laser data in a self-supervised fashion. Our approach projects the obstacles identified from the laser data into the camera image and learns a classifier that considers color and texture information. While the robot is walking, it then applies the learned classifiers to the images to decide which areas are traversable. As we illustrate in experiments with a real humanoid, our approach enables the robot to reliably avoid obstacles during navigation. Furthermore, the results show that our technique leads to significantly more efficient navigation compared to extracting obstacles solely based on 3D laser range data acquired while the robot is standing at certain intervals.

ICRA Conference 2010 Conference Paper

Consistent mapping of multistory buildings by introducing global constraints to graph-based SLAM

• Michael Karg
• Kai M. Wurm
• Cyrill Stachniss
• Klaus Dietmayer
• Wolfram Burgard

In the past, there has been a tremendous advance in the area of simultaneous localization and mapping (SLAM). However, there are relatively few approaches for incorporating prior information or knowledge about structural similarities into the mapping process. Consider, for example, office buildings in which most of the offices have an identical geometric layout. The same typically holds for the individual stories of buildings. In this paper, we propose an approach for generating alignment constraints between different floors of the same building in the context of graph-based SLAM. This is done under the assumption that the individual floors of a building share at least some structural properties. To identify such areas, we apply a particle filter-based localization approach using maps and observations from different floors. We evaluate our system using several real datasets as well as in simulation. The results demonstrate that our approach is able to correctly align multiple floors and allows the robot to generate consistent models of multi-story buildings.

IROS Conference 2010 Conference Paper

Coordinated exploration with marsupial teams of robots using temporal symbolic planning

• Kai M. Wurm
• Christian Dornhege
• Patrick Eyerich
• Cyrill Stachniss
• Bernhard Nebel
• Wolfram Burgard

The problem of autonomously exploring an environment with a team of robots received considerable attention in the past. However, there are relatively few approaches to coordinate teams of robots that are able to deploy and retrieve other robots. Efficiently coordinating the exploration with such marsupial robots requires advanced planning mechanisms that are able to consider symbolic deployment and retrieval actions. In this paper, we propose a novel approach for coordinating the exploration with marsupial robot teams. Our method integrates a temporal symbolic planner that explicitly considers deployment and retrieval actions with a traditional cost-based assignment procedure. Our approach has been implemented and evaluated in several simulated environments and with varying team sizes. The results demonstrate that our proposed method is able to coordinate marsupial teams of robots to efficiently explore unknown environments.

ICRA Conference 2010 Conference Paper

Hierarchical optimization on manifolds for online 2D and 3D mapping

• Giorgio Grisetti
• Rainer Kümmerle
• Cyrill Stachniss
• Udo Frese
• Christoph Hertzberg

In this paper, we present a new hierarchical optimization solution to the graph-based simultaneous localization and mapping (SLAM) problem. During online mapping, the approach corrects only the coarse structure of the scene and not the overall map. In this way, only updates for the parts of the map that need to be considered for making data associations are carried out. The hierarchical approach provides accurate non-linear map estimates while being highly efficient. Our error minimization approach exploits the manifold structure of the underlying space. In this way, it avoids singularities in the state space parameterization. The overall approach is accurate, efficient, designed for online operation, overcomes singularities, provides a hierarchical representation, and outperforms a series of state-of-the-art methods.

IROS Conference 2010 Conference Paper

Learning the elasticity parameters of deformable objects with a manipulation robot

• Barbara Frank
• Ruediger Schmedding
• Cyrill Stachniss
• Matthias Teschner
• Wolfram Burgard

In this paper, we consider the problem of determining the elasticity properties of deformable objects with a mobile manipulator equipped with a force sensorb. We learn the parameters by establishing a relation between the applied forces and the corresponding surface deformations. To determine the parameters, we minimize the difference between the observed surface of an object that is deformed by a real manipulator and the deformed surface obtained with a deformation simulator based on finite element methods. To establish the correspondences between the surfaces, our approach applies a 3D registration technique based on point-clouds which is used as the basis for comparing the results of the simulation system with the observations of the real deformations. As we demonstrate in real-world experiments, our system is able to estimate appropriate parameters that can be used to predict future deformations. This information can directly be incorporated into motion planning approaches that are designed for robots operating with deformable objects.

IROS Conference 2010 Conference Paper

Operating articulated objects based on experience

• Jürgen Sturm
• Advait Jain
• Cyrill Stachniss
• Charles C. Kemp
• Wolfram Burgard

Many tasks that would be of benefit to users in domestic environments require that robots manipulate articulated objects such as doors and drawers. In this paper, we present a novel approach that simultaneously estimates the kinematic model of an articulated object based on the trajectory described by the robot's end effector, and uses this model to predict the future trajectory of the end effector. One advantage of our approach is that the robot can directly use these predictions to generate an equilibrium point control path for operating the mechanism. Additionally, our approach can improve these predictions based on previously learned articulation models. We have implemented and tested our approach on a real mobile manipulator. Through 40 trials, we show that the robot can reliably open various household objects, including cabinet doors, sliding doors, office drawers, and a dishwasher. Furthermore, we demonstrate that using the information from previous interactions as a prior significantly improves the prediction accuracy.

ICRA Conference 2010 Conference Paper

Vision-based detection for learning articulation models of cabinet doors and drawers in household environments

• Jürgen Sturm
• Kurt Konolige
• Cyrill Stachniss
• Wolfram Burgard

Service robots deployed in domestic environments generally need the capability to deal with articulated objects such as doors and drawers in order to fulfill certain mobile manipulation tasks. This however, requires, that the robots are able to perceive the articulation models of such objects. In this paper, we present an approach for detecting, tracking, and learning articulation models for cabinet doors and drawers without using artificial markers. Our approach uses a highly efficient and sampling-based approach to rectangle detection in depth images obtained from a self-developed active stereo system. The robot can use the generative models learned for the articulated objects to estimate their articulation type, their current configuration, and to make predictions about possible configurations not observed before. We present experiments carried out on real data obtained from our active stereo system. The results demonstrate that our technique is able to learn accurate articulation models. We furthermore provide a detailed error analysis based on ground truth data obtained in a motion capturing studio.

IJCAI Conference 2009 Conference Paper

• Jürgen Sturm
• Vijay Pradeep
• Cyrill Stachniss
• Christian Plagemann
• Kurt Konolige
• Wolfram Burgard

Robots operating in home environments must be able to interact with articulated objects such as doors or drawers. Ideally, robots are able to autonomously infer articulation models by observation. In this paper, we present an approach to learn kinematic models by inferring the connectivity of rigid parts and the articulation models for the corresponding links. Our method uses a mixture of parameterized and parameter-free (Gaussian process) representations and finds low-dimensional manifolds that provide the best explanation of the given observations. Our approach has been implemented and evaluated using real data obtained in various realistic home environment settings.

IROS Conference 2009 Conference Paper

A comparison of SLAM algorithms based on a graph of relations

• Wolfram Burgard
• Cyrill Stachniss
• Giorgio Grisetti
• Bastian Steder
• Rainer Kümmerle
• Christian Dornhege
• Michael Ruhnke
• Alexander Kleiner

In this paper, we address the problem of creating an objective benchmark for comparing SLAM approaches. We propose a framework for analyzing the results of SLAM approaches based on a metric for measuring the error of the corrected trajectory. The metric uses only relative relations between poses and does not rely on a global reference frame. The idea is related to graph-based SLAM approaches in the sense that it considers the energy needed to deform the trajectory estimated by a SLAM approach to the ground truth trajectory. Our method enables us to compare SLAM approaches that use different estimation techniques or different sensor modalities since all computations are made based on the corrected trajectory of the robot. We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the SLAM community. The relations have been obtained by manually matching laser-range observations. We believe that our benchmarking framework allows the user an easy analysis and objective comparisons between different SLAM approaches.

ICRA Conference 2009 Conference Paper

Imitation learning with generalized task descriptions

• Clemens Eppner
• Jürgen Sturm
• Maren Bennewitz
• Cyrill Stachniss
• Wolfram Burgard

In this paper, we present an approach that allows a robot to observe, generalize, and reproduce tasks observed from multiple demonstrations. Motion capture data is recorded in which a human instructor manipulates a set of objects. In our approach, we learn relations between body parts of the demonstrator and objects in the scene. These relations result in a generalized task description. The problem of learning and reproducing human actions is formulated using a dynamic Bayesian network (DBN). The posteriors corresponding to the nodes of the DBN are estimated by observing objects in the scene and body parts of the demonstrator. To reproduce a task, we seek for the maximum-likelihood action sequence according to the DBN. We additionally show how further constraints can be incorporated online, for example, to robustly deal with unforeseen obstacles. Experiments carried out with a real 6-DoF robotic manipulator as well as in simulation show that our approach enables a robot to reproduce a task carried out by a human demonstrator. Our approach yields a high degree of generalization illustrated by performing a pick-and-place and a whiteboard cleaning task.

IROS Conference 2009 Conference Paper

Improving robot navigation in structured outdoor environments by identifying vegetation from laser data

• Kai M. Wurm
• Rainer Kümmerle
• Cyrill Stachniss
• Wolfram Burgard

This paper addresses the problem of vegetation detection from laser measurements. The ability to detect vegetation is important for robots operating outdoors, since it enables a robot to navigate more efficiently and safely in such environments. In this paper, we propose a novel approach for detecting low, grass-like vegetation using laser remission values. In our algorithm, the laser remission is modeled as a function of distance, incidence angle, and material. We classify surface terrain based on 3D scans of the surroundings of the robot. The model is learned in a self-supervised way using vibration-based terrain classification. In all real world experiments we carried out, our approach yields a classification accuracy of over 99%. We furthermore illustrate how the learned classifier can improve the autonomous navigation capabilities of mobile robots.

IROS Conference 2009 Conference Paper

Object identification with tactile sensors using bag-of-features

• Alexander Schneider
• Jürgen Sturm
• Cyrill Stachniss
• Marco Reisert
• Hans Burkhardt
• Wolfram Burgard

In this paper, we present a novel approach for identifying objects using touch sensors installed in the finger tips of a manipulation robot. Our approach operates on low-resolution intensity images that are obtained when the robot grasps an object. We apply a bag-of-words approach for object identification. By means of unsupervised clustering on training data, our approach learns a vocabulary from tactile observations which is used to generate a histogram codebook. The histogram codebook models distributions over the vocabulary and is the core identification mechanism. As the objects are larger than the sensor, the robot typically needs multiple grasp actions at different positions to uniquely identify an object. To reduce the number of required grasp actions, we apply a decision-theoretic framework that minimizes the entropy of the probabilistic belief about the type of the object. In our experiments carried out with various industrial and household objects, we demonstrate that our approach is able to discriminate between a large set of objects. We furthermore show that using our approach, a robot is able to distinguish visually similar objects that have different elasticity properties by using only the information from the touch sensor.

ICRA Conference 2009 Conference Paper

Real-world robot navigation amongst deformable obstacles

• Barbara Frank
• Cyrill Stachniss
• Ruediger Schmedding
• Matthias Teschner
• Wolfram Burgard

In this paper, we consider the problem of mobile robots navigating in environments with non-rigid objects. Whereas robots can plan their paths more effectively when they utilize the information about the deformability of objects, they also need to consider the influence of the interaction with the deformable objects on their measurements during the execution of their navigation task. In this paper, we present a probabilistic approach to identify the measurements influenced by the deformable objects. Based on a learned statistics about the influence of the deformable objects on the measurements, the robot is able to perform a sensor-based collision avoidance of unforeseen objects. We present experiments carried out with a real robot that illustrate the practicability of our approach.

ICRA Conference 2009 Conference Paper

Utilizing reflection properties of surfaces to improve mobile robot localization

• Maren Bennewitz
• Cyrill Stachniss
• Sven Behnke
• Wolfram Burgard

A main difficulty that arises in the context of probabilistic localization is the design of an appropriate observation model, i. e. , determining the likelihood of a sensor measurement given the pose of the robot and a map of the environment. Many successful approaches to localization rely on data provided by range sensors, e. g. , laser range scanners. When using such data one normally has to deal with erroneous maximum-range readings that occur due to poor-reflecting surfaces. In general, these readings cannot be distinguished from readings obtained when no obstacle is within the measurement range of the sensor. Therefore, existing localization techniques treat these readings alike in the observation model. In this paper, we present a novel approach that explicitly considers the reflection properties of surfaces and thus the expectation of valid range measurements. In addition to the expected range measurement, we compute the probability of reflectance for a beam given the relative pose of the robot to the obstacle taking into account the angle of incidence of the beam. We estimate the reflection properties of surfaces using data collected with a mobile robot equipped with a laser range scanner. As we demonstrate in experiments carried out with a real robot, our technique leads to significantly improved localization results compared to a state-of-the-art observation model.

ICRA Conference 2009 Conference Paper

Which landmark is useful? Learning selection policies for navigation in unknown environments

• Hauke Strasdat
• Cyrill Stachniss
• Wolfram Burgard

In general, a mobile robot that operates in unknown environments has to maintain a map and has to determine its own location given the map. This introduces significant computational and memory constraints for most autonomous systems, especially for lightweight robots such as humanoids or flying vehicles. In this paper, we present a novel approach for learning a landmark selection policy that allows a robot to discard landmarks that are not valuable for its current navigation task. This enables the robot to reduce the computational burden and to carry out its task more efficiently by maintaining only the important landmarks. Our approach applies an unscented Kalman filter for addressing the simultaneous localization and mapping problems and uses Monte-Carlo reinforcement learning to obtain the selection policy. Based on real world and simulation experiments, we show that the learned policies allow for efficient robot navigation and outperform handcrafted strategies. We furthermore demonstrate that the learned policies are not only usable in a specific scenario but can also be generalized towards environments with varying properties.

IROS Conference 2008 Conference Paper

Coordinated multi-robot exploration using a segmentation of the environment

• Kai M. Wurm
• Cyrill Stachniss
• Wolfram Burgard

This paper addresses the problem of exploring an unknown environment with a team of mobile robots. The key issue in coordinated multi-robot exploration is how to assign target locations to the individual robots such that the overall mission time is minimized. In this paper, we propose a novel approach to distribute the robots over the environment that takes into account the structure of the environment. To achieve this, it partitions the space into segments, for example, corresponding to individual rooms. Instead of only considering frontiers between unknown and explored areas as target locations, we send the robots to the individual segments with the task to explore the corresponding area. Our approach has been implemented and tested in simulation as well as in real world experiments. The experiments demonstrate that the overall exploration time can be significantly reduced by considering our segmentation method.

ICRA Conference 2008 Conference Paper

Efficient path planning for mobile robots in environments with deformable objects

• Barbara Frank
• Markus Becker 0003
• Cyrill Stachniss
• Wolfram Burgard
• Matthias Teschner

The ability to reliably navigate through the environment is an important prerequisite for truly autonomous robots. In this paper, we consider the problem of path planning in environments with non-rigid obstacles such as curtains or plants. We present an approach that combines probabilistic roadmaps with a physical simulation of object deformations to determine a path that optimizes the trade-off between the deformation cost and the distance to be traveled. We describe how our approach utilizes Finite Element theory for calculating the deformation cost. Since the high computational requirements of the corresponding simulation prevent this method from being applicable online, we present an approximation that uses a preprocessing step to determine a deformation cost function for each object. This cost function allows us to estimate the deformation costs of arbitrary paths through the objects and is used to evaluate the trajectories generated by the roadmap planner online. We present experiments which demonstrate that the resulting algorithm plans nearly identical trajectories compared to the method that relies on computationally intense simulations. At the same time, our approach allows the robot to quickly calculate paths in environments with deformable objects.

IROS Conference 2008 Conference Paper

Efficiently learning high-dimensional observation models for Monte-Carlo localization using Gaussian mixtures

• Patrick Pfaff
• Cyrill Stachniss
• Christian Plagemann
• Wolfram Burgard

Whereas probabilistic approaches are a powerful tool for mobile robot localization, they heavily rely on the proper definition of the so-called observation model which defines the likelihood of an observation given the position and orientation of the robot and the map of the environment. Most of the sensor models for range sensors proposed in the past either consider the individual beam measurements independently or apply uni-modal models to represent the likelihood function. In this paper, we present an approach that learns place-dependent sensor models for entire range scans using Gaussian mixture models. To deal with the high dimensionality of the measurement space, we utilize principle component analysis for dimensionality reduction. In practical experiments carried out with data obtained from a real robot, we demonstrate that our model substantially outperforms existing and popular sensor models.

IROS Conference 2008 Conference Paper

Estimating landmark locations from geo-referenced photographs

• Henrik Kretzschmar
• Cyrill Stachniss
• Christian Plagemann
• Wolfram Burgard

The problem of estimating the positions of landmarks using a mobile robot equipped with a camera has intensively been studied in the past. In this paper, we consider a variant of this problem in which the robot should estimate the locations of observed landmarks based on a sparse set of geo-referenced images for which no heading information is available. Sources for such kind of data are image portals such as Flickr or Google Image Search. We formulate the problem of estimating the landmark locations as an optimization problem and show that it is possible to accurately localize the landmarks in real world settings.

ICRA Conference 2008 Conference Paper

How to learn accurate grid maps with a humanoid

• Cyrill Stachniss
• Maren Bennewitz
• Giorgio Grisetti
• Sven Behnke
• Wolfram Burgard

Humanoids have recently become a popular research platform in the robotics community. Such robots offer various fields for new applications. However, they have several drawbacks compared to wheeled vehicles such as stability problems, limited payload capabilities, violation of the flat world assumption, and they typically provide only very rough odometry information, if at all. In this paper, we investigate the problem of learning accurate grid maps with humanoid robots. We present techniques to deal with some of the above-mentioned difficulties. We describe how an existing approach to the simultaneous localization and mapping (SLAM) problem can be adapted to robustly learn accurate maps with a humanoid equipped with a laser range finder. We present an experiment in which our mapping system builds a highly accurate map with a size of around 20 m by 20 m using data acquired with a humanoid in our office environment containing two loops. The resulting maps have a similar accuracy as maps built with a wheeled robot.

ICRA Conference 2008 Conference Paper

Monocular range sensing: A non-parametric learning approach

• Christian Plagemann
• Felix Endres
• Jürgen Hess 0001
• Cyrill Stachniss
• Wolfram Burgard

Mobile robots rely on the ability to sense the geometry of their local environment in order to avoid obstacles or to explore the surroundings. For this task, dedicated proximity sensors such as laser range finders or sonars are typically employed. Cameras are a cheap and lightweight alternative to such sensors, but do not directly offer proximity information. In this paper, we present a novel approach to learning the relationship between range measurements and visual features extracted from a single monocular camera image. As the learning engine, we apply Gaussian processes, a non-parametric learning technique that not only yields the most likely range prediction corresponding to a certain visual input but also the predictive uncertainty. This information, in turn, can be utilized in an extended grid-based mapping scheme to more accurately update the map. In practical experiments carried out in different environments with a mobile robot equipped with an omnidirectional camera system, we demonstrate that our system is able to produce proximity estimates with an accuracy comparable to that of dedicated sensors such as sonars or infrared range finders.

ICRA Conference 2008 Conference Paper

Online constraint network optimization for efficient maximum likelihood map learning

• Giorgio Grisetti
• Dario Lodi Rizzini
• Cyrill Stachniss
• Edwin Olson
• Wolfram Burgard

In this paper, we address the problem of incrementally optimizing constraint networks for maximum likelihood map learning. Our approach allows a robot to efficiently compute configurations of the network with small errors while the robot moves through the environment. We apply a variant of stochastic gradient descent and use a tree-based parameterization of the nodes in the network. By integrating adaptive learning rates in the parameterization of the network, our algorithm can use previously computed solutions to determine the result of the next optimization run. Additionally, our approach updates only the parts of the network which are affected by the newly incorporated measurements and starts the optimization approach only if the new data reveals inconsistencies with the network constructed so far. These improvements yield an efficient solution for this class of online optimization problems. Our approach has been implemented and tested on simulated and on real data. We present comparisons to recently proposed online and offline methods that address the problem of optimizing constraint network. Experiments illustrate that our approach converges faster to a network configuration with small errors than the previous approaches.

IROS Conference 2007 Conference Paper

Analyzing gaussian proposal distributions for mapping with rao-blackwellized particle filters

• Cyrill Stachniss
• Giorgio Grisetti
• Wolfram Burgard
• Nicholas Roy

Particle filters are a frequently used filtering technique in the robotics community. They have been successfully applied to problems such as localization, mapping, or tracking. The particle filter framework allows the designer to freely choose the proposal distribution which is used to obtain the next generation of particles in estimating dynamical processes. This choice greatly influences the performance of the filter. Many approaches have achieved good performance through informed proposals which explicitly take into account the current observation. A popular approach is to approximate the desired proposal distribution by a Gaussian. This paper presents a statistical analysis of the quality of such Gaussian approximations. We also propose a way to obtain the optimal proposal in a non-parametric way and then identify the error introduced by the Gaussian approximation. Furthermore, we present an alternative sampling strategy that better deals with situations in which the target distribution is multi-modal. Experimental results indicate that our alternative sampling strategy leads to accurate maps more frequently that the Gaussian approach while requiring only minimal additional computational overhead.

IROS Conference 2007 Conference Paper

Efficient estimation of accurate maximum likelihood maps in 3D

• Giorgio Grisetti
• Slawomir Grzonka
• Cyrill Stachniss
• Patrick Pfaff
• Wolfram Burgard

Learning maps is one of the fundamental tasks of mobile robots. In the past, numerous efficient approaches to map learning have been proposed. Most of them, however, assume that the robot lives on a plane. In this paper, we consider the problem of learning maps with mobile robots that operate in non-flat environments and apply maximum likelihood techniques to solve the graph-based SLAM problem. Due to the non-commutativity of the rotational angles in 3D, major problems arise when applying approaches designed for the two-dimensional world. The non-commutativity introduces serious difficulties when distributing a rotational error over a sequence of poses. In this paper, we present an efficient solution to the SLAM problem that is able to distribute a rotational error over a sequence of nodes. Our approach applies a variant of gradient descent to solve the error minimization problem. We implemented our technique and tested it on large simulated and real world datasets. We furthermore compared our approach to solving the problem by LU-decomposition. As the experiments illustrate, our technique converges significantly faster to an accurate map with low error and is able to correct maps with bigger noise than existing methods.

IROS Conference 2007 Conference Paper

Learning maps in 3D using attitude and noisy vision sensors

• Bastian Steder
• Giorgio Grisetti
• Slawomir Grzonka
• Cyrill Stachniss
• Axel Rottmann
• Wolfram Burgard

In this paper, we address the problem of learning 3D maps of the environment using a cheap sensor setup which consists of two standard web cams and a low cost inertial measurement unit. This setup is designed for lightweight or flying robots. Our technique uses visual features extracted from the web cams and estimates the 3D location of the landmarks via stereo vision. Feature correspondences are estimated using a variant of the PROSAC algorithm. Our mapping technique constructs a graph of spatial constraints and applies an efficient gradient descent-based optimization approach to estimate the most likely map of the environment. Our approach has been evaluated in comparably large outdoor and indoor environments. We furthermore present experiments in which our technique is applied to build a map with a blimp.

ICRA Conference 2007 Conference Paper

Towards Mapping of Cities

• Patrick Pfaff
• Rudolph Triebel
• Cyrill Stachniss
• Pierre Lamon
• Wolfram Burgard
• Roland Siegwart

Map learning is a fundamental task in mobile robotics because maps are required for a series of high level applications. In this paper, we address the problem of building maps of large-scale areas like villages or small cities. We present our modified car-like robot which we use to acquire the data about the environment. We introduce our localization system which is based on an information filter and is able to merge the information obtained by different sensors. We furthermore describe out mapping technique that is able to compactly model three-dimensional scenes and allows us efficient and accurate incremental map learning. We additionally apply a global optimization techniques in order to accurately close loops in the environment. Our approach has been implemented and deeply tested on a real car equipped with a series of sensors. Experiments described in this paper illustrate the accuracy and efficiency of the presented techniques.

IROS Conference 2006 Conference Paper

Improving Data Association in Vision-based SLAM

• Arturo Gil
• Óscar Reinoso
• Óscar Martínez Mozos
• Cyrill Stachniss
• Wolfram Burgard

This paper presents an approach to vision-based simultaneous localization and mapping (SLAM). Our approach uses the scale invariant feature transform (SIFT) as features and applies a rejection technique to concentrate on a reduced set of distinguishable, stable features. We track detected SIFT features over consecutive frames obtained by a stereo camera and select only those features that appear to be stable from different views. Whenever a feature is selected, we compute a representative feature given the previous observations. This approach is applied within a Rao-Blackwellized particle filter to make the data association easier and furthermore to reduce the number of landmarks that need to be maintained in the map. Our system has been implemented and tested on data gathered with a mobile robot in a typical office environment. Experiments presented in this paper demonstrate that our method improves the data association and in this way leads to more accurate maps

ICRA Conference 2006 Conference Paper

Speeding-up Multi-robot Exploration by Considering Semantic Place Information

• Cyrill Stachniss
• Óscar Martínez Mozos
• Wolfram Burgard

In this paper, we consider the problem of exploring an unknown environment with a team of mobile robots. One of the key issues in multi-robot exploration is how to assign target locations to the individual robots. To better distribute the robots over the environment and to avoid redundant work, we take into account the type of place a potential target is located in (e. g. , a corridor or a room). To determine the type of a place, we apply a classifier learned with AdaBoost which additionally considers spatial dependencies between nearby locations. Our approach to incorporate the type of places in the coordination of the robots has been implemented and tested in different environments. The experiments demonstrate that our system effectively distributes the robots over the environment and allows them to accomplish their mission faster compared to approaches that ignore the semantic place labels

ICRA Conference 2006 Conference Paper

Speeding-up Rao-blackwellized SLAM

• Giorgio Grisetti
• Gian Diego Tipaldi
• Cyrill Stachniss
• Wolfram Burgard
• Daniele Nardi

Recently, Rao-Blackwellized particle filters have become a popular tool to solve the simultaneous localization and mapping problem. This technique applies a particle filter in which each particle carries an individual map of the environment. Accordingly, a key issue is to reduce the number of particles and/or to make use of compact map representations. This paper presents an approximative but highly efficient approach to mapping with Rao-Blackwellized particle filters. Moreover, it provides a compact map model. A key advantage is that the individual particles can share large parts of the model of the environment. Furthermore, they are able to re-use an already computed proposal distribution. Both techniques substantially speed up the overall process and reduce the memory requirements. Experimental results obtained with mobile robots in large-scale indoor environments and based on published, standard datasets illustrate the advantages of our methods over previous Rao-Blackwellized mapping approaches

ICRA Conference 2005 Conference Paper

Improving Grid-based SLAM with Rao-Blackwellized Particle Filters by Adaptive Proposals and Selective Resampling

• Giorgio Grisetti
• Cyrill Stachniss
• Wolfram Burgard

Recently Rao-Blackwellized particle filters have been introduced as effective means to solve the simultaneous localization and mapping (SLAM) problem. This approach uses a particle filter in which each particle carries an individual map of the environment. Accordingly, a key question is how to reduce the number of particles. In this paper we present adaptive techniques to reduce the number of particles in a Rao-Blackwellized particle filter for learning grid maps. We propose an approach to compute an accurate proposal distribution taking into account not only the movement of the robot but also the most recent observation. This drastically decrease the uncertainty about the robot's pose in the prediction step of the filter. Furthermore, we present an approach to selectively carry out re-sampling operations which seriously reduces the problem of particle depletion. Experimental results carried out with mobile robots in large-scale indoor as well as in outdoor environments illustrate the advantages of our methods over previous approaches.

AAAI Conference 2005 Conference Paper

Mobile Robot Mapping and Localization in Non-Static Environments

• Cyrill Stachniss

Whenever mobile robots act in the real world, they need to be able to deal with non-static objects. In the context of mapping, a common technique to deal with dynamic objects is to filter out the spurious measurements corresponding to such objects. In this paper, we present a novel approach to estimate typical configurations of dynamic areas in the environment of a mobile robot. Our approach clusters local grid maps to identify the possible configurations. We furthermore describe how these clusters can be utilized within a Rao-Blackwellized particle filter to localize a mobile robot in a non-static environment. In practical experiments carried out with a mobile robot in a typical office environment, we demonstrate the advantages of our approach compared to alternative techniques for mapping and localization in dynamic environments.

ICRA Conference 2005 Conference Paper

Recovering Particle Diversity in a Rao-Blackwellized Particle Filter for SLAM After Actively Closing Loops

• Cyrill Stachniss
• Giorgio Grisetti
• Wolfram Burgard

Acquiring models of the environment belongs to the fundamental tasks of mobile robots. Approaches addressing the problem of simultaneous localization and mapping (SLAM) typically process the perceived sensor data and do not influence the motion of the mobile robot. In this paper, we present an approach to actively closing loops during exploration. It applies a Rao-Blackwellized particle filter to maintain multiple hypotheses about potential trajectories of the robot and corresponding maps. To prevent the particle filter from becoming overly confident, we present a technique to recover the particle diversity after successfully closing a loop. This way the particle depletion problem is avoided. The combination of our approach with the active loop closing strategy allows to deal with multiple nested loops. Experimental results presented in this paper illustrate the advantage of our method over pervious approaches to mapping with Rao-Blackwellized particle filters.

AAAI Conference 2005 Conference Paper

Semantic Place Classification of Indoor Environments with Mobile Robots Using Boosting

• Axel Rottmann
• Cyrill Stachniss

Indoor environments can typically be divided into places with different functionalities like kitchens, offices, or seminar rooms. We believe that such semantic information enables a mobile robot to more efficiently accomplish a variety of tasks such as human-robot interaction, path-planning, or localization. This paper presents a supervised learning approach to label different locations using boosting. We train a classifier using features extracted from vision and laser range data. Furthermore, we apply a Hidden Markov Model to increase the robustness of the final classification. Our technique has been implemented and tested on real robots as well as in simulation. The experiments demonstrate that our approach can be utilized to robustly classify places into semantic categories. We also present an example of localization using semantic labeling.

ICRA Conference 2005 Conference Paper

Supervised Learning of Places from Range Data using AdaBoost

• Óscar Martínez Mozos
• Cyrill Stachniss
• Wolfram Burgard

This paper addresses the problem of classifying places in the environment of a mobile robot into semantic categories. We believe that semantic information about the type of place improves the capabilities of a mobile robot in various domains including localization, path-planning, or human-robot interaction. Our approach uses AdaBoost, a supervised learning algorithm, to train a set of classifiers for place recognition based on laser range data. In this paper we describe how this approach can be applied to distinguish between rooms, corridors, doorways, and hallways. Experimental results obtained in simulation and with real robots demonstrate the effectiveness of our approach in various environments.

IROS Conference 2004 Conference Paper

Exploration with active loop-closing for FastSLAM

• Cyrill Stachniss
• Dirk Hähnel
• Wolfram Burgard

Acquiring models of the environment belongs to the fundamental tasks of mobile robots. In the last few years several researchers have focused on the problem of simultaneous localization and mapping (SLAM). Classic SLAM approaches are passive in the sense that they only process the perceived sensor data and do not influence the motion of the mobile robot. In this paper we present a novel and integrated approach that combines autonomous exploration with simultaneous localization and mapping. Our method uses a grid-based version of the FastSLAM algorithm and at each point in time considers actions to actively close loops during exploration. By re-entering already visited areas the robot reduces its localization error and this way learns more accurate maps. Experimental results presented in this paper illustrate the advantage of our method over pervious approaches lacking the ability to actively close loops.

IJCAI Conference 2003 Conference Paper

Exploring Unknown Environments with Mobile Robots using Coverage Maps

• Cyrill Stachniss
• Wolfram Burgard

In this paper we introduce coverage maps as a new way of representing the environment of a mobile robot. Coverage maps store for each cell of a given grid a posterior about the amount the corresponding cell is covered by an obstacle. Using this representation a mobile robot can more accurately reason about its uncertainty in the map of the environment than with standard occupancy grids. We present a model for proximity sensors designed to update coverage maps upon sensory input. We also describe how coverage maps can be used to formulate a decision-theoretic approach for mobile robot exploration. We present experiments carried out with real robots in which accurate maps are build from noisy ultrasound data. Finally, we present a comparison of different view-point selection strategies for mobile robot exploration.

IROS Conference 2003 Conference Paper

Mapping and exploration with mobile robots using coverage maps

• Cyrill Stachniss
• Wolfram Burgard

Exploration and mapping belongs to the fundamental tasks of mobile robots. In the past, many approaches have used occupancy grid maps to represent the environment during the map building process. Occupancy grids, however, are based on the assumption that each cell is either occupied or free. In this paper we introduce coverage maps as an alternative way of representing the environment of a robot. Coverage maps store for each cell of a given grid a posterior about the amount the corresponding cell is covered by an obstacle. We also present a model that allows us to update coverage maps upon input obtained from proximity sensors. We furthermore describe how to use coverage maps for a decision theoretic approach to exploration. Finally we present experimental results illustrating that coverage maps can be used to efficiently learn highly accurate models even if noisy sensors such as ultrasounds are used.

IROS Conference 2002 Conference Paper

An integrated approach to goal-directed obstacle avoidance under dynamic constraints for dynamic environments

• Cyrill Stachniss
• Wolfram Burgard

Whenever robots are installed in populated environments, they need appropriate techniques to avoid collisions with unexpected obstacles. Over the past years several reactive techniques have been developed that use heuristic evaluation functions to choose appropriate actions whenever a robot encounters an unforeseen obstacle. Whereas the majority of these approaches determines only the next steering command, some additionally consider sequences of possible poses. However, they generally do not consider sequences of actions in the velocity space. Accordingly, these methods are not able to slow down the robot early enough before it has to enter a narrow passage. In this paper we present a new approach that integrates path planning with sensor-based collision avoidance. Our algorithm simultaneously considers the robot's pose and velocities during the planning process. We employ different strategies to deal with the huge state space that has to be explored. Our method has been implemented and tested on real robots and in simulation runs. Extensive experiments demonstrate that our technique can reliably control mobile robots moving at high speeds.