Wolfram Burgard

TMLR Journal 2026 Journal Article

Goal Achievement Guided Exploitation: A Principled Performance-Based Scheduling Framework for Reinforcement Learning

• Shengchao Yan
• Baohe Zhang
• Chenguang Huang
• Joschka Boedecker
• Wolfram Burgard

In dense-reward tasks, Reinforcement learning (RL) algorithms often employ soft entropy regularization to promote exploration. By integrating an entropy term into the objective function, they regularize exploration via tuning the coefficient. However, the entropy coefficient only indirectly influences the action distribution through gradient updates, making it difficult to precisely control exploration, and requires careful scheduling to balance exploration and exploitation throughout training. As a solution, we propose Goal Achievement Guided Exploitation~(GAGE), a performance-based scheduling framework that adaptively regulates exploration by linking policy stochasticity directly to the agent's performance relative to a target value. Unlike soft entropy regularizers, GAGE enforces hard, performance-dependent constraints on action distribution's standard deviation for continuous actions and logit range for discrete actions. Consequently, GAGE ensures a guaranteed lower bound on action probabilities that naturally decays as the agent approaches optimal performance. Across a suite of challenging robotic control tasks, GAGE improves learning efficiency and stability across various strong baselines, achieving competitive or superior final performance. By providing a more interpretable and robust alternative to entropy-based exploration heuristics, GAGE offers a scalable path toward solving complex dense reward tasks with pronounced local optima.

ICRA Conference 2025 Conference Paper

CloudTrack: Scalable UAV Tracking with Cloud Semantics

• Yannik Blei
• Michael Krawez
• Nisarga Nilavadi
• Tanja Katharina Kaiser
• Wolfram Burgard

Nowadays, unmanned aerial vehicles (UAVs) are commonly used in search and rescue scenarios to gather information in the search area. The automatic identification of the person searched for in aerial footage could increase the autonomy of such systems, reduce the search time, and thus increase the missed person's chances of survival. In this paper, we present a novel approach to perform semantically conditioned open vocabulary object tracking that is specifically designed to cope with the limitations of UAV hardware. Our approach has several advantages: It can run with verbal descriptions of the missing person, e. g. , the color of the shirt, it does not require dedicated training to execute the mission, and can efficiently track a potentially moving person. Our experimental results demonstrate the versatility and efficacy of our approach. We publish the methods source code at https://github.com/utn-blei/CloudTrack.

IROS Conference 2025 Conference Paper

FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigation

• Samiran Gode
• Abhijeet Nayak
• Débora N. P. Oliveira
• Michael Krawez
• Cordelia Schmid
• Wolfram Burgard

Effective robot navigation in unseen environments is a challenging task that requires precise control actions at high frequencies. Recent advances have framed it as an image-goal-conditioned control problem, where the robot generates navigation actions using frontal RGB images. Current state-of-the-art methods in this area use diffusion policies to generate these control actions. Despite their promising results, these models are computationally expensive and suffer from weak perception. To address these limitations, we present FlowNav, a novel approach that uses a combination of Conditional Flow Matching (CFM) and depth priors from off-the-shelf foundation models to learn action policies for robot navigation. FlowNav is significantly more accurate and faster at navigation and exploration than state-of-the-art methods. We validate our contributions using real robot experiments in multiple environments, demonstrating improved navigation reliability and accuracy. Code and trained models are publicly available †.

IROS Conference 2025 Conference Paper

Label-Efficient LiDAR Panoptic Segmentation

• Ahmet Selim Çanakçi
• Niclas Vödisch
• Kürsat Petek
• Wolfram Burgard
• Abhinav Valada

A main bottleneck of learning-based robotic scene understanding methods is the heavy reliance on extensive annotated training data, which often limits their generalization ability. In LiDAR panoptic segmentation, this challenge becomes even more pronounced due to the need to simultaneously address both semantic and instance segmentation from complex, high-dimensional point cloud data. In this work, we address the challenge of LiDAR panoptic segmentation with very few labeled samples by leveraging recent advances in label-efficient vision panoptic segmentation. To this end, we propose a novel method, Limited-Label LiDAR Panoptic Segmentation (L 3 PS), which requires only a minimal amount of labeled data. Our approach first utilizes a label-efficient 2D network to generate panoptic pseudo-labels from a small set of annotated images, which are subsequently projected onto point clouds. We then introduce a novel 3D refinement module that capitalizes on the geometric properties of point clouds. By incorporating clustering techniques, sequential scan accumulation, and ground point separation, this module significantly enhances the accuracy of the pseudo-labels, improving segmentation quality by up to +10. 6 PQ and +7. 9 mIoU. We demonstrate that these refined pseudo-labels can be used to effectively train off-the-shelf LiDAR segmentation networks. Through extensive experiments, we show that L 3 PS not only outperforms existing methods but also substantially reduces the annotation burden. We release the code of our work at https://l3ps.cs.uni-freiburg.de.

ICRA Conference 2025 Conference Paper

LUMOS: Language-Conditioned Imitation Learning with World Models

• Iman Nematollahi
• Branton DeMoss
• Akshay L. Chandra
• Nick Hawes
• Wolfram Burgard
• Ingmar Posner

We introduce LUMOS, a language-conditioned multi-task imitation learning framework for robotics. LUMOS learns skills by practicing them over many long-horizon rollouts in the latent space of a learned world model and transfers these skills zero-shot to a real robot. By learning on-policy in the latent space of the learned world model, our algorithm mitigates policy-induced distribution shift which most offline imitation learning methods suffer from. LUMOS learns from unstructured play data with fewer than 1 % hindsight language annotations but is steerable with language commands at test time. We achieve this coherent long-horizon performance by combining latent planning with both image-and language-based hindsight goal relabeling during training, and by optimizing an intrinsic reward defined in the latent space of the world model over multiple time steps, effectively reducing covariate shift. In experiments on the difficult long-horizon CALVIN benchmark, LUMOS outperforms prior learning-based methods with com-parable approaches on chained multi-task evaluations. To the best of our knowledge, we are the first to learn a language-conditioned continuous visuomotor control for a real-world robot within an offline world model. Videos, dataset and code are available at http://lumos.cs.uni-freiburg.de.

IROS Conference 2025 Conference Paper

Refined Policy Distillation: From VLA Generalists to RL Experts

• Tobias Jülg
• Wolfram Burgard
• Florian Walter

Vision-Language-Action Models (VLAs) have demonstrated remarkable generalization capabilities in real-world experiments. However, their success rates are often not on par with expert policies, and they require fine-tuning when the setup changes. In this work, we introduce Refined Policy Distillation (RPD), a novel Reinforcement Learning (RL)-based policy refinement method that bridges this performance gap through a combination of on-policy RL with behavioral cloning. The core idea of RPD is to distill and refine VLAs into compact, high-performing expert policies by guiding the student policy during RL exploration using the actions of a teacher VLA, resulting in increased sample efficiency and faster convergence. We complement our method by fine-tuned versions of Octo and OpenVLA for ManiSkill3 to evaluate RPD in simulation. While this is a key requirement for applying RL, it also yields new insights beyond existing studies on VLA performance in real-world settings. Our experimental results across various manipulation tasks show that RPD enables the RL student to learn expert policies that outperform the VLA teacher in both dense and sparse reward settings, while also achieving faster convergence than the RL baseline. Our approach is even robust to changes in camera perspective and can generalize to task variations that the underlying VLA cannot solve. Our code, dataset, VLA checkpoints, and videos are available at https://refined-policy-distillation.github.io

IROS Conference 2025 Conference Paper

REGRACE: A Robust and Efficient Graph-based Re-localization Algorithm using Consistency Evaluation

• Débora N. P. Oliveira
• Joshua Knights
• Sebastián Barbas Laina
• Simon Boche
• Wolfram Burgard
• Stefan Leutenegger

Loop closures are essential for correcting odometry drift and creating consistent maps, especially in the context of large-scale navigation. Current methods using dense point clouds for accurate place recognition do not scale well due to computationally expensive scan-to-scan comparisons. Alternative object-centric approaches are more efficient but often struggle with sensitivity to viewpoint variation. In this work, we introduce REGRACE, a novel approach that addresses these challenges of scalability and perspective difference in re-localization by using LiDAR-based submaps. We introduce rotation-invariant features for each labeled object and enhance them with neighborhood context through a graph neural network. To identify potential revisits, we employ a scalable bag-of-words approach, pooling one learned global feature per submap. Additionally, we define a revisit with geometrical consistency cues rather than embedding distance, allowing us to recognize far-away loop closures. Our evaluations demonstrate that REGRACE achieves similar results compared to state-ofthe-art place recognition and registration baselines while being twice as fast. Code and models are publicly available †.

ICRA Conference 2025 Conference Paper

VLM-Vac: Enhancing Smart Vacuums Through VLM Knowledge Distillation and Language-Guided Experience Replay

• Reihaneh Mirjalili
• Michael Krawez
• Florian Walter
• Wolfram Burgard

In this paper, we propose VLM-Vac, a novel framework designed to enhance the autonomy of smart robot vacuum cleaners. Our approach integrates the zero-shot object detection capabilities of a Vision-Language Model (VLM) with a Knowledge Distillation (KD) strategy. By leveraging the VLM, the robot can categorize objects into actionable classes-either to avoid or to suck-across diverse backgrounds. However, frequently querying the VLM is computationally expensive and impractical for real-world deployment. To address this issue, we implement a KD process that gradually transfers the essential knowledge of the VLM to a smaller, more efficient model. Our real-world experiments demonstrate that this smaller model progressively learns from the VLM and requires significantly fewer queries over time. Additionally, we tackle the challenge of continual learning in dynamic home environments by exploiting a novel experience replay method based on languageguided sampling. Our results show that this approach not only reduces energy consumption by 53 % compared to cumulative learning but also surpasses conventional vision-based clustering methods, particularly in detecting small objects across diverse backgrounds.

IROS Conference 2024 Conference Paper

Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving

• Shengchao Yan
• Lukas König
• Wolfram Burgard

Active traffic management with autonomous vehicles offers the potential for reduced congestion and improved traffic flow. However, developing effective algorithms for real-world scenarios requires overcoming challenges related to infinite-horizon traffic flow and partial observability. To address these issues and further decentralize traffic management, we propose an asymmetric actor-critic model that learns decentralized cooperative driving policies for autonomous vehicles using single-agent reinforcement learning. By employing attention neural networks with masking, our approach efficiently manages real-world traffic dynamics and partial observability, eliminating the need for predefined agents or agent-specific experience buffers in multi-agent reinforcement learning. Extensive evaluations across various traffic scenarios demonstrate our method’s significant potential in improving traffic flow at critical bottleneck points. Moreover, we address the challenges posed by conservative autonomous vehicle driving behaviors that adhere strictly to traffic rules, showing that our cooperative policy effectively alleviates potential slowdowns without compromising safety.

IROS Conference 2024 Conference Paper

Bayesian Optimization for Sample-Efficient Policy Improvement in Robotic Manipulation

• Adrian Röfer
• Iman Nematollahi
• Tim Welschehold
• Wolfram Burgard
• Abhinav Valada

Sample efficient learning of manipulation skills poses a major challenge in robotics. While recent approaches demonstrate impressive advances in the type of task that can be addressed and the sensing modalities that can be incorporated, they still require large amounts of training data. Especially with regard to learning actions on robots in the real world, this poses a major problem due to the high costs associated with both demonstrations and real-world robot interactions. To address this challenge, we introduce BOpt-GMM, a hybrid approach that combines imitation learning with own experience collection. We first learn a skill model as a dynamical system encoded in a Gaussian Mixture Model from a few demonstrations. We then improve this model with Bayesian optimization building on a small number of autonomous skill executions in a sparse reward setting. We demonstrate the sample efficiency of our approach on multiple complex manipulation skills in both simulations and real-world experiments. Furthermore, we make the code and pre-trained models publicly available at http://bopt-gmm.cs.uni-freiburg.de.

IROS Conference 2024 Conference Paper

BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentation

• Jonas Schramm
• Niclas Vödisch
• Kürsat Petek
• Bangalore Ravi Kiran
• Senthil Kumar Yogamani
• Wolfram Burgard
• Abhinav Valada

Semantic scene segmentation from a bird’s-eye-view (BEV) perspective plays a crucial role in facilitating planning and decision-making for mobile robots. Although recent vision-only methods have demonstrated notable advancements in performance, they often struggle under adverse illumination conditions such as rain or nighttime. While active sensors offer a solution to this challenge, the prohibitively high cost of LiDARs remains a limiting factor. Fusing camera data with automotive radars poses a more inexpensive alternative but has received less attention in prior research. In this work, we aim to advance this promising avenue by introducing BEVCar, a novel approach for joint BEV object and map segmentation. The core novelty of our approach lies in first learning a point-based encoding of raw radar data, which is then leveraged to efficiently initialize the lifting of image features into the BEV space. We perform extensive experiments on the nuScenes dataset and demonstrate that BEVCar outperforms the current state of the art. Moreover, we show that incorporating radar information significantly enhances robustness in challenging environmental conditions and improves segmentation performance for distant objects. To foster future research, we provide the weather split of the nuScenes dataset used in our experiments, along with our code and trained models at http://bevcar.cs.uni-freiburg.de.

ICRA Conference 2024 Conference Paper

Collaborative Dynamic 3D Scene Graphs for Automated Driving

• Elias Greve
• Martin Büchner
• Niclas Vödisch
• Wolfram Burgard
• Abhinav Valada

Maps have played an indispensable role in enabling safe and automated driving. Although there have been many advances on different fronts ranging from SLAM to semantics, building an actionable hierarchical semantic representation of urban dynamic scenes and processing information from multiple agents are still challenging problems. In this work, we present Collaborative URBan Scene Graphs (CURB-SG) that enable higher-order reasoning and efficient querying for many functions of automated driving. CURB-SG leverages panoptic LiDAR data from multiple agents to build large-scale maps using an effective graph-based collaborative SLAM approach that detects inter-agent loop closures. To semantically decompose the obtained 3D map, we build a lane graph from the paths of ego agents and their panoptic observations of other vehicles. Based on the connectivity of the lane graph, we segregate the environment into intersecting and non-intersecting road areas. Subsequently, we construct a multi-layered scene graph that includes lane information, the position of static landmarks and their assignment to certain map sections, other vehicles observed by the ego agents, and the pose graph from SLAM including 3D panoptic point clouds. We extensively evaluate CURB-SG in urban scenarios using a photorealistic simulator. We release our code at http://curb. cs. uni-freiburg. de.

ICRA Conference 2024 Conference Paper

Few-Shot Panoptic Segmentation With Foundation Models

• Markus Käppeler
• Kürsat Petek
• Niclas Vödisch
• Wolfram Burgard
• Abhinav Valada

Current state-of-the-art methods for panoptic segmentation require an immense amount of annotated training data that is both arduous and expensive to obtain posing a significant challenge for their widespread adoption. Concurrently, recent breakthroughs in visual representation learning have sparked a paradigm shift leading to the advent of large foundation models that can be trained with completely unlabeled images. In this work, we propose to leverage such task-agnostic image features to enable few-shot panoptic segmentation by presenting Segmenting Panoptic Information with Nearly 0 labels (SPINO). In detail, our method combines a DINOv2 backbone with lightweight network heads for semantic segmentation and boundary estimation. We show that our approach, albeit being trained with only ten annotated images, predicts high-quality pseudo-labels that can be used with any existing panoptic segmentation method. Notably, we demonstrate that SPINO achieves competitive results compared to fully supervised baselines while using less than 0. 3% of the ground truth labels, paving the way for learning complex visual recognition tasks leveraging foundation models. To illustrate its general applicability, we further deploy SPINO on real-world robotic vision systems for both outdoor and indoor environments. To foster future research, we make the code and trained models publicly available at http://spino.cs.uni-freiburg.de.

IROS Conference 2024 Conference Paper

Improving Out-of-Distribution Generalization of Trajectory Prediction for Autonomous Driving via Polynomial Representations

• Yue Yao
• Shengchao Yan
• Daniel Goehring
• Wolfram Burgard
• Joerg Reichardt

Robustness against Out-of-Distribution (OoD) samples is a key performance indicator of a trajectory prediction model. However, the development and ranking of state-of-the-art (SotA) models are driven by their In-Distribution (ID) performance on individual competition datasets. We present an OoD testing protocol that homogenizes datasets and prediction tasks across two large-scale motion datasets. We introduce a novel prediction algorithm based on polynomial representations for agent trajectory and road geometry on both the input and output sides of the model. With a much smaller model size, training effort, and inference time, we reach near SotA performance for ID testing and significantly improve robustness in OoD testing. Within our OoD testing protocol, we further study two augmentation strategies of SotA models and their effects on model generalization. Highlighting the contrast between ID and OoD performance, we suggest adding OoD testing to the evaluation criteria of trajectory prediction models.

ICRA Conference 2024 Conference Paper

Learning Continuous Control with Geometric Regularity from Robot Intrinsic Symmetry

• Shengchao Yan
• Baohe Zhang
• Yuan Zhang 0027
• Joschka Boedecker
• Wolfram Burgard

Geometric regularity, which leverages data symmetry, has been successfully incorporated into deep learning architectures such as CNNs, RNNs, GNNs, and Transformers. While this concept has been widely applied in robotics to address the curse of dimensionality when learning from high-dimensional data, the inherent reflectional and rotational symmetry of robot structures has not been adequately explored. Drawing inspiration from cooperative multi-agent reinforcement learning, we introduce novel network structures for single-agent control learning that explicitly capture these symmetries. Moreover, we investigate the relationship between the geometric prior and the concept of Parameter Sharing in multi-agent reinforcement learning. Last but not the least, we implement the proposed framework in online and offline learning methods to demonstrate its ease of use. Through experiments conducted on various challenging continuous control tasks on simulators and real robots, we highlight the significant potential of the proposed geometric regularity in enhancing robot learning capabilities.

ICRA Conference 2024 Conference Paper

Open X-Embodiment: Robotic Learning Datasets and RT-X Models: Open X-Embodiment Collaboration

• Abby O'Neill
• Abdul Rehman
• Abhiram Maddukuri
• Abhishek Gupta 0004
• Abhishek Padalkar
• Abraham Lee
• Acorn Pooley
• Agrim Gupta

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train "generalist" X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. The project website is robotics-transformer-x. github.io.

ICLR Conference 2023 Conference Paper

Dynamic Update-to-Data Ratio: Minimizing World Model Overfitting

• Nicolai Dorka
• Tim Welschehold
• Wolfram Burgard

Early stopping based on the validation set performance is a popular approach to find the right balance between under- and overfitting in the context of supervised learning. However, in reinforcement learning, even for supervised sub-problems such as world model learning, early stopping is not applicable as the dataset is continually evolving. As a solution, we propose a new general method that dynamically adjusts the update to data (UTD) ratio during training based on under- and overfitting detection on a small subset of the continuously collected experience not used for training. We apply our method to DreamerV2, a state-of-the-art model-based reinforcement learning algorithm, and evaluate it on the DeepMind Control Suite and the Atari 100k benchmark. The results demonstrate that one can better balance under- and overestimation by adjusting the UTD ratio with our approach compared to the default setting in DreamerV2 and that it is competitive with an extensive hyperparameter search which is not feasible for many applications. Our method eliminates the need to set the UTD hyperparameter by hand and even leads to a higher robustness with regard to other learning-related hyperparameters further reducing the amount of necessary tuning.

IROS Conference 2023 Conference Paper

EvCenterNet: Uncertainty Estimation for Object Detection Using Evidential Learning

• Monish R. Nallapareddy
• Kshitij Sirohi
• Paulo Lilles Jorge Drews
• Wolfram Burgard
• Chih-Hong Cheng
• Abhinav Valada

Uncertainty estimation is crucial in safety-critical settings such as automated driving as it provides valuable information for several downstream tasks including high-level decision making and path planning. In this work, we propose EvCenterNet, a novel uncertainty-aware 2D object detection framework using evidential learning to directly estimate both classification and regression uncertainties. To employ evidential learning for object detection, we devise a combination of evidential and focal loss functions for the sparse heatmap inputs. We introduce class-balanced weighting for regression and heatmap prediction to tackle the class imbalance encountered by evidential learning. Moreover, we propose a learning scheme to actively utilize the predicted heatmap uncertainties to improve the detection performance by focusing on the most uncertain points. We train our model on the KITTI dataset and evaluate it on challenging out-of-distribution datasets including BDD100K and nuImages. Our experiments demonstrate that our approach improves the precision and minimizes the execution time loss in relation to the base model.

IROS Conference 2023 Conference Paper

FM-Loc: Using Foundation Models for Improved Vision-Based Localization

• Reihaneh Mirjalili
• Michael Krawez
• Wolfram Burgard

Visual place recognition is essential for vision-based robot localization and SLAM. Despite the tremendous progress made in recent years, place recognition in changing environments remains challenging. A promising approach to cope with appearance variations is to leverage high-level semantic features like objects or place categories. In this paper, we propose FM-Loc which is a novel image-based localization approach based on Foundation Models. Our approach uses the Large Language Model GPT-3 in combination with the Visual-Language Model CLIP to construct a semantic image descriptor that is robust to severe changes in scene geometry and camera viewpoint. We deploy CLIP to detect objects in an image, GPT-3 to suggest potential room labels based on the detected objects, and CLIP again to propose the most likely location label. The object labels and the scene label constitute an image descriptor that we use to calculate a similarity score between the query and database images. We validate our approach on real-world data that exhibit significant changes in camera viewpoints and object placement between the database and query trajectories. The experimental results demonstrate that our method is applicable to a wide range of indoor scenarios without the need for training or fine-tuning.

ICRA Conference 2023 Conference Paper

Grounding Language with Visual Affordances over Unstructured Data

• Oier Mees
• Jessica Borja-Diaz
• Wolfram Burgard

Recent works have shown that Large Language Models (LLMs) can be applied to ground natural language to a wide variety of robot skills. However, in practice, learning multi-task, language-conditioned robotic skills typically requires large-scale data collection and frequent human intervention to reset the environment or help correcting the current policies. In this work, we propose a novel approach to efficiently learn general-purpose language-conditioned robot skills from unstructured, offline and reset-free data in the real world by exploiting a self-supervised visuo-lingual affordance model, which requires annotating as little as 1% of the total data with language. We evaluate our method in extensive experiments both in simulated and real-world robotic tasks, achieving state-of-the-art performance on the challenging CALVIN benchmark and learning over 25 distinct visuomotor manipulation tasks with a single policy in the real world. We find that when paired with LLMs to break down abstract natural language instructions into subgoals via few-shot prompting, our method is capable of completing long-horizon, multi-tier tasks in the real world, while requiring an order of magnitude less data than previous approaches. Code and videos are available at http://hulc2.cs.uni-freiburg.de.

IROS Conference 2023 Conference Paper

Improving Deep Dynamics Models for Autonomous Vehicles with Multimodal Latent Mapping of Surfaces

• Johan Vertens
• Nicolai Dorka
• Tim Welschehold
• Michael Thompson
• Wolfram Burgard

The safe deployment of autonomous vehicles relies on their ability to effectively react to environmental changes. This can require maneuvering on varying surfaces which is still a difficult problem, especially for slippery terrains. To address this issue we propose a new approach that learns a surface-aware dynamics model by conditioning it on a latent variable vector storing surface information about the current location. A latent mapper is trained to update these latent variables during inference from multiple modalities on every traversal of the corresponding locations and stores them in a map. By training everything end-to-end with the loss of the dynamics model, we enforce the latent mapper to learn an update rule for the latent map that is useful for the subsequent dynamics model. We implement and evaluate our approach on a real miniature electric car. The results show that the latent map is updated to allow more accurate predictions of the dynamics model compared to a model without this information. We further show that by using this model, the driving performance can be improved on varying and challenging surfaces.

ICRA Conference 2023 Conference Paper

Uncertainty-aware LiDAR Panoptic Segmentation

• Kshitij Sirohi
• Sajad Marvi
• Daniel Büscher
• Wolfram Burgard

Modern autonomous systems often rely on LiDAR scanners, in particular for autonomous driving scenarios. In this context, reliable scene understanding is indispensable. Conventional learning-based methods generally try to achieve maximum performance for this task, while neglecting a proper estimation of the associated uncertainties. In this work, we introduce a novel approach for solving the task of uncertainty- aware panoptic segmentation using LiDAR point clouds. Our proposed EvLPSNet network is the first to solve this task efficiently in a sampling-free manner. It aims to predict per-point semantic and instance segmentations, together with per-point uncertainty estimates. Moreover, it incorporates methods that utilize the uncertainties to improve the segmentation performance. We provide several strong baselines combining state-of- the-art LiDAR panoptic segmentation networks with sampling- free uncertainty estimation techniques. Extensive evaluations show that we achieve the best performance on uncertainty- aware panoptic segmentation quality and calibration compared to these baselines. We make our code available at: https://github.com/kshitij3112/EvLPSNet

ICRA Conference 2023 Conference Paper

Visual Language Maps for Robot Navigation

• Chenguang Huang
• Oier Mees
• Andy Zeng 0001
• Wolfram Burgard

Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visual-language models pretrained on Internet-scale data (e. g. , image captions). While this is useful for matching images to natural language descriptions of object goals, it remains disjoint from the process of mapping the environment, so that it lacks the spatial precision of classic geometric maps. To address this problem, we propose VLMaps, a spatial map representation that directly fuses pretrained visual-language features with a 3D reconstruction of the physical world. VLMaps can be autonomously built from video feed on robots using standard exploration approaches and enables natural language indexing of the map without additional labeled data. Specifically, when combined with large language models (LLMs), VLMaps can be used to (i) translate natural language commands into a sequence of open-vocabulary navigation goals (which, beyond prior work, can be spatial by construction, e. g. , “in between the sofa and the TV” or “three meters to the right of the chair”) directly localized in the map, and (ii) can be shared among multiple robots with different embodiments to generate new obstacle maps on-the-fly (by using a list of obstacle categories). Extensive experiments carried out in simulated and real-world environments show that VLMaps enable navigation according to more complex language instructions than existing methods. Videos are available at https://vlmaps.github.io.

ICRA Conference 2022 Conference Paper

Affordance Learning from Play for Sample-Efficient Policy Learning

• Jessica Borja-Diaz
• Oier Mees
• Gabriel Kalweit
• Lukás Hermann
• Joschka Boedecker
• Wolfram Burgard

Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visual affordance model from human teleoperated play data and leverages it to enable efficient policy learning and motion planning. We combine model-based planning with model-free deep reinforcement learning (RL) to learn policies that favor the same object regions favored by people, while requiring minimal robot interactions with the environment. We evaluate our algorithm, Visual Affordance-guided Policy Optimization (VAPO), with both diverse simulation manipulation tasks and real world robot tidy-up experiments to demonstrate the effectiveness of our affordance-guided policies. We find that our policies train 4 × faster than the baselines and generalize better to novel objects because our visual affordance model can anticipate their affordance regions.

IROS Conference 2022 Conference Paper

OpenDR: An Open Toolkit for Enabling High Performance, Low Footprint Deep Learning for Robotics

• Nikolaos Passalis
• S. Pedrazzi
• Robert Babuska
• Wolfram Burgard
• D. Dias
• F. Ferro
• Moncef Gabbouj
• Ole Green

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often leads to the need of employing specialized hardware accelerators, further increase the effort and cost needed to employ DL models in robotics. Also, most of the existing DL methods follow a static inference paradigm, as inherited by the traditional computer vision pipelines, ignoring active perception, which can be employed to actively interact with the environment in order to increase perception accuracy. In this paper, we present the Open Deep Learning Toolkit for Robotics (OpenDR). OpenDR aims at developing an open, non-proprietary, efficient, and modular toolkit that can be easily used by robotics companies and research institutions to efficiently develop and deploy AI and cognition technologies to robotics applications, providing a solid step towards addressing the aforementioned challenges. We also detail the design choices, along with an abstract interface that was created to overcome these challenges. This interface can describe various robotic tasks, spanning beyond traditional DL cognition and inference, as known by existing frameworks, incorporating openness, homogeneity and robotics-oriented perception e. g. , through active perception, as its core design principles.

IROS Conference 2022 Conference Paper

Realistic Real-Time Simulation of RGB and Depth Sensors for Dynamic Scenarios using Augmented Image Based Rendering

• Johan Vertens
• Wolfram Burgard

Simulation remains one of the key methods for testing and validation of robotic perception systems and it also becomes increasingly important for training visuomotor policies for autonomous driving or manipulation. Further, as perception pipelines tend to leverage increasing amounts of modalities, it appears vital to simulate additional cues such as depth maps aside from RGB images. To align simulation with real-world observations, it is key to achieve realistic renderings of these maps, which includes the capability of rendering other dynamic objects in the scene. In this work, we propose an approach to real-time simulation of photo-realistic RGB images and sensor-realistic depth maps, that can contain dynamic objects at user-defined locations. Our method employs a selection of static samples of a pre-recorded database and multimodal cues from CAD models that are fused and warped to synthesize new imagery for a target camera pose. We show the efficacy of our method on newly proposed datasets recorded in a variety of different setups.

ICRA Conference 2022 Conference Paper

Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models

• Iman Nematollahi
• Erick Rosete-Beas
• Adrian Röfer
• Tim Welschehold
• Abhinav Valada
• Wolfram Burgard

$A$ core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner through trajectories rather than making instantaneous decisions individually at each time step. To this end, we propose the Soft Actor- Critic Gaussian Mixture Model (SAC-GMM), a novel hybrid approach that learns robot skills through a dynamical system and adapts the learned skills in their own trajectory distribution space through interactions with the environment. Our approach combines classical robotics techniques of learning from demonstration with the deep reinforcement learning framework and exploits their complementary nature. We show that our method utilizes sensors solely available during the execution of preliminarily learned skills to extract relevant features that lead to faster skill refinement. Extensive evaluations in both simulation and real-world environments demonstrate the effectiveness of our method in refining robot skills by leveraging physical interactions, high-dimensional sensory data, and sparse task completion rewards. Videos, code, and pre-trained models are available at http://sac-gmm.cs.uni-freiburg.de.

ICRA Conference 2022 Conference Paper

Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation

• Kürsat Petek
• Kshitij Sirohi
• Daniel Büscher
• Wolfram Burgard

Robust localization in dense urban scenarios using a low-cost sensor setup and sparse HD maps is highly relevant for the current advances in autonomous driving, but remains a challenging topic in research. We present a novel monocular localization approach based on a sliding-window pose graph that leverages predicted uncertainties for increased precision and robustness against challenging scenarios and per-frame failures. To this end, we propose an efficient multi-task uncertainty-aware perception module, which covers semantic segmentation, as well as bounding box detection, to enable the localization of vehicles in sparse maps, containing only lane borders and traffic lights. Further, we design differentiable cost maps that are directly generated from the estimated uncertainties. This opens up the possibility to minimize the reprojection loss of amorphous map elements in an association-free and uncertainty-aware manner. Extensive evaluation on the Lyft 5 dataset shows that, despite the sparsity of the map, our approach enables robust and accurate 6D localization in challenging urban scenarios using only monocular camera images and vehicle odometry.

IROS Conference 2022 Conference Paper

T3VIP: Transformation-based 3D Video Prediction

• Iman Nematollahi
• Erick Rosete-Beas
• Seyed Mahdi B. Azad
• Raghu Rajan
• Frank Hutter
• Wolfram Burgard

For autonomous skill acquisition, robots have to learn about the physical rules governing the 3D world dynamics from their own past experience to predict and reason about plausible future outcomes. To this end, we propose a transformation-based 3D video prediction (T3VIP) approach that explicitly models the 3D motion by decomposing a scene into its object parts and predicting their corresponding rigid transformations. Our model is fully unsupervised, captures the stochastic nature of the real world, and the observational cues in image and point cloud domains constitute its learning signals. To fully leverage all the 2D and 3D observational signals, we equip our model with automatic hyperparameter optimization (HPO) to interpret the best way of learning from them. To the best of our knowledge, our model is the first generative model that provides an RGB-D video prediction of the future for a static camera. Our extensive evaluation with simulated and real-world datasets demonstrates that our formulation leads to interpretable 3D models that predict future depth videos while achieving on-par performance with 2D models on RGB video prediction. Moreover, we demonstrate that our model outperforms 2D baselines on visuomotor control. Videos, code, dataset, and pre-trained models are available at http://t3vip.cs.uni-freiburg.de.

ICRA Conference 2020 Conference Paper

Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control

• Lukás Hermann
• Max Argus
• Andreas Eitel
• Artemij Amiranashvili
• Wolfram Burgard
• Thomas Brox

We propose Adaptive Curriculum Generation from Demonstrations (ACGD) for reinforcement learning in the presence of sparse rewards. Rather than designing shaped reward functions, ACGD adaptively sets the appropriate task difficulty for the learner by controlling where to sample from the demonstration trajectories and which set of simulation parameters to use. We show that training vision-based control policies in simulation while gradually increasing the difficulty of the task via ACGD improves the policy transfer to the real world. The degree of domain randomization is also gradually increased through the task difficulty. We demonstrate zero-shot transfer for two real-world manipulation tasks: pick-and-stow and block stacking. A video showing the results can be found at https://lmb.informatik.uni-freiburg.de/projects/curriculum/.

ICRA Conference 2020 Conference Paper

Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video

• Oier Mees
• Markus Merklinger
• Gabriel Kalweit
• Wolfram Burgard

Key challenges for the deployment of reinforcement learning (RL) agents in the real world are the discovery, representation and reuse of skills in the absence of a reward function. To this end, we propose a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view videos. Our method learns a general skill embedding independently from the task context by using an adversarial loss. We combine a metric learning loss, which utilizes temporal video coherence to learn a state representation, with an entropy-regularized adversarial skill-transfer loss. The metric learning loss learns a disentangled representation by attracting simultaneous viewpoints of the same observations and repelling visually similar frames from temporal neighbors. The adversarial skill-transfer loss enhances re-usability of learned skill embeddings over multiple task domains. We show that the learned embedding enables training of continuous control policies to solve novel tasks that require the interpolation of previously seen skills. Our extensive evaluation with both simulation and real world data demonstrates the effectiveness of our method in learning transferable skills from unlabeled interaction videos and composing them for new tasks. Code, pretrained models and dataset are available at http://robotskills.cs.uni-freiburg.de.

ICRA Conference 2020 Conference Paper

Camera Tracking in Lighting Adaptable Maps of Indoor Environments

• Tim Caselitz
• Michael Krawez
• Jugesh Sundram
• Mark Van Loock
• Wolfram Burgard

Tracking the pose of a camera is at the core of visual localization methods used in many applications. As the observations of a camera are inherently affected by lighting, it has always been a challenge for these methods to cope with varying lighting conditions. Thus far, this issue has mainly been approached with the intent to increase robustness by choosing lighting invariant map representations. In contrast, our work aims at explicitly exploiting lighting effects for camera tracking. To achieve this, we propose a lighting adaptable map representation for indoor environments that allows real-time rendering of the scene illuminated by an arbitrary subset of the lamps contained in the model. Our method for estimating the light setting from the current camera observation enables us to adapt the model according to the lighting conditions present in the scene. As a result, lighting effects like cast shadows do no longer act as disturbances that demand robustness but rather as beneficial features when matching observations against the map. We leverage these capabilities in a direct dense camera tracking approach and demonstrate its performance in realworld experiments in scenes with varying lighting conditions.

ICRA Conference 2020 Conference Paper

DeepTemporalSeg: Temporally Consistent Semantic Segmentation of 3D LiDAR Scans

• Ayush Dewan
• Wolfram Burgard

Understanding the semantic characteristics of the environment is a key enabler for autonomous robot operation. In this paper, we propose a deep convolutional neural network (DCNN) for semantic segmentation of a LiDAR scan into the classes car, pedestrian and bicyclist. This architecture is based on dense blocks and efficiently utilizes depth separable convolutions to limit the number of parameters while still maintaining the state-of-the-art performance. To make the predictions from the DCNN temporally consistent, we propose a Bayes filter based method. This method uses the predictions from the neural network to recursively estimate the current semantic state of a point in a scan. This recursive estimation uses the knowledge gained from previous scans, thereby making the predictions temporally consistent and robust towards isolated erroneous predictions. We compare the performance of our proposed architecture with other state-of-the-art neural network architectures and report substantial improvement. For the proposed Bayes filter approach, we shows results on various sequences in the KITTI tracking benchmark.

IROS Conference 2020 Conference Paper

Driving Through Ghosts: Behavioral Cloning with False Positives

• Andreas Bühler
• Adrien Gaidon
• Andrei Cramariuc
• Rares Ambrus
• Guy Rosman
• Wolfram Burgard

Safe autonomous driving requires robust detection of other traffic participants. However, robust does not mean perfect, and safe systems typically minimize missed detections at the expense of a higher false positive rate. This results in conservative and yet potentially dangerous behavior such as avoiding imaginary obstacles. In the context of behavioral cloning, perceptual errors at training time can lead to learning difficulties or wrong policies, as expert demonstrations might be inconsistent with the perceived world state. In this work, we propose a behavioral cloning approach that can safely leverage imperfect perception without being conservative. Our core contribution is a novel representation of perceptual uncertainty for learning to plan. We propose a new probabilistic birds-eye-view semantic grid to encode the noisy output of object perception systems. We then leverage expert demonstrations to learn an imitative driving policy using this probabilistic representation. Using the CARLA simulator, we show that our approach can safely overcome critical false positives that would otherwise lead to catastrophic failures or conservative behavior.

IROS Conference 2020 Conference Paper

Efficiency and Equity are Both Essential: A Generalized Traffic Signal Controller with Deep Reinforcement Learning

• Shengchao Yan
• Jingwei Zhang 0001
• Daniel Büscher
• Wolfram Burgard

Traffic signal controllers play an essential role in today's traffic system. However, the majority of them currently is not sufficiently flexible or adaptive to generate optimal traffic schedules. In this paper we present an approach to learn policies for signal controllers using deep reinforcement learning aiming for optimized traffic flow. Our method uses a novel formulation of the reward function that simultaneously considers efficiency and equity. We furthermore present a general approach to find the bound for the proposed equity factor and we introduce the adaptive discounting approach that greatly stabilizes learning and helps to maintain a high flexibility of green light duration. The experimental evaluations on both simulated and real-world data demonstrate that our proposed algorithm achieves state-of-the-art performance (previously held by traditional non-learning methods) on a wide range of traffic situations.

IROS Conference 2020 Conference Paper

HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images

• Johan Vertens
• Jannik Zürn
• Wolfram Burgard

The majority of learning-based semantic segmentation methods are optimized for daytime scenarios and favorable lighting conditions. Real-world driving scenarios, however, entail adverse environmental conditions such as nighttime illumination or glare which remain a challenge for existing approaches. In this work, we propose a multimodal semantic segmentation model that can be applied during daytime and nighttime. To this end, besides RGB images, we leverage thermal images, making our network significantly more robust. We avoid the expensive annotation of nighttime images by leveraging an existing daytime RGB-dataset and propose a teacher-student training approach that transfers the dataset's knowledge to the nighttime domain. We further adopt a domain adaptation method to align the learned feature spaces across the domains and propose a novel two-stage training scheme. Furthermore, due to a lack of thermal data for autonomous driving, we present a new dataset comprising over 20, 000 time-synchronized and aligned RGB-thermal image pairs. In this context, we also present a novel target-less calibration method that allows for automatic robust extrinsic and intrinsic thermal camera calibration. Among others, we use our new dataset to show state-of-the-art results for nighttime semantic segmentation.

IROS Conference 2020 Conference Paper

Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction

• Iman Nematollahi
• Oier Mees
• Lukás Hermann
• Wolfram Burgard

A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces. In order to scale learning through interaction to many objects and scenes, robots should be able to improve their own performance from real-world experience without requiring human supervision. To this end, we propose a novel approach for modeling the dynamics of a robot's interactions directly from unlabeled 3D point clouds and images. Unlike previous approaches, our method does not require ground-truth data associations provided by a tracker or any pre-trained perception network. To learn from unlabeled real-world interaction data, we enforce consistency of estimated 3D clouds, actions and 2D images with observed ones. Our joint forward and inverse network learns to segment a scene into salient object parts and predicts their 3D motion under the effect of applied actions. Moreover, our object-centric model outputs action-conditioned 3D scene flow, object masks and 2D optical flow as emergent properties. Our extensive evaluation both in simulation and with real-world data demonstrates that our formulation leads to effective, interpretable models that can be used for visuomotor control and planning. Videos, code and dataset are available at http://hind4sight.cs.uni-freiburg.de.

ICRA Conference 2020 Conference Paper

How to Keep HD Maps for Automated Driving Up To Date

• David Pannen
• Martin Liebner
• Wolfgang Hempel
• Wolfram Burgard

The current state of the art in automotive high definition digital (HD) map generation based on dedicated mapping vehicles cannot reliably keep these maps up to date because of the low traversal frequencies. Anonymized data collected from the fleet of vehicles that is already on the road provides a huge potential to outperform such state of the art solutions in robustness, safety and up-to-dateness of the map while achieving comparable quality. We thus present a solution based on crowdsourced data to (i) detect changes in the map independent of the type of change, (ii) automatically trigger map update jobs for parts of the map, and (iii) create and integrate map patches to keep the map always up to date. The developed solution provides a crowdsourced up to date HD map to make reliable prior information on lane markings and road edges available to automated driving functions.

IROS Conference 2020 Conference Paper

Improving Unimodal Object Recognition with Multimodal Contrastive Learning

• Johannes Meyer
• Andreas Eitel
• Thomas Brox
• Wolfram Burgard

Robots perceive their environment using various sensor modalities, e. g. , vision, depth, sound or touch. Each modality provides complementary information for perception. However, while it can be assumed that all modalities are available for training, when deploying the robot in real-world scenarios the sensor setup often varies. In order to gain flexibility with respect to the deployed sensor setup we propose a new multimodal approach within the framework of contrastive learning. In particular, we consider the case of learning from RGB-D images while testing with one modality available, i. e. , exclusively RGB or depth. We leverage contrastive learning to capture high-level information between different modalities in a compact feature embedding. We extensively evaluate our multimodal contrastive learning method on the Falling Things dataset and learn representations that outperform prior methods for RGB-D object recognition on the NYU-D dataset. Our code and details on the used datasets are available at: https://github.com/meyerjo/MultiModalContrastiveLearning.

IROS Conference 2020 Conference Paper

Learning Human-Aware Robot Navigation from Physical Interaction via Inverse Reinforcement Learning

• Marina Kollmitz
• Torsten Koller
• Joschka Boedecker
• Wolfram Burgard

Autonomous systems, such as delivery robots, are increasingly employed in indoor spaces to carry out activities alongside humans. This development poses the question of how robots can carry out their tasks while, at the same time, behaving in a socially compliant manner. Further, humans need to be able to communicate their preferences in a simple and intuitive way, and robots should adapt their behavior accordingly. This paper investigates force control as a natural means to interact with a mobile robot by pushing it along the desired trajectory. We employ inverse reinforcement learning (IRL) to learn from human interaction and adapt the robot behavior to its users' preferences, thereby eliminating the need to program the desired behavior manually. We evaluate our approach in a real-world experiment where test subjects interact with an autonomously navigating robot in close proximity. The results suggest that force control presents an intuitive means to interact with a mobile robot and show that our robot can quickly adapt to the test subjects' personal preferences.

ICRA Conference 2020 Conference Paper

Learning Object Placements For Relational Instructions by Hallucinating Scene Representations

• Oier Mees
• Alp Emek
• Johan Vertens
• Wolfram Burgard

Robots coexisting with humans in their environment and performing services for them need the ability to interact with them. One particular requirement for such robots is that they are able to understand spatial relations and can place objects in accordance with the spatial relations expressed by their user. In this work, we present a convolutional neural network for estimating pixelwise object placement probabilities for a set of spatial relations from a single input image. During training, our network receives the learning signal by classifying hallucinated high-level scene representations as an auxiliary task. Unlike previous approaches, our method does not require ground truth data for the pixelwise relational probabilities or 3D models of the objects, which significantly expands the applicability in practical applications. Our results obtained using real-world data and human-robot experiments demonstrate the effectiveness of our method in reasoning about the best way to place objects to reproduce a spatial relation. Videos of our experiments can be found at https://youtu.be/zaZkHTWFMKM.

IROS Conference 2020 Conference Paper

Modality-Buffet for Real-Time Object Detection

• Nicolai Dorka
• Johannes Meyer
• Wolfram Burgard

Real-time object detection in videos using lightweight hardware is a crucial component of many robotic tasks. Detectors using different modalities and with varying computational complexities offer different trade-offs. One option is to have a very lightweight model that can predict from all modalities at once for each frame. However, in some situations (e. g. , in static scenes) it might be better to have a more complex but more accurate model and to extrapolate from previous predictions for the frames coming in at processing time. We formulate this task as a sequential decision making problem and use reinforcement learning (RL) to generate a policy that decides from the RGB input which detector out of a portfolio of different object detectors to take for the next prediction. The objective of the RL agent is to maximize the accuracy of the predictions per image. We evaluate the approach on the Waymo Open Dataset and show that it exceeds the performance of each single detector.

IROS Conference 2020 Conference Paper

PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Driving

• Kuan-Hui Lee
• Matthew Kliemann
• Adrien Gaidon
• Jie Li 0031
• Chao Fang
• Sudeep Pillai
• Wolfram Burgard

In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning. However, this perception task is difficult, particularly for generic obstacles/objects, due to appearance and occlusion changes. To tackle this problem, we propose an end-to-end deep learning framework for LIDAR-based flow estimation in bird's eye view (BeV). Our method takes consecutive point cloud pairs as input and produces a 2-D BeV flow grid describing the dynamic state of each cell. The experimental results show that the proposed method not only estimates 2-D BeV flow accurately but also improves tracking performance of both dynamic and static objects.

ICRA Conference 2020 Conference Paper

Predicting Obstacle Footprints from 2D Occupancy Maps by Learning from Physical Interactions

• Marina Kollmitz
• Daniel Büscher
• Wolfram Burgard

Horizontally scanning 2D laser rangefinders are a popular approach for indoor robot localization because of the high accuracy of the sensors and the compactness of the required 2D maps. As the scanners in this configuration only provide information about one slice of the environment, the measurements typically do not capture the full extent of a large variety of obstacles, including chairs or tables. Accordingly, obstacle avoidance based on laser scanners mounted in such a fashion is likely to fail. In this paper, we propose a learning-based approach to predict collisions in 2D occupancy maps. Our approach is based on a convolutional neural network which is trained on a 2D occupancy map and collision events recorded with a bumper while the robot is navigating in its environment. As the network operates on local structures only, it can generalize to new environments. In addition, the robot can collect and integrate new collision examples after an initial training phase. Extensive experiments carried out in simulation and a realistic real-world environment confirm that our approach allows robots to learn from collision events to avoid collisions in the future.

ICRA Conference 2019 Conference Paper

A Maximum Likelihood Approach to Extract Finite Planes from 3-D Laser Scans

• Alexander Schaefer
• Johan Vertens
• Daniel Büscher
• Wolfram Burgard

Whether it is object detection, model reconstruction, laser odometry, or point cloud registration: Plane extraction is a vital component of many robotic systems. In this paper, we propose a strictly probabilistic method to detect finite planes in organized 3-D laser range scans. An agglomerative hierarchical clustering technique, our algorithm builds planes from bottom up, always extending a plane by the point that decreases the measurement likelihood of the scan the least. In contrast to most related methods, which rely on heuristics like orthogonal point-to-plane distance, we leverage the ray path information to compute the measurement likelihood. We evaluate our approach not only on the popular SegComp benchmark, but also provide a challenging synthetic dataset that overcomes SegComp's deficiencies. Both our implementation and the suggested dataset are available at [1].

ICRA Conference 2019 Conference Paper

Augmenting Action Model Learning by Non-Geometric Features

• Iman Nematollahi
• Daniel Kuhner
• Tim Welschehold
• Wolfram Burgard

Learning from demonstration is a powerful tool for teaching manipulation actions to a robot. It is, however, an unsolved problem how to consider knowledge about the world and action-induced reactions such as forces imposed onto the gripper or measured liquid levels during pouring without explicit and case dependent programming. In this paper, we present a novel approach to include such knowledge directly in form of measured features. To this end, we use action demonstrations together with external features to learn a motion encoded by a dynamic system in a Gaussian Mixture Model (GMM) representation. Accordingly, during action imitation, the system is able to couple the geometric trajectory of the motion to measured features in the scene. We demonstrate the feasibility of our approach with a broad range of external features in real-world robot experiments including a drinking, a handover and a pouring task.

IROS Conference 2019 Conference Paper

Combined Task and Action Learning from Human Demonstrations for Mobile Manipulation Applications

• Tim Welschehold
• Nichola Abdo
• Christian Dornhege
• Wolfram Burgard

Learning from demonstrations is a promising paradigm for transferring knowledge to robots. However, learning mobile manipulation tasks directly from a human teacher is a complex problem as it requires learning models of both the overall task goal and of the underlying actions. Additionally, learning from a small number of demonstrations often introduces ambiguity with respect to the intention of the teacher, making it challenging to commit to one model for generalizing the task to new settings. In this paper, we present an approach to learning flexible mobile manipulation action models and task goal representations from teacher demonstrations. Our action models enable the robot to consider different likely outcomes of each action and to generate feasible trajectories for achieving them. Accordingly, we leverage a probabilistic framework based on Monte Carlo tree search to compute sequences of feasible actions imitating the teacher intention in new settings without requiring the teacher to specify an explicit goal state. We demonstrate the effectiveness of our approach in complex tasks carried out in real-world settings.

ICRA Conference 2019 Conference Paper

HD Map Change Detection with a Boosted Particle Filter

• David Pannen
• Martin Liebner
• Wolfram Burgard

In this paper, we present a change detection algorithm that can run in real time as part of a backend-based stream processing pipeline. It can process the floating car data collected by series-production vehicles to detect changes in an automotive high definition digital (HD) map used for automated driving. The algorithm uses a particle filter approach with odometry, GNSS and landmark readings to localize the vehicle within the digital map. While all particles together represent the probability distribution for the vehicle's position at a given time, each individual particle also serves as a hypothesis about the vehicle's position. This is used to compute various metrics for how well the current sensor readings match the world model encoded in the HD map. The different metrics are evaluated by a number of weak classifiers that are used as input for a trained Adaboost classifier. The achievable detection rate of a single vehicle is then compared to that of a simple crowd-based approach, where each vehicle votes on whether or not the current section of the road has changed.

IROS Conference 2019 Conference Paper

Lane Marking Learning based on Crowdsourced Data

• David Pannen
• Martin Liebner
• Wolfram Burgard

In this paper, we propose a new algorithm that derives lane marking maps from crowdsourced data. We process the data in four steps: (i) We make use of a point landmark map and, if available, an existent lane marking map for trajectory optimization and alignment, (ii) use a custom DBSCAN variant to cluster observations that belong to the same lane marking, (iii) apply a novel graph fitting approach to extract lane marking dashes, lines and even complex structures such as splits and merges and (iv) optimize the graph geometry with domain knowledge. The process of point-landmark- and lane marking-based trajectory alignment and the lane marking derivation is repeated iteratively to improve the results. Evaluation is carried out on a 9km highway section by comparison with high accuracy aerial photographs and manually labeled ground truth lane markings.

ICRA Conference 2019 Conference Paper

Modeling and Planning Manipulation in Dynamic Environments

• Philipp S. Schmitt
• Florian Wirnshofer
• Kai M. Wurm
• Georg von Wichert
• Wolfram Burgard

In this paper we propose a new model for sequential manipulation tasks that also considers robot dynamics and time-variant environments. From this model we automatically derive constraint-based controllers and use them as steering functions in a kinodynamic manipulation planner. The resulting plan is not a trajectory but a sequence of controllers that react online to disturbances. We validated our approach in simulation and on a real robot. In the experiments our approach plans and executes dual-robot manipulation tasks with online collision avoidance and reactions to estimates of object poses.

IROS Conference 2019 Conference Paper

Planning Reactive Manipulation in Dynamic Environments

• Philipp S. Schmitt
• Florian Wirnshofer
• Kai M. Wurm
• Georg von Wichert
• Wolfram Burgard

When robots perform manipulation tasks, they need to determine their own movement, as well as how to make and break contact with objects in their environment. Reasoning about the motions of robots and objects simultaneously leads to a constrained planning problem in a high-dimensional state-space. Additionally, when environments change dynamically motions must be computed in real-time. To this end, we propose a feedback planner for manipulation. We model manipulation as constrained motion and use this model to automatically derive a set of constraint-based controllers. These controllers are used in a switching-control scheme, where the active controller is chosen by a reinforcement learning agent. Our approach is capable of addressing tasks with second-order dynamics, closed kinematic chains, and time-variant environments. We validated our approach in simulation and on a real, dual-arm robot. Extensive simulation of three distinct robots and tasks show a significant increase in robustness compared to a previous approach.

IROS Conference 2019 Conference Paper

Robot Localization in Floor Plans Using a Room Layout Edge Extraction Network

• Federico Boniardi
• Abhinav Valada
• Rohit Mohan
• Tim Caselitz
• Wolfram Burgard

Indoor localization is one of the crucial enablers for deployment of service robots. Although several successful techniques for indoor localization have been proposed, the majority of them relies on maps generated from data gathered with the same sensor modality used for localization. Typically, tedious labor by experts is needed to acquire this data, thus limiting the readiness of the system as well as its ease of installation for inexperienced operators. In this paper, we propose a memory and computationally efficient monocular camera-based localization system that allows a robot to estimate its pose given an architectural floor plan. Our method employs a convolutional neural network to predict room layout edges from a single camera image and estimates the robot pose using a particle filter that matches the extracted edges to the given floor plan. We evaluate our localization system using multiple real-world experiments and demonstrate that it has the robustness and accuracy required for reliable indoor navigation.

IROS Conference 2019 Conference Paper

Robust, Compliant Assembly with Elastic Parts and Model Uncertainty

• Florian Wirnshofer
• Philipp S. Schmitt
• Philine Meister
• Georg von Wichert
• Wolfram Burgard

In this paper, we present an approach to generate robot motions for robust parts assembly. The computation of motions for parts assembly usually requires an exact model of all relevant objects. Generating detailed object models, including friction and dynamics, is often complex and time-consuming, especially in the context of elastic parts. In addition, executing motions on real hardware will usually introduce further uncertainty. For this reason, we propose an approach that is inherently robust against model parameter uncertainties and unknown characteristics of elastic parts. Our planner explicitly takes into account the internal states of articulated objects, as well as uncertain model parameters, by constructing a search tree in the belief-parameter-space. It yields successful assembly motions from coarse object models and thus eliminates the need for detailed parameter tuning. We evaluated our approach with respect to four assembly tasks. Extensive simulations show that our planner significantly increases the success-rate compared to previous approaches. Numerous experiments on a real robot confirm the simulated results.

IROS Conference 2019 Conference Paper

Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics

• Oier Mees
• Maxim Tatarchenko
• Thomas Brox
• Wolfram Burgard

We present a convolutional neural network for joint 3D shape prediction and viewpoint estimation from a single input image. During training, our network gets the learning signal from a silhouette of an object in the input image-a form of self-supervision. It does not require ground truth data for 3D shapes and the viewpoints. Because it relies on such a weak form of supervision, our approach can easily be applied to real-world data. We demonstrate that our method produces reasonable qualitative and quantitative results on natural images for both shape estimation and viewpoint prediction. Unlike previous approaches, our method does not require multiple views of the same object instance in the dataset, which significantly expands the applicability in practical robotics scenarios. We showcase it by using the hallucinated shapes to improve the performance on the task of grasping real-world objects both in simulation and with a PR2 robot.

IROS Conference 2019 Conference Paper

Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction

• Andreas Eitel
• Nico Hauff
• Wolfram Burgard

Instance segmentation of unknown objects from images is regarded as relevant for several robot skills including grasping, tracking and object sorting. Recent results from computer vision have shown that large hand-labeled datasets enable high segmentation performance. To overcome the time-consuming process of manually labeling data for new environments, we present a transfer learning approach for robots that learn to segment objects by interacting with their environment in a self-supervised manner. Our robot pushes unknown objects on a table and uses information from optical flow to create training labels given by object masks. To achieve this, we fine-tune an existing DeepMask instance segmentation network on the self-labeled training data acquired by the robot. We evaluate our trained network (SelfDeepMask) on a set of real images showing challenging and cluttered scenes with novel objects. Here, SelfDeepMask outperforms the DeepMask network trained on the COCO dataset by 8. 6% in average precision.

ICRA Conference 2019 Conference Paper

State Estimation in Contact-Rich Manipulation

• Florian Wirnshofer
• Philipp S. Schmitt
• Philine Meister
• Georg von Wichert
• Wolfram Burgard

This paper introduces a Bayesian state estimator for contact-rich manipulation tasks with application in non-prehensile manipulation, industrial assembly or in-hand localization. The core idea of our approach is to explicitly model both the contact dynamics and a torque-based robot controller as part of the underlying system model. Our approach is capable of estimating the state of movable objects for various robot kinematics and geometries of robots and objects. This includes complex scenarios with multiple robots, multiple objects and articulated objects. We have validated our approach in simulation and on a physical robot. The experiments show that multimodal distributions of six degrees of freedom object poses can be accurately tracked in real-time in a complex manipulation scenario.

ICRA Conference 2018 Conference Paper

3D Human Pose Estimation in RGBD Images for Robotic Task Learning

• Christian Zimmermann 0001
• Tim Welschehold
• Christian Dornhege
• Wolfram Burgard
• Thomas Brox

We propose an approach to estimate 3D human pose in real world units from a single RGBD image and show that it exceeds performance of monocular 3D pose estimation approaches from color as well as pose estimation exclusively from depth. Our approach builds on robust human keypoint detectors for color images and incorporates depth for lifting into 3D. We combine the system with our learning from demonstration framework to instruct a service robot without the need of markers. Experiments in real world settings demonstrate that our approach enables a PR2 robot to imitate manipulation actions observed from a human teacher.

IROS Conference 2018 Conference Paper

A Maximum Likelihood Approach to Extract Polylines from 2-D Laser Range Scans

• Alexander Schaefer
• Daniel Büscher
• Lukas Luft
• Wolfram Burgard

Man-made environments such as households, offices, or factory floors are typically composed of linear structures. Accordingly, polylines are a natural way to accurately represent their geometry. In this paper, we propose a novel probabilistic method to extract polylines from raw 2-D laser range scans. The key idea of our approach is to determine a set of polylines that maximizes the likelihood of a given scan. In extensive experiments carried out on publicly available real-world datasets and on simulated laser scans, we demonstrate that our method substantially outperforms existing state-of-the-art approaches in terms of accuracy, while showing comparable computational requirements. Our implementation is available under https://github.com/acschaefer/ple.

IROS Conference 2018 Conference Paper

Building Dense Reflectance Maps of Indoor Environments Using an RGB-D Camera

• Michael Krawez
• Tim Caselitz
• Daniel Büscher
• Mark Van Loock
• Wolfram Burgard

The ability to build models of the environment is an essential prerequisite for many robotic applications. In recent years, mapping of dense surface geometry using RGB-D cameras has seen extensive progress. Many approaches build colored models, typically directly using the intensity values provided by the camera. Unfortunately, these intensities are inherently affected by illumination. Therefore, the resulting maps only represent the environment for one specific lighting condition. To overcome this limitation, we propose to build reflectance maps that are invariant against changes in lighting. Our approach estimates the diffuse reflectance of a surface by recovering its radiosity and the corresponding irradiance. As imperfections in this process can significantly degrade the reflectance estimate, we remove outliers in the high dynamic range radiosity estimation and propose a method to refine the reflectance estimate. Our system implements the whole pipeline for offline reconstruction of dense reflectance maps including the segmentation of light emitters in the scene. We demonstrate the applicability of our approach in real-world experiments under varying lighting conditions.

IROS Conference 2018 Conference Paper

Closed-Loop Robot Task Planning Based on Referring Expressions

• Daniel Kuhner
• Johannes Aldinger
• Felix Burget
• Moritz Göbelbecker
• Wolfram Burgard
• Bernhard Nebel

Increasing the accessibility of autonomous robots also for inexperienced users requires user-friendly and high-level control opportunities of robotic systems. While automated planning is able to decompose a complex task into a sequence of steps which reaches an intended goal, it is difficult to formulate such a goal without knowing the internals of the planning system and the exact capabilities of the robot. This becomes even more important in dynamic environments in which manipulable objects are subject to change. In this paper, we present an adaptive control interface which allows users to specify goals based on an internal world model by incrementally building referring expressions to the objects in the world. We consider fetch-and-carry tasks and automatically deduce potential high-level goals from the world model to make them available to the user. Based on its perceptions our system can react to changes in the environment by adapting the goal formulation within the domain-independent planning system.

IROS Conference 2018 Conference Paper

Coupling Mobile Base and End-Effector Motion in Task Space

• Tim Welschehold
• Christian Dornhege
• Fabian Paus
• Tamim Asfour
• Wolfram Burgard

Dynamic systems are a practical alternative to motion planning in executing robot actions. They are of particular interest in Learning from Demonstration, as here we aim to carry out actions in a certain fashion, without a model or in-depth knowledge about the world, which might be difficult to achieve with a planner. Using model-based dynamic systems in task space enables robots to flexibly reproduce demonstrated actions. Nevertheless, when dealing with mobile manipulators, we face the challenge of including the kinematic constraints of the robot in the action models. In this paper we propose to couple robot base and end-effector motions generated by arbitrary dynamical systems modulating the base velocity, while respecting the robots kinematic design. To this end we learn an approximation of the inverse reachability in closed form. In real-world robot experiments we demonstrate that we are able to maintain kinematically feasible trajectories in the presence of obstacles and in configurations differing profoundly from the training scene.

ICRA Conference 2018 Conference Paper

Deep Auxiliary Learning for Visual Localization and Odometry

• Abhinav Valada
• Noha Radwan
• Wolfram Burgard

Localization is an indispensable component of a robot's autonomy stack that enables it to determine where it is in the environment, essentially making it a precursor for any action execution or planning. Although convolutional neural networks have shown promising results for visual localization, they are still grossly outperformed by state-of-the-art local feature-based techniques. In this work, we propose VLocNet, a new convolutional neural network architecture for 6-DoF global pose regression and odometry estimation from consecutive monocular images. Our multitask model incorporates hard parameter sharing, thus being compact and enabling real-time inference, in addition to being end-to-end trainable. We propose a novel loss function that utilizes auxiliary learning to leverage relative pose information during training, thereby constraining the search space to obtain consistent pose estimates. We evaluate our proposed VLocNet on indoor as well as outdoor datasets and show that even our single task model exceeds the performance of state-of-the-art deep architectures for global localization, while achieving competitive performance for visual odometry estimation. Furthermore, we present extensive experimental evaluations utilizing our proposed Geometric Consistency Loss that show the effectiveness of multitask learning and demonstrate that our model is the first deep learning technique to be on par with, and in some cases outperforms state-of-the-art SIFT-based approaches.

ICRA Conference 2018 Conference Paper

DPDB-Net: Exploiting Dense Connections for Convolutional Encoders

• Gabriel L. Oliveira
• Wolfram Burgard
• Thomas Brox

Densely connected networks for classification enable feature exploration and result in state-of-the-art performance on multiple classification tasks. The alternative to dense networks is the residual network which enables feature re-usage. In this work, we combine these orthogonal concepts for encoder-decoder architectures, which we call Dual-Path Dense-Block Network (DPDB-Net). We introduce a dense block which incorporates feature re-usage and new feature exploration in the encoder. Moreover, we discuss that feature re-usage by the residual network architecture leads to a feature map explosion in the decoder and, thus, is not advantageous in this part of the network. We evaluated our proposed architecture in multiple segmentation tasks and report state-of-the-art performance on the Freiburg Forest dataset and competitive results on the Cam Vid dataset.

IROS Conference 2018 Conference Paper

Guess What I Attend: Interface-Free Object Selection Using Brain Signals

• Henrich Kolkhorst
• Michael Tangermann
• Wolfram Burgard

Interpreting the brain activity to identify user goals or to ground a robot's hypotheses about them is a promising direction for non-intrusive and intuitive communication. Such a capability can be of particular relevance in the context of human-robot cooperation scenarios. This paper proposes a novel approach to utilize the natural brain responses to highlighted objects in the scene for object selection. By this, it circumvents the need for additional interfaces or user training. Our approach uses methods from information geometry to classify the target/non-target response of these event-related potentials. Online experiments carried out with a real robot demonstrate an accurate detection of target objects solely based on the user's attention.

IROS Conference 2018 Conference Paper

Learning a Local Feature Descriptor for 3D LiDAR Scans

• Ayush Dewan
• Tim Caselitz
• Wolfram Burgard

Robust data association is necessary for virtually every SLAM system and finding corresponding points is typically a preprocessing step for scan alignment algorithms. Traditionally, handcrafted feature descriptors were used for these problems but recently learned descriptors have been shown to perform more robustly. In this work, we propose a local feature descriptor for 3D LiDAR scans. The descriptor is learned using a Convolutional Neural Network (CNN). Our proposed architecture consists of a Siamese network for learning a feature descriptor and a metric learning network for matching the descriptors. We also present a method for estimating local surface patches and obtaining ground-truth correspondences. In extensive experiments, we compare our learned feature descriptor with existing 3D local descriptors and report highly competitive results for multiple experiments in terms of matching accuracy and computation time.

IROS Conference 2018 Conference Paper

Learning to Pour using Deep Deterministic Policy Gradients

• Chau Do
• Camilo Gordillo
• Wolfram Burgard

Pouring is a fundamental skill for robots in both domestic and industrial environments. Ideally, a robot should be able to pour with high accuracy to specific, pre-defined heights and without spilling. However, due to the complex dynamics of liquids, it is difficult to learn how to pour to achieve these goals. In this paper we present an approach to learn a policy for pouring using Deep Deterministic Policy Gradients (DDPG). We remove the need for collecting training experiences on a real robot, by using a state-of-the-art liquid simulator, which allows for learning the liquid dynamics. We show through our experiments, performed with a PR2 robot, that it is possible to successfully transfer the learned policy to a real robot and even apply it to different liquids.

ICRA Conference 2018 Conference Paper

Mapping with Dynamic-Object Probabilities Calculated from Single 3D Range Scans

• Philipp Ruchti
• Wolfram Burgard

Various autonomous robotic systems require maps for robust and safe navigation. Particularly when robots are employed in dynamic environments, accurate knowledge about which components of the robot perceptions belong to dynamic and static aspects in the environment can greatly improve navigation functions. In this paper we propose a novel method for building 3D grid maps using laser range data in dynamic environments. Our approach uses a neural network to estimate the pointwise probability of a point belonging to a dynamic object. The output from our network is fed to the mapping module for building a 3D grid map containing only static parts of the environment. We present experimental results obtained by training our neural network using the KITTI dataset and evaluating it in a mapping process using our own dataset. In extensive experiments, we show that maps generated using the proposed probability about dynamic objects increases the accuracy of the resulting maps.

ICRA Conference 2018 Conference Paper

Optimization Beyond the Convolution: Generalizing Spatial Relations with End-to-End Metric Learning

• Philipp Jund
• Andreas Eitel
• Nichola Abdo
• Wolfram Burgard

To operate intelligently in domestic environments, robots require the ability to understand arbitrary spatial relations between objects and to generalize them to objects of varying sizes and shapes. In this work, we present a novel end-to-end approach to generalize spatial relations based on distance metric learning. We train a neural network to transform 3D point clouds of objects to a metric space that captures the similarity of the depicted spatial relations, using only geometric models of the objects. Our approach employs gradient-based optimization to compute object poses in order to imitate an arbitrary target relation by reducing the distance to it under the learned metric. Our results based on simulated and real-world experiments show that the proposed method enables robots to generalize spatial relations to unknown objects over a continuous spectrum.

ICRA Conference 2018 Conference Paper

Robust, Compliant Assembly via Optimal Belief Space Planning

• Florian Wirnshofer
• Philipp S. Schmitt
• Wendelin Feiten
• Georg von Wichert
• Wolfram Burgard

In automated manufacturing, robots must reliably assemble parts of various geometries and low tolerances. Ideally, they plan the required motions autonomously. This poses a substantial challenge due to high-dimensional state spaces and non-linear contact-dynamics. Furthermore, object poses and model parameters, such as friction, are not exactly known and a source of uncertainty. The method proposed in this paper models the task of parts assembly as a belief space planning problem over an underlying impedance-controlled, compliant system. To solve this planning problem we introduce an asymptotically optimal belief space planner by extending an optimal, randomized, kinodynamic motion planner to nondeterministic domains. Under an expansiveness assumption we establish probabilistic completeness and asymptotic optimality. We validate our approach in thorough, simulated and realworld experiments of multiple assembly tasks. The experiments demonstrate our planner's ability to reliably assemble objects, solely based on CAD models as input.

ICRA Conference 2018 Conference Paper

Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning

• Lei Tai
• Jingwei Zhang 0001
• Ming Liu 0001
• Wolfram Burgard

We present an approach for mobile robots to learn to navigate in dynamic environments with pedestrians via raw depth inputs, in a socially compliant manner. To achieve this, we adopt a generative adversarial imitation learning (GAIL) strategy, which improves upon a pre-trained behavior cloning policy. Our approach overcomes the disadvantages of previous methods, as they heavily depend on the full knowledge of the location and velocity information of nearby pedestrians, which not only requires specific sensors, but also the extraction of such state information from raw sensory input could consume much computation time. In this paper, our proposed GAIL-based model performs directly on raw depth inputs and plans in real-time. Experiments show that our GAIL-based approach greatly improves the safety and efficiency of the behavior of mobile robots from pure behavior cloning. The real-world deployment also shows that our method is capable of guiding autonomous vehicles to navigate in a socially compliant manner directly through raw depth inputs. In addition, we release a simulation plugin for modeling pedestrian behaviors based on the social force model.

YNIMG Journal 2018 Journal Article

The dynamics of error processing in the human brain as reflected by high-gamma activity in noninvasive and intracranial EEG

• Martin Völker
• Lukas D.J. Fiederer
• Sofie Berberich
• Jiří Hammer
• Joos Behncke
• Pavel Kršek
• Martin Tomášek
• Petr Marusič

Error detection in motor behavior is a fundamental cognitive function heavily relying on local cortical information processing. Neural activity in the high-gamma frequency band (HGB) closely reflects such local cortical processing, but little is known about its role in error processing, particularly in the healthy human brain. Here we characterize the error-related response of the human brain based on data obtained with noninvasive EEG optimized for HGB mapping in 31 healthy subjects (15 females, 16 males), and additional intracranial EEG data from 9 epilepsy patients (4 females, 5 males). Our findings reveal a multiscale picture of the global and local dynamics of error-related HGB activity in the human brain. On the global level as reflected in the noninvasive EEG, the error-related response started with an early component dominated by anterior brain regions, followed by a shift to parietal regions, and a subsequent phase characterized by sustained parietal HGB activity. This phase lasted for more than 1 s after the error onset. On the local level reflected in the intracranial EEG, a cascade of both transient and sustained error-related responses involved an even more extended network, spanning beyond frontal and parietal regions to the insula and the hippocampus. HGB mapping appeared especially well suited to investigate late, sustained components of the error response, possibly linked to downstream functional stages such as error-related learning and behavioral adaptation. Our findings establish the basic spatio-temporal properties of HGB activity as a neural correlate of error processing, complementing traditional error-related potential studies.

ICRA Conference 2018 Conference Paper

Whole-Body Sensory Concept for Compliant Mobile Robots

• Marina Kollmitz
• Daniel Büscher
• Tobias Schubert 0002
• Wolfram Burgard

Most of the conventional approaches to mobile robot navigation avoid any kind of contact with the environment or with humans. As nowadays distance sensors typically have a limited - and often only two-dimensional - field of view, collisions with the environment or contacts with humans cannot be fully avoided in practical mobile robot applications. On the other hand, direct physical contact can be used for intuitive communication between a robot and humans. In this paper, we present a whole-body sensory concept based on a 6-DoF force-torque sensor to perceive physical interaction between the robot and humans. To distinguish between external contact and disturbance forces that result from the motion of the mobile platform or oscillations, we present a novel model-free filtering approach based on a neural network. In extensive experiments carried out with our robot Canny we demonstrate the effectiveness and advantages of the neural network approach, which clearly outperforms a classical model-based one.

ICRA Conference 2017 Conference Paper

AdapNet: Adaptive semantic segmentation in adverse environmental conditions

• Abhinav Valada
• Johan Vertens
• Ankit Dhall
• Wolfram Burgard

Robust scene understanding of outdoor environments using passive optical sensors is a onerous and essential task for autonomous navigation. The problem is heavily characterized by changing environmental conditions throughout the day and across seasons. Robots should be equipped with models that are impervious to these factors in order to be operable and more importantly to ensure safety in the real-world. In this paper, we propose a novel semantic segmentation architecture and the convoluted mixture of deep experts (CMoDE) fusion technique that enables a multi-stream deep neural network to learn features from complementary modalities and spectra, each of which are specialized in a subset of the input space. Our model adaptively weighs class-specific features of expert networks based on the scene condition and further learns fused representations to yield robust segmentation. We present results from experimentation on three publicly available datasets that contain diverse conditions including rain, summer, winter, dusk, fall, night and sunset, and show that our approach exceeds the state-of-the-art. In addition, we evaluate the performance of autonomously traversing several kilometres of a forested environment using only the segmentation for perception.

IROS Conference 2017 Conference Paper

An online system for tracking the performance of Parkinson's patients

• Andreas Kuhner
• Tobias Schubert 0002
• Christoph Maurer
• Wolfram Burgard

An objective performance measure for movement tasks is widely regarded as having utmost relevance for the therapy of movement disorders. Existing systems typically rely on human experts, which is known to produce substantial inter- and intra-rater variability. Present solutions are either based on simple features or invasive motion capture techniques. They typically work on a specific motion task only and fail to generalize to other tasks. In addition, they often require manual offline pre- and post-processing. In this paper we present a novel approach to compute a continuous and objective performance measure online during a patient session, without tedious and time-consuming pre- or post-processing steps. Our approach is able to generalize between different motion capture devices and different motion tasks. It runs on live motion data extracted with a non-invasive marker-less off-the-shelf vision-based tracking system as well as on data extracted from an inertial measurement unit suit. In the experiments we show that our approach is competitive with an offline approach as well as with the Unified Parkinson's Disease Rating Scale. Our approach is robust with respect to motion execution speed and it outperforms the offline approach regarding movement task generalization. We show promising results to track the current state of a Parkinson's subject online during a therapy session.

IROS Conference 2017 Conference Paper

Closed-form full map posteriors for robot localization with lidar sensors

• Lukas Luft
• Alexander Schaefer
• Tobias Schubert 0002
• Wolfram Burgard

A popular class of lidar-based grid mapping algorithms computes for each map cell the probability that it reflects an incident laser beam. These algorithms typically determine the map as the set of reflection probabilities that maximizes the likelihood of the underlying laser data and do not compute the full posterior distribution over all possible maps. Thereby, they discard crucial information about the confidence of the estimate. The approach presented in this paper preserves this information by determining the full map posterior. In general, this problem is hard because distributions over real-valued quantities can possess infinitely many dimensions. However, for two state-of-the-art beam-based lidar models, our approach yields closed-form map posteriors that possess only two parameters per cell. Even better, these posteriors come for free, in the sense that they use the same parameters as the traditional approaches, without the need for additional computations. An important use case for grid maps is robot localization, which we formulate as Bayesian filtering based on the closed-form map posterior rather than based on a single map. The resulting measurement likelihoods can also be expressed in closed form. In simulations and extensive real-world experiments, we show that leveraging the full map posterior improves the localization accuracy compared to approaches that use the most likely map.

IROS Conference 2017 Conference Paper

Deep regression for monocular camera-based 6-DoF global localization in outdoor environments

• Tayyab Naseer
• Wolfram Burgard

Precise localization of robots is imperative for their safe and autonomous navigation in both indoor and outdoor environments. In outdoor scenarios, the environment typically undergoes significant perceptual changes and requires robust methods for accurate localization. Monocular camera-based approaches provide an inexpensive solution to such challenging problems compared to 3D LiDAR-based methods. Recently, approaches have leveraged deep convolutional neural networks (CNNs) to perform place recognition and they turn out to outperform traditional handcrafted features under challenging perceptual conditions. In this paper, we propose an approach for directly regressing a 6-DoF camera pose using CNNs and a single monocular RGB image. We leverage the idea of transfer learning for training our network as this technique has shown to perform better when the number of training samples are not very high. Furthermore, we propose novel data augmentation in 3D space for additional pose coverage which leads to more accurate localization. In contrast to the traditional visual metric localization approaches, our resulting map size is constant with respect to the database. During localization, our approach has a constant time complexity of O(1) and is independent of the database size and runs in real-time at ~80 Hz using a single GPU. We show the localization accuracy of our approach on publicly available datasets and that it outperforms CNN-based state-of-the-art methods.

IROS Conference 2017 Conference Paper

Deep reinforcement learning with successor features for navigation across similar environments

• Jingwei Zhang 0001
• Jost Tobias Springenberg
• Joschka Boedecker
• Wolfram Burgard

In this paper we consider the problem of robot navigation in simple maze-like environments where the robot has to rely on its onboard sensors to perform the navigation task. In particular, we are interested in solutions to this problem that do not require localization, mapping or planning. Additionally, we require that our solution can quickly adapt to new situations (e. g. , changing navigation goals and environments). To meet these criteria we frame this problem as a sequence of related reinforcement learning tasks. We propose a successor-feature-based deep reinforcement learning algorithm that can learn to transfer knowledge from previously mastered navigation tasks to new problem instances. Our algorithm substantially decreases the required learning time after the first task instance has been solved, which makes it easily adaptable to changing environments. We validate our method in both simulated and real robot experiments with a Robotino and compare it to a set of baseline methods including classical planning-based navigation.

IROS Conference 2017 Conference Paper

Deep semantic classification for 3D LiDAR data

• Ayush Dewan
• Gabriel L. Oliveira
• Wolfram Burgard

Robots are expected to operate autonomously in dynamic environments. Understanding the underlying dynamic characteristics of objects is a key enabler for achieving this goal. In this paper, we propose a method for pointwise semantic classification of 3D LiDAR data into three classes: non-movable, movable and dynamic. We concentrate on understanding these specific semantics because they characterize important information required for an autonomous system. To learn the distinction between movable and non-movable points in the environment, we introduce an approach based on deep neural network and for detecting the dynamic points, we estimate pointwise motion. We propose a Bayes filter framework for combining the learned semantic cues with the motion cues to infer the required semantic classification. In extensive experiments, we compare our approach with other methods on a standard benchmark dataset and report competitive results in comparison to the existing state-of-the-art. Furthermore, we show an improvement in the classification of points by combining the semantic cues retrieved from the neural network with the motion cues.

ICRA Conference 2017 Conference Paper

Efficient path planning for mobile robots with adjustable wheel positions

• Freya Fleckenstein
• Christian Dornhege
• Wolfram Burgard

Efficient navigation planning for mobile robots in complex environments is a challenging problem. In this paper we consider the path planning problem for mobile robots with adjustable relative wheel positions, which further increase the navigation capabilities. In particular we account for changes of these relative wheel positions during planning time, thus fully leveraging the capabilities of the robot. Whereas these additional degrees of freedom increase flexibility, they introduce a more challenging planning problem. The approach proposed in this paper is built upon a search-based planner. We describe how to flexibly integrate joint angle changes in the path planning process and furthermore propose a representation of the robot configuration that substantially reduces the computational burden. In addition, we introduce search guidance heuristics that are particularly useful in environments in which a robot is required to pass over obstacles, such as on agricultural fields. An extensive evaluation on simulated and real-world data with our BoniRob agricultural robot demonstrates the efficiency of our approach.

ICRA Conference 2017 Conference Paper

Global outer-urban navigation with OpenStreetMap

• Benjamin Suger
• Wolfram Burgard

Publicly available map services are widely used by humans for navigation and nowadays provide almost complete road network data. When utilizing such maps for autonomous navigation with mobile robots one is faced with the problem of inaccuracies of the map and the uncertainty about the position of the robot relative to the map. In this paper, we present a probabilistic approach to autonomous robot navigation using data from OpenStreetMap that associates tracks from Open-StreeetMap with the trails detected by the robot based on its 3D-LiDAR data. It combines semantic terrain information, derived from the 3D-LiDAR data, with a Markov-Chain Monte-Carlo technique to match the tracks from OpenStreetMap with the sensor data. This enables our robot to utilize OpenStreetMap for navigation planning and to still stay on the trails during the execution of these plans. We present the results of extensive experiments carried out in real world settings that demonstrate the robustness of our system regarding the alignment of the vehicle pose relative to the OpenStreetMap data.

IROS Conference 2017 Conference Paper

Learning mobile manipulation actions from human demonstrations

• Tim Welschehold
• Christian Dornhege
• Wolfram Burgard

Over the past years learning from demonstration has become a popular method to intuitively teach new skills to service robots without explicit programming. However, most teaching approaches in literature use kinesthetic training and do not include mobile platforms. Here, we present a novel approach to learn joint robot base and gripper action models from observing demonstrations carried out by a human teacher. To achieve this we adapt RGBD observations of the human teacher to the capabilities of the robot. We formulate a graph optimization problem that links observations with robot grasping capabilities and kinematic constraints between co-occurring base and gripper poses. In real world experiments we show that the robot is able to learn complex mobile manipulation tasks such as opening and driving through a door.

IROS Conference 2017 Conference Paper

Metric learning for generalizing spatial relations to new objects

• Oier Mees
• Nichola Abdo
• Mladen Mazuran
• Wolfram Burgard

Human-centered environments are rich with a wide variety of spatial relations between everyday objects. For autonomous robots to operate effectively in such environments, they should be able to reason about these relations and generalize them to objects with different shapes and sizes. For example, having learned to place a toy inside a basket, a robot should be able to generalize this concept using a spoon and a cup. This requires a robot to have the flexibility to learn arbitrary relations in a lifelong manner, making it challenging for an expert to pre-program it with sufficient knowledge to do so beforehand. In this paper, we address the problem of learning spatial relations by introducing a novel method from the perspective of distance metric learning. Our approach enables a robot to reason about the similarity between pairwise spatial relations, thereby enabling it to use its previous knowledge when presented with a new relation to imitate. We show how this makes it possible to learn arbitrary spatial relations from non-expert users using a small number of examples and in an interactive manner. Our extensive evaluation with real-world data demonstrates the effectiveness of our method in reasoning about a continuous spectrum of spatial relations and generalizing them to new objects.

ICRA Conference 2017 Conference Paper

Optimal, sampling-based manipulation planning

• Philipp S. Schmitt
• Werner Neubauer
• Wendelin Feiten
• Kai M. Wurm
• Georg von Wichert
• Wolfram Burgard

When robots perform manipulation tasks, they need to determine their own movement, as well as how to grasp and release an object. Reasoning about the motion of the robot and the object simultaneously leads to a multi-modal planning problem in a high-dimensional configuration space. In this paper we propose an asymptotically optimal manipulation planner. Our approach extends optimal sampling-based roadmap planners to efficiently explore the configuration space of the robot and the object. We prove probabilistic completeness and global, asymptotic optimality. Extensive simulations of a typical pick-and-place scenario show that our approach significantly outperforms a (nonoptimal) state-of-the-art approach. We implemented our planner on a real manipulator and were able to compute high quality solutions in less than a second.

IROS Conference 2017 Conference Paper

Robot localization with sparse scan-based maps

• Alexander Schiotka
• Benjamin Suger
• Wolfram Burgard

Occupancy grid maps are a popular method for representing the environment in the context of robot navigation tasks. However, occupancy grid maps can have a high memory demand that grows quadratically with the range of the sensor. In this paper, we introduce a memory-efficient map representation that is based on a constant set of individual scans. To make these scan-based maps suitable for autonomous robot navigation, we propose probabilistically sound methods for both mapping and localization. To solve the mapping problem, our approach incrementally selects scans based on the additional information they provide relative to the scans previously selected. Using these selected scans, we perform an Monte Carlo Localization (MCL) approach with a sensor model optimized for the scan-based representation of our map. We present extensive experiments in which we evaluate our approach using real world data recorded in a garage parking scenario with an autonomous car as well as a robot localization problem in an indoor environment. The results demonstrate that our approach can cope with high sensor noise and that it achieves comparable localization accuracy while at the same time consuming only a fraction of memory compared to regular occupancy grid maps.

IROS Conference 2017 Conference Paper

Robust LiDAR-based localization in architectural floor plans

• Federico Boniardi
• Tim Caselitz
• Rainer Kümmerle
• Wolfram Burgard

Modern automation demands mobile robots to be robustly localized in complex scenarios. Current localization systems typically use maps that require to be built and interpreted by experienced operators, increasing deployment costs as well as reducing the adaptability of robots to rearrangements in the environment. In contrast, architectural floor plans can be easily understood by non-expert users and typically represent only the non-rearrangeable parts of buildings. In this paper we propose a system for robot localization in architectural CAD drawings. Our method employs a simultaneous localization and mapping approach to online augment the floor plan with a map represented as a pose-graph with LiDAR measurements. Whenever the environment is accurately mapped in the vicinity of the robot, we use the graph to perform relative localization. We thoroughly evaluate our system in challenging real-world scenarios. Experiments demonstrate that our method is able to robustly track the robot pose even when the floor plan shows major discrepancies from the real-world. We show that our system achieves sub-centimeter accuracy and is suitable for real-time application.

ICRA Conference 2017 Conference Paper

Semantics-aware visual localization under challenging perceptual conditions

• Tayyab Naseer
• Gabriel L. Oliveira
• Thomas Brox
• Wolfram Burgard

Visual place recognition under difficult perceptual conditions remains a challenging problem due to changing weather conditions, illumination and seasons. Long-term visual navigation approaches for robot localization should be robust to these dynamics of the environment. Existing methods typically leverage feature descriptions of whole images or image regions from Deep Convolutional Neural Networks. Some approaches also exploit sequential information to alleviate the problem of spatially inconsistent and non-perfect image matches. In this paper, we propose a novel approach for learning a discriminative holistic image representation which exploits the image content to create a dense and salient scene description. These salient descriptions are learnt over a variety of datasets under large perceptual changes. Such an approach enables us to precisely segment the regions of an image which are geometrically stable over large time lags. We combine features from these salient regions and an off-the-shelf holistic representation to form a more robust scene descriptor. We also introduce a semantically labeled dataset which captures extreme perceptual and structural scene dynamics over the course of 3 years. We evaluated our approach with extensive experiments on data collected over several kilometers in Freiburg and show that our learnt image representation outperforms off-the-shelf features from the deep networks and hand-crafted features.

IROS Conference 2017 Conference Paper

SMSnet: Semantic motion segmentation using deep convolutional neural networks

• Johan Vertens
• Abhinav Valada
• Wolfram Burgard

Interpreting the semantics and motion of objects are prerequisites for autonomous robots that enable them to reason and operate in dynamic real-world environments. Existing approaches that tackle the problem of semantic motion segmentation consist of long multistage pipelines and typically require several seconds to process each frame. In this paper, we present a novel convolutional neural network architecture that learns to predict both the object label and motion status of each pixel in an image. Given a pair of consecutive images, the network learns to fuse features from self-generated optical flow maps and semantic segmentation kernels to yield pixel-wise semantic motion labels. We also introduce the Cityscapes-Motion dataset which contains over 2, 900 manually annotated semantic motion labels, which is the largest dataset of its kind so far. We demonstrate that our network outperforms existing approaches achieving state-of-the-art performance on the KITTI dataset, as well as in the more challenging Cityscapes-Motion dataset while being substantially faster than existing techniques.

IROS Conference 2017 Conference Paper

Why did the robot cross the road? - Learning from multi-modal sensor data for autonomous road crossing

• Noha Radwan
• Wera Winterhalter
• Christian Dornhege
• Wolfram Burgard

We consider the problem of developing robots that navigate like pedestrians on sidewalks through city centers for performing various tasks including delivery and surveillance. One particular challenge for such robots is crossing streets without pedestrian traffic lights. To solve this task the robot has to decide based on its sensory input if the road is clear. In this work, we propose a novel multi-modal learning approach for the problem of autonomous street crossing. Our approach solely relies on laser and radar data and learns a classifier based on Random Forests to predict when it is safe to cross the road. We present extensive experimental evaluations using real-world data collected from multiple street crossing situations which demonstrate that our approach yields a safe and accurate street crossing behavior and generalizes well over different types of situations. A comparison to alternative methods demonstrates the advantages of our approach.

IROS Conference 2016 Conference Paper

A probabilistic approach based on Random Forests to estimating similarity of human motion in the context of Parkinson's Disease

• Andreas Kuhner
• Tobias Schubert 0002
• Massimo Cenciarini
• Christoph Maurer
• Wolfram Burgard

The objective characterization of human motion is required in a variety of fields including competitive sports, rehabilitation and the detection of motor deficits. Nowadays, typically human experts evaluate the motor behavior. These evaluations are based on their individual experience which leads to a low inter- and intra-expert reliability. Standardized tests improve on the reliability but are still prone to subjective ratings and require human expert knowledge. This paper presents a novel method to characterize the motor state of Parkinson patients using full body motion capturing data based on a combination of multiple metrics. Our approach merges various metrics with a Random Forest and uses a probabilistic formulation to compute a one-dimensional measure for the performed motion. We present an application of our approach to the problem of relating subject motion to different classes like healthy subjects and Parkinson disease patients with deep brain stimulation switched on or off. In the experimental session we show that our measure leads to high classification rates and high entropy values for real-world data. Besides, we show that our method discriminates between Parkinson's subjects (with and without stimulation) and healthy persons as good as the Unified Parkinson's Disease Rating Scale (UPDRS).

IROS Conference 2016 Conference Paper

A probabilistic approach to liquid level detection in cups using an RGB-D camera

• Chau Do
• Tobias Schubert 0002
• Wolfram Burgard

Robotic assistants have the potential to greatly improve our quality of life by supporting us in our daily activities. A service robot acting autonomously in an indoor environment is faced with very complex tasks. Consider the problem of pouring a liquid into a cup, the robot should first determine if the cup is empty or partially filled. RGB-D cameras provide noisy depth measurements which depend on the opaqueness and refraction index of the liquid. In this paper, we present a novel probabilistic approach for estimating the fill-level of a liquid in a cup using an RGB-D camera. Our approach does not make any assumptions about the properties of the liquid like its opaqueness or its refraction index. We develop a probabilistic model using features extracted from RGB and depth data. Our experiments demonstrate the robustness of our method and an improvement over the state of the art.

ICRA Conference 2016 Conference Paper

Automatic bone parameter estimation for skeleton tracking in optical motion capture

• Tobias Schubert 0002
• Katharina Eggensperger
• Alexis Gkogkidis
• Frank Hutter
• Tonio Ball
• Wolfram Burgard

Motion analysis is important in a broad range of contexts, including animation, bio-mechanics, robotics and experiments investigating animal behavior. For applications, in which tracking accuracy is one of the main requirements, passive optical motion capture systems are widely used. Many skeleton tracking methods based on such systems use a predefined skeleton model, which is scaled once in the initialization step to the individual size of the character to be tracked. However, there are remarkable differences in the bone length relations across gender and even more across mammal races. In practice, the optimal skeleton model has to be determined in a manual and time-consuming process. In this paper, we reformulate this task as an optimization problem aiming to rescale a rough hierarchical skeleton structure to optimize probabilistic skeleton tracking performance. We solve this optimization problem by means of state-of-the-art blackbox optimization methods based on sequential model-based Bayesian optimization (SMBO). We compare different SMBO methods on three real-world datasets with an animal and humans, demonstrating that we can automatically find skeleton structures for previously unseen mammals. The same methods also allow an automated choice of a suitable starting frame for initializing tracking.

IROS Conference 2016 Conference Paper

Automatic channel selection in neural microprobes: A combinatorial multi-armed bandit approach

• Camilo Gordillo
• Barbara Frank
• István Ulbert
• Oliver Paul
• Patrick Ruther
• Wolfram Burgard

State-of-the-art neural microprobes contain hundreds of electrodes within a single shaft. Due to hardware and wiring restrictions, it is usually only possible to measure a small subset of the available electrodes simultaneously. The selection of the best channels is typically performed offline either manually or automatically. However, having a fixed selection for long-term observation does not allow the system to react to changes in the neural activity, and may therefore lead to the loss of important information. In this paper, we formulate the process of autonomously selecting the best subset of electrodes as a combinatorial multi-armed bandit problem with non-stationary rewards, thus allowing the probe to adapt its selection policies online. In order to minimize exploratory actions of the probe, we furthermore take advantage of the existing dependencies between neighboring channels. Our approach is an adaptation of the discounted upper confidence bounds (D-UCB) algorithm, and identifies the electrodes providing the largest amount of non-redundant information. To the best of our knowledge, this is the first online approach for the problem of electrode selection. In extensive experiments, we demonstrate that our solution is not only able to converge towards an average optimal selection policy, but it is also able to react to changes in the neural activity or to damages of the recording electrodes.

ICRA Conference 2016 Conference Paper

Autonomous indoor robot navigation using a sketch interface for drawing maps and routes

• Federico Boniardi
• Abhinav Valada
• Wolfram Burgard
• Gian Diego Tipaldi

Hand-Drawn sketches are natural means by which abstract descriptions of environments can be provided. They represent weak prior information about the scene, thereby enabling a robot to perform autonomous navigation and exploration when a full metrical description of the environment is not available beforehand. In this paper, we present an extensive evaluation of our navigation system that uses a sketch interface to allow the operator of a robot to draw a rough map of an indoor environment as well as a desired trajectory for the robot to follow. We employ a theoretical framework for sketch interpretation, in which associations between the sketch and the real world are modeled as local deformations of a suitable metric manifold. We investigate the effectiveness of our system and present empirical results from a set of experiments in real-world scenarios, focusing both on the navigation capabilities and the usability of the interface.

IROS Conference 2016 Conference Paper

BI 2 RRT*: An efficient sampling-based path planning framework for task-constrained mobile manipulation

• Felix Burget
• Maren Bennewitz
• Wolfram Burgard

Mobile manipulators installed in warehouses and factories for conveying goods between working stations need to meet the requirements of time-critical workflows. Moreover, the systems are expected to deal with changing tasks, cluttered environments and constraints imposed by the goods to be delivered. In this paper, we present a novel planning framework for generating asymptotically optimal paths for mobile manipulators subject to task constraints. Our approach introduces the Bidirectional Informed RRT* (BI 2 RRT*) that extends the Informed RRT* [1] towards bidirectional search and satisfaction of end-effector task constraints. In various experiments, we demonstrate the efficiency of BI 2 RRT* for both unconstrained and constrained mobile manipulation planning problems. As the results show, our planning framework finds better solutions than Informed RRT* and Bidirectional RRT* in less planning.

IROS Conference 2016 Conference Paper

Choosing smartly: Adaptive multimodal fusion for object detection in changing environments

• Oier Mees
• Andreas Eitel
• Wolfram Burgard

Object detection is an essential task for autonomous robots operating in dynamic and changing environments. A robot should be able to detect objects in the presence of sensor noise that can be induced by changing lighting conditions for cameras and false depth readings for range sensors, especially RGB-D cameras. To tackle these challenges, we propose a novel adaptive fusion approach for object detection that learns weighting the predictions of different sensor modalities in an online manner. Our approach is based on a mixture of convolutional neural network (CNN) experts and incorporates multiple modalities including appearance, depth and motion. We test our method in extensive robot experiments, in which we detect people in a combined indoor and outdoor scenario from RGB-D data, and we demonstrate that our method can adapt to harsh lighting changes and severe camera motion blur. Furthermore, we present a new RGB-D dataset for people detection in mixed in- and outdoor environments, recorded with a mobile robot.

ICRA Conference 2016 Conference Paper

Deep learning for human part discovery in images

• Gabriel L. Oliveira
• Abhinav Valada
• Claas Bollen
• Wolfram Burgard
• Thomas Brox

This paper addresses the problem of human body part segmentation in conventional RGB images, which has several applications in robotics, such as learning from demonstration and human-robot handovers. The proposed solution is based on Convolutional Neural Networks (CNNs). We present a network architecture that assigns each pixel to one of a predefined set of human body part classes, such as head, torso, arms, legs. After initializing weights with a very deep convolutional network for image classification, the network can be trained end-to-end and yields precise class predictions at the original input resolution. Our architecture particularly improves on over-fitting issues in the up-convolutional part of the network. Relying only on RGB rather than RGB-D images also allows us to apply the approach outdoors. The network achieves state-of-the-art performance on the PASCAL Parts dataset. Moreover, we introduce two new part segmentation datasets, the Freiburg sitting people dataset and the Freiburg people in disaster dataset. We also present results obtained with a ground robot and an unmanned aerial vehicle.

ICRA Conference 2016 Conference Paper

Do you see the bakery? Leveraging geo-referenced texts for global localization in public maps

• Noha Radwan
• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard

Text is one of the richest sources of information in an urban environment. Although textual information is heavily relied on by humans for a majority of the daily tasks, its usage has not been completely exploited in the field of robotics. In this work, we propose a localization approach utilizing textual features in urban environments. Starting at an unknown location, equipped with an RGB-camera and a compass, our approach uses off-the-shelf text extraction methods to identify text labels in the vicinity. We then apply a probabilistic localization approach with specific sensor models to integrate multiple observations. An extensive evaluation with real-world data gathered in different cities reveals an improvement over GPS-based localization when using our method.

IROS Conference 2016 Conference Paper

Efficient deep models for monocular road segmentation

• Gabriel L. Oliveira
• Wolfram Burgard
• Thomas Brox

This paper addresses the problem of road scene segmentation in conventional RGB images by exploiting recent advances in semantic segmentation via convolutional neural networks (CNNs). Segmentation networks are very large and do not currently run at interactive frame rates. To make this technique applicable to robotics we propose several architecture refinements that provide the best trade-off between segmentation quality and runtime. This is achieved by a new mapping between classes and filters at the expansion side of the network. The network is trained end-to-end and yields precise road/lane predictions at the original input resolution in roughly 50ms. Compared to the state of the art, the network achieves top accuracies on the KITTI dataset for road and lane segmentation while providing a 20× speed-up. We demonstrate that the improved efficiency is not due to the road segmentation task. Also on segmentation datasets with larger scene complexity, the accuracy does not suffer from the large speed-up.

IROS Conference 2016 Conference Paper

Learning manipulation actions from human demonstrations

• Tim Welschehold
• Christian Dornhege
• Wolfram Burgard

Learning from demonstration is a popular approach for teaching robots as it allows service robots to acquire new skills without explicit programming. However, for manipulation actions mostly kinesthetic teaching is used as these actions require precise knowledge about the interactions between the robot and the object. In this paper, we present a novel approach that allows a robot to learn actions carried out by a teacher from observations. We achieve this by first transforming RGBD observations to consistent hand-object trajectories, which are then adapted to the robot's grasping capabilities. Experimental results show that the robot is able to learn complex tasks such as opening doors or drawers.

IROS Conference 2016 Conference Paper

Monocular camera localization in 3D LiDAR maps

• Tim Caselitz
• Bastian Steder
• Michael Ruhnke
• Wolfram Burgard

Localizing a camera in a given map is essential for vision-based navigation. In contrast to common methods for visual localization that use maps acquired with cameras, we propose a novel approach, which tracks the pose of monocular camera with respect to a given 3D LiDAR map. We employ a visual odometry system based on local bundle adjustment to reconstruct a sparse set of 3D points from image features. These points are continuously matched against the map to track the camera pose in an online fashion. Our approach to visual localization has several advantages. Since it only relies on matching geometry, it is robust to changes in the photometric appearance of the environment. Utilizing panoramic LiDAR maps additionally provides viewpoint invariance. Yet low-cost and lightweight camera sensors are used for tracking. We present real-world experiments demonstrating that our method accurately estimates the 6-DoF camera pose over long trajectories and under varying conditions.

ICRA Conference 2016 Conference Paper

Motion-based detection and tracking in 3D LiDAR scans

• Ayush Dewan
• Tim Caselitz
• Gian Diego Tipaldi
• Wolfram Burgard

Robots are expected to operate autonomously in increasingly complex scenarios such as crowded streets or heavy traffic situations. Perceiving the dynamics of moving objects in the environment is crucial for safe and smart navigation and therefore a key enabler for autonomous driving. In this paper we present a novel model-free approach for detecting and tracking dynamic objects in 3D LiDAR scans obtained by a moving sensor. Our method only relies on motion cues and does not require any prior information about the objects. We sequentially detect multiple motions in the scene and segment objects using a Bayesian approach. For robustly tracking objects, we utilize their estimated motion models. We present extensive quantitative results based on publicly available datasets and show that our approach outperforms the state of the art.

IROS Conference 2016 Conference Paper

Rigid scene flow for 3D LiDAR scans

• Ayush Dewan
• Tim Caselitz
• Gian Diego Tipaldi
• Wolfram Burgard

The perception of the dynamic aspects of the environment is a highly relevant precondition for the realization of autonomous robot system acting in the real world. In this paper, we propose a novel method for estimating dense rigid scene flow in 3D LiDAR scans. We formulate the problem as an energy minimization problem, where we assume local geometric constancy and incorporate regularization for smooth motion fields. Analyzing the dynamics at point level helps in inferring the fine-grained details of motion. We show results on multiple sequences of the KITTI odometry dataset, where we seamlessly estimate multiple motions pertaining to different dynamic objects. Furthermore, we test our approach on a dataset with pedestrians to show how our method adapts to a case with non-rigid motion. For comparison we use the ground truth from KITTI and show how our method outperforms different ICP-based methods.

IROS Conference 2016 Conference Paper

Terrain-adaptive obstacle detection

• Benjamin Suger
• Bastian Steder
• Wolfram Burgard

Reliable detection and avoidance of obstacles is a crucial prerequisite for autonomously navigating robots as both guarantee safety and mobility. To ensure safe mobility, the obstacle detection needs to run online, thereby taking limited resources of autonomous systems into account. At the same time, robust obstacle detection is highly important. Here, a too conservative approach might restrict the mobility of the robot, while a more reckless one might harm the robot or the environment it is operating in. In this paper, we present a terrain-adaptive approach to obstacle detection that relies on 3D-Lidar data and combines computationally cheap and fast geometric features, like step height and steepness, which are updated with the frequency of the lidar sensor, with semantic terrain information, which is updated with at lower frequency. We provide experiments in which we evaluate our approach on a real robot on an autonomous run over several kilometers containing different terrain types. The experiments demonstrate that our approach is suitable for autonomous systems that have to navigate reliable on different terrain types including concrete, dirt roads and grass.

ICRA Conference 2015 Conference Paper

3D-reconstruction of indoor environments from human activity

• Barbara Frank
• Michael Ruhnke
• Maxim Tatarchenko
• Wolfram Burgard

Observing human activities can reveal a lot about the structure of the environment, the objects contained therein and also their functionality. This knowledge, in turn, can be useful for robots interacting with humans or for robots performing mobile manipulation tasks. In this paper, we present an approach to infer the geometric and functional structure of the environment and the position of certain relevant objects in it from human activity. We observe this activity using a full-body motion capture system consisting of a set of inertial measurement units. This is a hard problem since our data suit provides odometry estimates only, which severely drift over time. Therefore, we regard the objects inferred from the activities as landmarks in a graph-based simultaneous localization and mapping problem, which we optimize to obtain accurate estimates about the poses of the objects and the trajectory of the human. In extensive experiments, we demonstrate the effectiveness of the proposed method for the reconstruction of 3D representations. The resulting models not only contain a geometric but also a functional description of the environment and naturally provide a segmentation into individual objects.

IROS Conference 2015 Conference Paper

A comparative study of contact models for contact-aware state estimation

• Shuai Li 0015
• Siwei Lyu
• Jeffrey C. Trinkle
• Wolfram Burgard

We study the contact-aware state estimation (CASE) problem, i. e. , the problem of estimating the state of an object while it is being actively manipulated by a robot. Several researchers have developed particle filters for this problem. They estimate the state (pose and velocity) of manipulated objects, some physical properties (such as mass and shape), and contact information (such as, gain or loss of contact and transitions between sliding and sticking). However, the effects of various contact and noise models, which can have a huge impact on the estimation results, are obfuscated by implementation details. In this paper, we study the CASE problem arising from a simple pushing task with the goal of shedding light on the fundamental contact modeling choices. Specifically, we evaluate four particle filters based upon four probabilistic state transition models generated from a deterministic multibody dynamics models with rigid or compliant contacts, each of which is augmented by one of two different noise models. Comparisons of these state transition models are carried out through the analysis of real and simulated experiments, the results of which, provide guidance to filter designers.

IROS Conference 2015 Conference Paper

Accurate indoor localization for RGB-D smartphones and tablets given 2D floor plans

• Wera Winterhalter
• Freya Fleckenstein
• Bastian Steder
• Luciano Spinello
• Wolfram Burgard

Accurate localization in indoor environments is widely regarded as a key opener for various location-based services. Despite tremendous advancements in the development of innovative sensor concepts, the most effective and accurate solutions to this problem make use of a map computed from sensory data. In this paper, we present an efficient approach to localize an RGB-D smartphone or tablet that only makes use of a two-dimensional outline of the environment as a map as it is typically available from architectural drawings. Our technique employs a particle filter to estimate the 6DoF pose. We propose a sensor model that robustly estimates the likelihood of measurements and accommodates the disagreements between floor plans and real world data. In extensive experiments, we demonstrate that our approach is able to globally localize a user in a given 2D floor plan using a Google Tango device and to accurately track the user in such an environment.

IROS Conference 2015 Conference Paper

Accurate localization with respect to moving objects via multiple-body registration

• Jörg Röwekämper
• Benjamin Suger
• Wolfram Burgard
• Gian Diego Tipaldi

Many mobile manipulation tasks require the robot to be accurately localized with respect to the object where the manipulation has to be executed. These tasks include autonomous docking and positioning as well as pick and place or logistics tasks. State-of-the-art approaches to the problem commonly assume that the environment is static and localize the robot with respect to predetermined locations. In this paper, we present an approach that relaxes the static assumption and enables a robot to accurately localize with respect to a reference object that could be moved in the environment. The core of the paper is an extension of the generalized ICP method to handle multiple rigid bodies that move independently to each others. Experiments in both simulated and real world scenarios show that our approach is able to localize the robot with respect to a moved object with an accuracy of less than one centimeter.

ICRA Conference 2015 Conference Paper

An autonomous robotic assistant for drinking

• Sebastian Schroer
• Ingo Killmann
• Barbara Frank
• Martin Voelker
• Lukas Dominique Josef Fiederer
• Tonio Ball
• Wolfram Burgard

Stroke and neurodegenerative diseases, among a range of other neurologic disorders, can cause chronic paralysis. Patients suffering from paralysis may remain unable to achieve even basic everyday tasks such as liquid intake. Currently, there is a great interest in developing robotic assistants controlled via brain-machine interfaces (BMIs) to restore the ability to perform such tasks. This paper describes an autonomous robotic assistant for liquid intake. The components of the system include autonomous online detection both of the cup to be grasped and of the mouth of the user. It plans motions of the robot arm under the constraints that the cup stays upright while moving towards the mouth and that the cup stays in direct contact with the user's mouth while the robot tilts it during the drinking phase. To achieve this, our system also includes a technique for online estimation of the location of the user's mouth even under partial occlusions by the cup or robot arm. We tested our system in a real environment and in a shared-control setting using frequency-specific modulations recorded by electroencephalography (EEG) from the brain of the user. Our experiments demonstrate that our BMI-controlled robotic system enables a reliable liquid intake. We believe that our approach can easily be extended to other useful tasks including food intake and object manipulation.

ICRA Conference 2015 Conference Paper

Automatic extrinsic calibration of multiple laser range sensors with little overlap

• Jörg Röwekämper
• Michael Ruhnke
• Bastian Steder
• Wolfram Burgard
• Gian Diego Tipaldi

Networks of laser range finders are a popular tool for monitoring large cluttered areas and to track people. Whenever multiple scanners are used for this purpose, one major problem is how to determine the relative positions of all the scanners. In this paper, we present a novel approach to calibrate a network of multiple planar laser range finders scanning horizontally. To robustly deal with the potentially restricted overlap between the fields of view, our approach only requires a dynamic object, e. g. , a person, moving through the observed area. We employ a RANSAC-like algorithm to find the correspondences between the measurements of the different laser range finders. Based on these correspondences we formulate a graph-based optimization problem to determine the maximum likelihood extrinsic parameters of the sensor network. Furthermore, we present a method to evaluate the consistency of the resulting calibration based on visibility constraints. Experiments on real and simulated data show that the proposed approach yields better results than techniques that only perform pairwise calibration.

ICRA Conference 2015 Conference Paper

Automatic initialization for skeleton tracking in optical motion capture

• Tobias Schubert 0002
• Alexis Gkogkidis
• Tonio Ball
• Wolfram Burgard

The ability to track skeletal movements is important in a variety of applications including animation, biological studies and animal experiments. To detect even small movements, such a method should provide highly accurate estimates. Besides that it should not impede the mammal in its motion. This motivates the usage of a passive optical motion capture system. Thereby the main challenges are the initialization, the association of the unlabeled markers to their corresponding segment also across the frames, and the estimation of the skeleton configuration. While many existing approaches can deal with the latter two problems, they typically need a specific pose for initialization. This is rather unpractical in the context of animal tracking and often requires a manual initialization process. In this paper, we present an approach to reliably track animals and humans in marker-based optical motion capture systems with freely attached markers. Our method is also able to perform an automatic initialization without any pre-or post-processing of the data. To achieve this, our approach utilizes a large database of previously observed poses. We present our algorithm and its evaluation on real-world data sets with an animal and humans. The results demonstrate that our initialization method performs accurately for the most kind of initial poses and our tracking approach outperforms a popular fully automatic skeleton tracking method especially with respect to the smoothness of the motion.

ICRA Conference 2015 Conference Paper

Efficient and effective matching of image sequences under substantial appearance changes exploiting GPS priors

• Olga Vysotska
• Tayyab Naseer
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

The ability to localize a robot is an important capability and matching of observations under substantial changes is a prerequisite for robust long-term operation. This paper investigates the problem of efficiently coping with seasonal changes in image data. We present an extension of a recent approach [15] to visual image matching using sequence information. Our extension allows for exploiting GPS priors in the matching process to overcome the main computational bottleneck of the previous method and to handle loops within the image sequences. We present an experimental evaluation using real world data containing substantial seasonal changes and show that our approach outperforms the previous method in case a noisy GPS pose prior is available.

ICRA Conference 2015 Conference Paper

Inverse reinforcement learning of behavioral models for online-adapting navigation strategies

• Michael Herman
• Volker Fischer 0003
• Tobias Gindele
• Wolfram Burgard

To increase the acceptance of autonomous systems in populated environments, it is indispensable to teach them social behavior. We would expect a social robot, which plans its motions among humans, to consider both the social acceptability of its behavior as well as task constraints, such as time limits. These requirements are often contradictory and therefore resulting in a trade-off. For example, a robot has to decide whether it is more important to quickly achieve its goal or to comply with social conventions, such as the proximity to humans, i. e. , the robot has to react adaptively to task-specific priorities. In this paper, we present a method for priority-adaptive navigation of mobile autonomous systems, which optimizes the social acceptability of the behavior while meeting task constraints. We learn acceptability-dependent behavioral models from human demonstrations by using maximum entropy (MaxEnt) inverse reinforcement learning (IRL). These models are generative and describe the learned stochastic behavior. We choose the optimum behavioral model by maximizing the social acceptability under constraints on expected time-limits and reliabilities. This approach is evaluated in the context of driving behaviors based on the highway scenario of Levine et al. [1].

ICRA Conference 2015 Conference Paper

Learning driving styles for autonomous vehicles from demonstration

• Markus Kuderer
• Shilpa Gulati
• Wolfram Burgard

It is expected that autonomous vehicles capable of driving without human supervision will be released to market within the next decade. For user acceptance, such vehicles should not only be safe and reliable, they should also provide a comfortable user experience. However, individual perception of comfort may vary considerably among users. Whereas some users might prefer sporty driving with high accelerations, others might prefer a more relaxed style. Typically, a large number of parameters such as acceleration profiles, distances to other cars, speed during lane changes, etc. , characterize a human driver's style. Manual tuning of these parameters may be a tedious and error-prone task. Therefore, we propose a learning from demonstration approach that allows the user to simply demonstrate the desired style by driving the car manually. We model the individual style in terms of a cost function and use feature-based inverse reinforcement learning to find the model parameters that fit the observed style best. Once the model has been learned, it can be used to efficiently compute trajectories for the vehicle in autonomous mode. We show that our approach is capable of learning cost functions and reproducing different driving styles using data from real drivers.

IROS Conference 2015 Conference Paper

Learning motor control parameters for motion strategy analysis of Parkinson's disease patients

• Felix Burget
• Christoph Maurer
• Wolfram Burgard
• Maren Bennewitz

Although the neurological impairments of Parkinson's disease (PD) patients are well known to go along with motor control deficits, e. g. , tremor, rigidity, and reduced movement, not much is known about the motor control parameters affected by the disease. In this paper, we therefore present a novel approach to human motions analysis using motor control strategies with joint weight parameterization. We record the motions of healthy subjects and PD patients performing a hand coordination task with the whole-body XSens MVN motion capture system. For our motion strategy analysis we then follow a two step approach. First, we perform a complexity reduction by mapping the recorded human motions to a simplified kinematic model of the upper body. Second, we reproduce the recorded motions using a Jacobian weighted damped least squares controller with adaptive joint weights. We developed a method to iteratively learn the joint weights of the controller with the mapped human joint trajectories as reference input. Finally, we use the learned joint weights for a quantitative comparison between the motion control strategies of healthy subjects and PD patients. Other than expected from clinical experience, we found that the joint weights are almost evenly distributed along the arm in the PD group. In contrast to that, the proximal joint weights of the healthy subjects are notably larger than the distal ones.

ICRA Conference 2015 Conference Paper

LexTOR: Lexicographic teach optimize and repeat based on user preferences

• Mladen Mazuran
• Christoph Sprunk
• Wolfram Burgard
• Gian Diego Tipaldi

In the last years, many researchers started to consider teach-and-repeat approaches for reliable autonomous navigation. The paradigm, in all its proposed forms, is deeply rooted in the idea that the robot should autonomously follow a route that has been demonstrated by a human during a teach phase. However, human demonstrations are often inefficient in terms of execution time or may cause premature wear of the robot components due to jittery behavior or strong accelerations. In this paper, we propose the concept of teach, optimize and repeat, which introduces a trajectory optimization step between the teach and repeat phases. To address this problem, we further propose LexTOR, a constrained trajectory optimization method for teach and repeat problems, where the constraints are defined according to user preferences. At its core, LexTOR optimizes both the execution time and the trajectory smoothness in a lexicographic sense. The experiments show that LexTOR is very effective, both qualitatively and quantitatively, in terms of execution time, smoothness, accuracy and bound satisfaction.

ICRA Conference 2015 Conference Paper

Localization on OpenStreetMap data using a 3D laser scanner

• Philipp Ruchti
• Bastian Steder
• Michael Ruhnke
• Wolfram Burgard

To determine the pose of a vehicle is a fundamental problem in mobile robotics. Most approaches relate the current sensor observations to a map generated with previously acquired data of the same system or by another system with a similar sensor setup. Unfortunately, previously acquired data is not always available. In outdoor settings, GPS is a very useful tool to determine a global estimate of the vehicles pose. Unfortunately, GPS tends to be unreliable in situations in which a clear view to the sky is restricted. Yet, one can make use of publicly available map material as prior information. In this paper, we describe an approach to localize a robot equipped with a 3D range scanner with respect to a road network created from OpenStreetMap data. To successfully localize a mobile robot we propose a road classification scheme for 3D range data together with a novel sensor model, which relates the classification results to a road network. Compared to other approaches, our system does not require the robot to actually travel on the road network. We evaluate our approach in extensive experiments on simulated and real data and compare favorably to two state-of-the-art methods on those data.

ICRA Conference 2015 Conference Paper

Maximum likelihood remission calibration for groups of heterogeneous laser scanners

• Bastian Steder
• Michael Ruhnke
• Rainer Kümmerle
• Wolfram Burgard

Laser range scanners are commonly used in mobile robotics to enable a robot to sense the spatial configuration of its environment. In addition to the range measurements, most scanners provide remission values, representing the intensity of the returned light pulse. These values add a visual component to the measurement and can be used to improve reasoning on the data. Unfortunately, a remission value does not directly tell us how bright a measured surface is in the infrared spectrum. Rather, it varies with respect to the incidence angle and the range at which it was measured. In addition, multiple scanners typically do not agree upon the values of a certain surface. In this paper, we present a calibration method for remission values of multiple laser scanners considering dependencies in range, incidence angle of the measured surface, and the respective scanner unit. Our system learns the calibration parameters based on a set of registered point clouds. It uses a graph optimization scheme to minimize the error between different measurements, so that all involved scanners yield consistent reflection values, independent of the perspective from which the corresponding surface is observed.

IROS Conference 2015 Conference Paper

Metric localization using Google Street View

• Pratik Agarwal
• Wolfram Burgard
• Luciano Spinello

Accurate metrical localization is one of the central challenges in mobile robotics. Many existing methods aim at localizing after building a map with the robot. In this paper, we present a novel approach that instead uses geo-tagged panoramas from the Google Street View as a source of global positioning. We model the problem of localization as a non-linear least squares estimation in two phases. The first estimates the 3D position of tracked feature points from short monocular camera sequences. The second computes the rigid body transformation between the Street View panoramas and the estimated points. The only input of this approach is a stream of monocular camera images and odometry estimates. We quantified the accuracy of the method by running the approach on a robotic platform in a parking lot by using visual fiducials as ground truth. Additionally, we applied the approach in the context of personal localization in a real urban scenario by using data from a Google Tango tablet.

IROS Conference 2015 Conference Paper

Monte Carlo localization in hand-drawn maps

• Bahram Behzadian
• Pratik Agarwal
• Wolfram Burgard
• Gian Diego Tipaldi

Robot localization is one of the most important problems in robotics. Most of the existing approaches assume that the map of the environment is available beforehand and focus on accurate metrical localization. In this paper, we address the localization problem when the map of the environment is not present beforehand, and the robot relies on a hand-drawn map from a non-expert user. We addressed this problem by expressing the robot pose in the pixel coordinate and simultaneously estimate a local deformation of the hand-drawn map. Experiments show that we can successfully identify the room in which the robot is located in 80% of the tests.

IROS Conference 2015 Conference Paper

Multimodal deep learning for robust RGB-D object recognition

• Andreas Eitel
• Jost Tobias Springenberg
• Luciano Spinello
• Martin A. Riedmiller
• Wolfram Burgard

Robust object recognition is a crucial ingredient of many, if not all, real-world robotics applications. This paper leverages recent progress on Convolutional Neural Networks (CNNs) and proposes a novel RGB-D architecture for object recognition. Our architecture is composed of two separate CNN processing streams - one for each modality - which are consecutively combined with a late fusion network. We focus on learning with imperfect sensor data, a typical problem in real-world robotics tasks. For accurate learning, we introduce a multi-stage training methodology and two crucial ingredients for handling depth data with CNNs. The first, an effective encoding of depth information for CNNs that enables learning without the need for large depth datasets. The second, a data augmentation scheme for robust learning with depth images by corrupting them with realistic noise patterns. We present state-of-the-art results on the RGB-D object dataset [15] and show recognition in challenging RGB-D real-world noisy settings.

IROS Conference 2015 Conference Paper

Navigating blind people with a smart walker

• Andreas Wachaja
• Pratik Agarwal
• Mathias Zink
• Miguel Reyes Adame
• Knut Möller
• Wolfram Burgard

Navigation in complex and unknown environments is a major challenge for blind people. The most popular, conventional navigation aids such as white canes and guide dogs, however, provide limited assistance in such settings as they are constrained to interpret the local environment only. At the same time, they can hardly be combined with a walker required by elderly people with walking disabilities. Technologies developed in the field of robotics have the potential to assist blind people in complex navigation tasks as they can provide information about obstacles and reason on both global and local environment models. The contribution of this paper is a smart walker that enables blind users to safely navigate. It includes an innovative vibro-tactile user interface and a controller that takes into account human characteristics based on a user study. The walker has been designed to deal with the fact that humans can only sense and interpret a limited number of commands and have a delayed response. Our experiments validate our claim that the technique outlined in this paper guides a user to the desired goal in less time and with shorter traveled distance compared to a standard robotic controller.

ICRA Conference 2015 Conference Paper

Robot, organize my shelves! Tidying up objects by predicting user preferences

• Nichola Abdo
• Cyrill Stachniss
• Luciano Spinello
• Wolfram Burgard

As service robots become more and more capable of performing useful tasks for us, there is a growing need to teach robots how we expect them to carry out these tasks. However, learning our preferences is a nontrivial problem, as many of them stem from a variety of factors including personal taste, cultural background, or common sense. Obviously, such factors are hard to formulate or model a priori. In this paper, we present a solution for tidying up objects in containers, e. g. , shelves or boxes, by following user preferences. We learn the user preferences using collaborative filtering based on crowdsourced and mined data. First, we predict pairwise object preferences of the user. Then, we subdivide the objects in containers by modeling a spectral clustering problem. Our solution is easy to update, does not require complex modeling, and improves with the amount of user data. We evaluate our approach using crowdsoucing data from over 1, 200 users and demonstrate its effectiveness for two tidy-up scenarios. Additionally, we show that a real robot can reliably predict user preferences using our approach.

IROS Conference 2015 Conference Paper

Robust visual SLAM across seasons

• Tayyab Naseer
• Michael Ruhnke
• Cyrill Stachniss
• Luciano Spinello
• Wolfram Burgard

In this paper, we present an appearance-based visual SLAM approach that focuses on detecting loop closures across seasons. Given two image sequences, our method first extracts one descriptor per image for both sequences using a deep convolutional neural network. Then, we compute a similarity matrix by comparing each image of a query sequence with a database. Finally, based on the similarity matrix, we formulate a flow network problem and compute matching hypotheses between sequences. In this way, our approach can handle partially matching routes, loops in the trajectory and different speeds of the robot. With a matching hypothesis as loop closure information and the odometry information of the robot, we formulate a graph based SLAM problem and compute a joint maximum likelihood trajectory.

IROS Conference 2015 Conference Paper

Trail-Map-based homing under the presence of sensor noise

• Annett Stelzer
• Michael Suppa
• Wolfram Burgard

The travel range of a mobile robot is directly linked to its energy consumption. Apart from the actuators, the computing resources are the main energy consumers. Thus, robots that can navigate with minimal computational resources would be able to travel longer distances and, hence, would be valuable tools in applications such as search and rescue or planetary exploration. Inspired from the navigational abilities of insects, we developed the Trail-Map as a data structure for biologically inspired homing that can easily be scaled in case of memory or computational shortage. The Trail-Map can be built in constant time and enables constant time homing vector calculation. In this paper, we evaluate the Trail-Map-based homing performance of a simulated mobile robot equipped with an omnidirectional camera under the presence of sensor noise, such as odometry errors and observation errors. Further, we will show that Trail-Map-based homing outperforms SLAM methods in terms of computational resources while achieving a comparable homing performance.

ICRA Conference 2015 Conference Paper

Traversability analysis for mobile robots in outdoor environments: A semi-supervised learning approach based on 3D-lidar data

• Benjamin Suger
• Bastian Steder
• Wolfram Burgard

The ability to safely navigate is a crucial prerequisite for truly autonomous systems. A robot has to distinguish obstacles from traversable ground. Failing on this task can cause great damage or restrict the robots movement unnecessarily. Due to the security relevance of this problem, great effort is typically spent to design models for individual robots and sensors, and the complexity of such models is correlated to the complexity of the environment and the capabilities of the robot. We present a semi supervised learning approach, where the robot learns its traversability capabilities from a human operating it. From this partially and only positive labeled training data, our approach infers a model for the traversability analysis, thereby requiring very little manual effort for the human. In practical experiments we show that our method can be used for robots that need to reliably navigate on dirt roads as well as for robots that have very restricted traversability capabilities.

ICRA Conference 2015 Conference Paper

Where to park? minimizing the expected time to find a parking space

• Igor Bogoslavskyi
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Quickly finding a free parking spot that is close to a desired target location can be a difficult task. This holds for human drivers and autonomous cars alike. In this paper, we investigate the problem of predicting the occupancy of parking spaces and exploiting this information during route planning. We propose an MDP-based planner that considers route information as well as the occupancy probabilities of parking spaces to compute the path that minimizes the expected total time for finding an unoccupied parking space and for walking from the parking location to the target destination. We evaluated our system on real world data gathered over several days in a real parking lot. We furthermore compare our approach to three parking strategies and show that our method outperforms the alternative behaviors.

IROS Conference 2014 Conference Paper

A catadioptric extension for RGB-D cameras

• Felix Endres
• Christoph Sprunk
• Rainer Kümmerle
• Wolfram Burgard

The typically restricted field of view of visual sensors often imposes limitations on the performance of localization and simultaneous localization and mapping (SLAM) approaches. In this paper, we propose and analyze the combination of an RGB-D camera with two planar mirrors to split the field of view such that it covers both front and rear view of a mobile robot. We describe how to estimate the extrinsic calibration parameters of the modified sensor using a standard parametrization and a reduced one that exploits the properties of the setup. Our experimental evaluation on real-world data demonstrates the robustness of the calibration procedure. Additionally, we show that our proposed sensor modification substantially improves the accuracy and the robustness in a simultaneous localization and mapping task.

ICRA Conference 2014 Conference Paper

A probabilistic approach to high-confidence cleaning guarantees for low-cost cleaning robots

• Jürgen Hess 0001
• Maximilian Beinhofer
• Wolfram Burgard

Cleaning is widely regarded as one of the most relevant applications of autonomous service robots. The goal of robotic cleaning is to achieve low dirt levels in the whole environment. Low cost consumer robots, however, are typically prone to high motion and sensor uncertainties. Additionally, their cleaning units do not always remove the dirt entirely. As a result, there is a substantial probability that some parts of the environment are not cleaned sufficiently. In this paper, we propose an approach to robotic cleaning that guarantees that in the whole environment, the dirt levels after cleaning are reduced below a user-defined threshold with high confidence. We introduce a novel probabilistic model for jointly estimating the trajectory of the robot and the current dirt distribution in the environment. Based on this estimate, we adapt the future cleaning path during operation such that the robot re-visits areas in which high dirt levels are still likely. We demonstrate the effectiveness of our approach in extensive experiments carried out both in simulation and with a real vacuum cleaning robot, also in comparison to previous approaches.

ICRA Conference 2014 Conference Paper

A statistical measure for map consistency in SLAM

• Mladen Mazuran
• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Map consistency is an important requirement for applications in which mobile robots need to effectively perform autonomous navigation tasks. While recent SLAM techniques provide an increased robustness even in the context of bad initializations or data association outliers, the question of how to determine whether or not the resulting map is consistent is still an open problem. In this paper, we introduce a novel measure for map consistency. We compute this measure by taking into account the discrepancies in the sensor data and leverage it to address two important problems in SLAM. First, we derive a statistical test for assessing whether a map is consistent or not. Second, we employ it to automatically set the free parameter of dynamic covariance scaling, a robust SLAM back-end. We present an evaluation of our approach on over 50 maps sourced from 16 publicly available datasets and illustrate its capability for the inconsistency detection and the tuning of the parameter of the back-end.

ICRA Conference 2014 Conference Paper

An approach to solving large-scale SLAM problems with a small memory footprint

• Benjamin Suger
• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard

In the past, highly effective solutions to the SLAM problem based on solving nonlinear optimization problems have been developed. However, most approaches put their major focus on runtime and accuracy rather than on memory consumption, which becomes especially relevant when large-scale SLAM problems have to be solved. In this paper, we consider the SLAM problem from the point of view of memory consumption and present a novel approximate approach to SLAM with low memory consumption. Our approach achieves this based on a hierarchical decomposition consisting of small submaps with limited size. We perform extensive experiments on synthetic and publicly available datasets. The results demonstrate that in situations in which the representation of the complete map requires more than the available main memory, our approach, in comparison to state-of-the-art exact solvers, reduces the memory consumption and the runtime up to a factor of 2 while still providing highly accurate maps.

IROS Conference 2014 Conference Paper

Automatic channel selection and neural signal estimation across channels of neural probes

• Olga Vysotska
• Barbara Frank
• István Ulbert
• Oliver Paul
• Patrick Ruther
• Cyrill Stachniss
• Wolfram Burgard

High-resolution microprobes are used to record single neuron activity in the brain. This technology is envisaged to be a central component for brain-controlled computers and robots. Current neural probes, however, allow for recording only a small number of the densely spaced electrodes simultaneously. Therefore, we address the problem of autonomously choosing, for a given number, the subset of electrodes with the corresponding size so as to extract as much information as possible. We first present an approach for predicting neural spikes across different channels of the probe. Our method employs nonparametric sparse Gaussian process regression to predict the signal of a channel given the signals recorded at neighboring sites. Second, we utilize the signal predictions for efficiently seeking for the subset of electrodes that minimizes the overall prediction error. In experiments carried out using real neural data, we demonstrate that our selection procedure provides highly accurate results. Furthermore, the solutions found in our experiments are close to the optimal solution.

ICRA Conference 2014 Conference Paper

Experimental analysis of dynamic covariance scaling for robust map optimization under bad initial estimates

• Pratik Agarwal
• Giorgio Grisetti
• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Non-linear error minimization methods became widespread approaches for solving the simultaneous localization and mapping problem. If the initial guess is far away from the global minimum, converging to the correct solution and not to a local one can be challenging and sometimes even impossible. This paper presents an experimental analysis of dynamic covariance scaling, a recently proposed method for robust optimization of SLAM graphs, in the context of a poor initialization. Our evaluation shows that dynamic covariance scaling is able to mitigate the effects of poor initializations. In contrast to other methods that first aim at finding a good initial guess to seed the optimization, our method is more elegant because it does not require an additional method for initialization. Furthermore, it can robustly handle data association outliers. Experiments performed with real world and simulated datasets show that dynamic covariance scaling outperforms existing methods, both in the presence and absence of data association outliers.

ICRA Conference 2014 Conference Paper

Helmert's and Bowie's geodetic mapping methods and their relation to graph-based SLAM

• Pratik Agarwal
• Wolfram Burgard
• Cyrill Stachniss

The problem of simultaneously localization a robot and modeling the environment is a prerequisite for several robotic applications and a large variety of solutions have been proposed allowing robots to build maps and use them for navigation. Also the geodetic community addressed large-scale mapping for centuries, computing maps which span across continents. These mapping processes had to deal with several challenges that are similar to those of the robotics community. In this paper, we explain two key geodetic mapping methods that we believe are relevant for robotics. We also aim at providing a geodetic perspective on current state-of-the-art SLAM methods and at identifying similarities between the solutions proposed by both communities. The central goal of this paper is to bring both fields close together and to enable future synergies.

ICRA Conference 2014 Conference Paper

Hierarchical sparse coded surface models

• Michael Ruhnke
• Liefeng Bo
• Dieter Fox
• Wolfram Burgard

In this paper, we describe a novel approach to construct textured 3D environment models in a hierarchical fashion based on local surface patches. Compared to previous approaches, the hierarchy enables our method to represent the environment with differently sized surface patches. The reconstruction scheme starts at a coarse resolution with large patches and in an iterative fashion uses the reconstruction error to guide the decision as to whether the resolution should be refined. This leads to variable resolution models that represent areas with few variations at low resolution and areas with large variations at high resolution. In addition, we compactly describe local surface attributes via sparse coding based on an overcomplete dictionary. In this way, we additionally exploit similarities in structure and texture, which leads to compact models. We learn the dictionary directly from the input data and independently for every level in the hierarchy in an unsupervised fashion. Practical experiments with large-scale datasets demonstrate that our method compares favorably with two state-of-the-art techniques while being comparable in accuracy.

ICRA Conference 2014 Conference Paper

Inferring what to imitate in manipulation actions by using a recommender system

• Nichola Abdo
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Learning from demonstrations is an intuitive way for instructing robots by non-experts. One challenge in learning from demonstrations is to infer what to imitate, especially when the robot only observes the teacher and does not have further knowledge about the demonstrated actions. In this paper, we present a novel approach to the problem of inferring what to imitate to successfully reproduce a manipulation action based on a small number of demonstrations. Our method employs techniques from recommender systems to include expert knowledge. It models the demonstrated actions probabilistically and formulates the problem of inferring what to imitate via model selection. We select an appropriate model for the action each time the robot has to reproduce it given a new starting condition. We evaluate our approach using data acquired with a PR2 robot and demonstrate that our method achieves high success rates in different scenarios.

ICRA Conference 2014 Conference Paper

Learning to give route directions from human demonstrations

• Stefan Oßwald
• Henrik Kretzschmar
• Wolfram Burgard
• Cyrill Stachniss

For several applications, robots and other computer systems must provide route descriptions to humans. These descriptions should be natural and intuitive for the human users. In this paper, we present an algorithm that learns how to provide good route descriptions from a corpus of human-written directions. Using inverse reinforcement learning, our algorithm learns how to select the information for the description depending on the context of the route segment. The algorithm then uses the learned policy to generate directions that imitate the style of the descriptions provided by humans, thus taking into account personal as well as cultural preferences and special requirements of the particular user group providing the learning demonstrations. We evaluate our approach in a user study and show that the directions generated by our policy sound similar to human-given directions and substantially more natural than directions provided by commercial web services.

ICRA Conference 2014 Conference Paper

Learning to predict trajectories of cooperatively navigating agents

• Henrik Kretzschmar
• Markus Kuderer
• Wolfram Burgard

The problem of modeling the navigation behavior of multiple interacting agents arises in different areas including robotics, computer graphics, and behavioral science. In this paper, we present an approach to learn the composite navigation behavior of interacting agents from demonstrations. The decision process that ultimately leads to the observed continuous trajectories of the agents often also comprises discrete decisions, which partition the space of composite trajectories into homotopy classes. Therefore, our method uses a mixture probability distribution that consists of a discrete distribution over the homotopy classes and continuous distributions over the composite trajectories for each homotopy class. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. We exploit that the distributions are highly structured due to physical constraints and guide the sampling process to regions of high probability. We apply our approach to learning the behavior of pedestrians and demonstrate that it outperforms state-of-the-art methods.

ICRA Conference 2014 Conference Paper

Online generation of homotopically distinct navigation paths

• Markus Kuderer
• Christoph Sprunk
• Henrik Kretzschmar
• Wolfram Burgard

In mobile robot navigation, cost functions are a popular approach to generate feasible, safe paths that avoid obstacles and that allow the robot to get from its starting position to the goal position. Alternative ways to navigate around the obstacles typically correspond to different local minima in the cost function. In this paper we present a highly effective approach to overcome such local minima and to quickly propose a set of alternative, topologically different and optimized paths. We furthermore describe how to maintain a set of optimized trajectory alternatives to reduce optimization efforts when the robot has to adapt to changes in the environment. We demonstrate in experiments that our method outperforms a state-of-the-art approach by an order of magnitude in computation time, which allows a robot to use our method online during navigation. We furthermore demonstrate that the approach of using a set of qualitatively different trajectories is beneficial in shared autonomy settings, where a user operating a wheelchair can quickly switch between topologically different trajectories.

ICRA Conference 2014 Conference Paper

Online marker labeling for fully automatic skeleton tracking in optical motion capture

• Johannes Meyer
• Markus Kuderer
• Jörg Müller 0004
• Wolfram Burgard

Methods to accurately capture the motion of humans in motion capture systems from optical markers are important for a large variety of applications including animation, interaction, orthopedics, and rehabilitation. Major challenges in this context are to associate the observed markers with skeleton segments, to track markers between consecutive frames, and to estimate the underlying skeleton configuration for each frame. Existing solutions to this problem often assume fully labeled markers, which usually requires labor-intensive manual labeling, especially when markers are temporally occluded during the movements. In this paper, we propose a fully automated method to initialize and track the skeleton configuration of humans from optical motion capture data without the need of any user intervention. Our method applies a flexible T-pose-based initialization that works with a wide range of marker placements, robustly estimates the skeleton configuration through least-squares optimization, and exploits the skeleton structure for fully automatic marker labeling. We demonstrate the capabilities of our approach for online skeleton tracking and show that our method outperforms solutions that are widely used and considered as state of the art.

ICRA Conference 2014 Conference Paper

Reconstruction of rigid body models from motion distorted laser range data using optical flow

• Eddy Ilg
• Rainer Kümmerle
• Wolfram Burgard
• Thomas Brox

The setup of tilting a 2D laser range finder up and down is a widespread strategy to acquire 3D point clouds. This setup requires that the scene is static while the robot takes a 3D scan. If an object moves through the scene during the measurement process and one does not take into account these movements, the resulting model will get distorted. This paper presents an approach to reconstruct the 3D model of a moving rigid object from the inconsistent set of 2D measurements by the help of a camera. Our approach utilizes optical flow in the camera images to estimate the motion in the image plane and point-line constraints to compensate the missing information about the motion in depth. We combine multiple sweeps and/or views into to a single consistent model using a point-to-plane ICP approach and optimize single sweeps by smoothing the resulting trajectory. Experiments obtained in real outdoor scenarios with moving cars demonstrate that our approach yields accurate models.

AAAI Conference 2014 Conference Paper

Robust Visual Robot Localization Across Seasons Using Network Flows

• Tayyab Naseer
• Luciano Spinello
• Wolfram Burgard
• Cyrill Stachniss

Image-based localization is an important problem in robotics and an integral part of visual mapping and navigation systems. An approach to robustly match images to previously recorded ones must be able to cope with seasonal changes especially when it is supposed to work reliably over long periods of time. In this paper, we present a novel approach to visual localization of mobile robots in outdoor environments, which is able to deal with substantial seasonal changes. We formulate image matching as a minimum cost flow problem in a data association graph to effectively exploit sequence information. This allows us to deal with non-matching image sequences that result from temporal occlusions or from visiting new places. We present extensive experimental evaluations under substantial seasonal changes. Our approach achieves accurate matching across seasons and outperforms existing state-of-the-art methods such as FABMAP2 and SeqSLAM.

ICRA Conference 2014 Conference Paper

W-RGB-D: Floor-plan-based indoor global localization using a depth camera and WiFi

• Seigo Ito
• Felix Endres
• Markus Kuderer
• Gian Diego Tipaldi
• Cyrill Stachniss
• Wolfram Burgard

Localization approaches typically rely on an already available map to identify the position of the sensor in the environment. Such maps are usually built beforehand and often require the user to record data from the same sensor used for localization. In this paper, we relax this assumption and present a localization approach based on architectural floor plans. In general, floor plans are readily available for most man-made buildings but only represent basic architectural structures. The incomplete knowledge leads to ambiguous pose estimates. To solve this problem, we present W-RGB-D, a new method for indoor global localization based on WiFi and an RGB-D camera. We introduce a sensor model for RGB-D cameras that is suitable to be used with abstract floor plans. To resolve ambiguities during global localization, we estimate a coarse initial distribution about the sensor position using the WiFi signal strength. We evaluate our W-RGB-D localization method in indoor environments and compare its performance with RGB-D-based Monte Carlo localization. Our results demonstrate that the use of WiFi information as proposed with our approach improves the localization in terms of convergence speed and quality of the solution.

ICRA Conference 2013 Conference Paper

A navigation system for robots operating in crowded urban environments

• Rainer Kümmerle
• Michael Ruhnke
• Bastian Steder
• Cyrill Stachniss
• Wolfram Burgard

Over the past years, there has been a tremendous progress in the area of robot navigation. Most of the systems developed thus far, however, are restricted to indoor scenarios, non-urban outdoor environments, or road usage with cars. Urban areas introduce numerous challenges to autonomous mobile robots as they are highly complex and in addition to that dynamic. In this paper, we present a navigation system for pedestrian-like autonomous navigation with mobile robots in city environments. We describe different components including a SLAM system for dealing with huge maps of city centers, a planning approach for inferring feasible paths taking also into account the traversability and type of terrain, and a method for accurate localization in dynamic environments. The navigation system has been implemented and tested in several large-scale field tests in which the robot Obelix managed to autonomously navigate from our university campus over a 3. 3 km long route to the city center of Freiburg.

AAAI Conference 2013 Conference Paper

Compact RGBD Surface Models Based on Sparse Coding

• Michael Ruhnke
• Liefeng Bo
• Dieter Fox
• Wolfram Burgard

In this paper, we describe a novel approach to construct compact colored 3D environment models representing local surface attributes via sparse coding. Our method decomposes a set of colored point clouds into local surface patches and encodes them based on an overcomplete dictionary. Instead of storing the entire point cloud, we store a dictionary, surface patch positions, and a sparse code description of the depth and RGB attributes for every patch. The dictionary is learned in an unsupervised way from surface patches sampled from indoor maps. We show that better dictionaries can be learned by extending the K-SVD method with a binary weighting scheme that ignores undefined surface cells. Through experimental evaluation on real world laser and RGBD datasets we demonstrate that our method produces compact and accurate models. Furthermore, we clearly outperform an existing state of the art method in terms of compactness, accuracy, and computation time. Additionally, we demonstrate that our sparse code descriptions can be utilized for other important tasks such as object detection.

ICRA Conference 2013 Conference Paper

Cooperative robot localization and target tracking based on least squares minimization

• Aamir Ahmad
• Gian Diego Tipaldi
• Pedro U. Lima
• Wolfram Burgard

In this paper we address the problem of cooperative localization and target tracking with a team of moving robots. We model the problem as a least squares minimization problem and show that this problem can be efficiently solved using sparse optimization methods. To achieve this, we represent the problem as a graph, where the nodes are robot and target poses at individual time-steps and the edges are their relative measurements. Static landmarks at known position are used to define a common reference frame for the robots and the targets. In this way, we mitigate the risk of using measurements and state estimates more than once, since all the relative measurements are i. i. d. and no marginalization is performed. Experiments performed using a set of real robots show higher accuracy compared to a Kalman filter.

ICRA Conference 2013 Conference Paper

Deploying artificial landmarks to foster data association in simultaneous localization and mapping

• Maximilian Beinhofer
• Henrik Kretzschmar
• Wolfram Burgard

Data association is an essential problem in simultaneous localization and mapping. It is hard to solve correctly, especially in ambiguous environments. We consider a scenario where the robot can ease the data association problem by deploying a limited number of uniquely identifiable artificial landmarks along its path and use them afterwards as fixed anchors. Obviously, the choice of the positions where the robot should drop these markers is crucial as poor choices might prevent the robot from establishing accurate data associations. In this paper, we present a novel approach for learning when to drop the landmarks so as to optimize the data association performance. We use Monte Carlo reinforcement learning for computing an optimal policy and apply a statistical convergence test to decide if the policy is converged and the learning process can be stopped. Extensive experiments also carried out with a real robot demonstrate that the data association performance using landmarks deployed according to our learned policies is significantly higher compared to other strategies.

IROS Conference 2013 Conference Paper

Efficient navigation for anyshape holonomic mobile robots in dynamic environments

• Marija Dakulovic
• Christoph Sprunk
• Luciano Spinello
• Ivan Petrovic
• Wolfram Burgard

Platforms with holonomic drives are particularly interesting due to their maneuvering capabilities. Robots used for transportation tasks usually have a non-circular footprint. In this work, we present a navigation strategy for a holonomic mobile robot with anyshape footprint. Our technique introduces an efficient navigation method based on a strategy that makes use of discrete and continuous techniques. We introduce compact discrete intervals to represent the free space for computing fast-to-update plans. Based on these, we provide a continuous motion generation approach to generate smooth motions that are fast to compute. We evaluated our approach by running simulated experiments and by using a real holonomic L-shaped robot. Our experiments demonstrate that our technique can be carried out online and is able to smoothly drive the robot to its goal locations even in dynamic environments.

ICRA Conference 2013 Conference Paper

Geometrical FLIRT phrases for large scale place recognition in 2D range data

• Gian Diego Tipaldi
• Luciano Spinello
• Wolfram Burgard

Place recognition, i. e. , the problem of recognizing if the robot is navigating in an already visited place, is a fundamental problem in mobile robot navigation. Efficient solutions to this problem are relevant for effectively localizing robots and for creating maps in real time. Relatively few methods have been proposed to efficiently solve this problem in very large environments using 2D range data. In this paper, we introduce geometrical FLIRT phrases (GFPs) as a novel retrieval method for very efficient and precise place recognition. GFPs perform approximate 2D range data matching, have low computational cost, can handle complicated partial matching patterns and are robust to noise. Experiments carried out with publicly available datasets demonstrate that GFPs largely outperform state-of-the-art approaches in 2D range-based place recognition in terms of efficiency and recall. We obtain retrieval performances with more than 85% recall at 99% precision in less than a second, even on data sets obtained from several kilometer long runs.

IJCAI Conference 2013 Conference Paper

Learning Probabilistic Models for Mobile Manipulation Robots

• Jürgen Sturm
• Wolfram Burgard

Mobile manipulation robots are envisioned to provide many useful services both in domestic environments as well as in the industrial context. In this paper, we present novel approaches to allow mobile maniplation systems to autonomously adapt to new or changing situations. The approaches developed in this paper cover the following four topics: (1) learning the robot’s kinematic structure and properties using actuation and visual feedback, (2) learning about articulated objects in the environment in which the robot is operating, (3) using tactile feedback to augment visual perception, and (4) learning novel manipulation tasks from human demonstrations.

IROS Conference 2013 Conference Paper

Learning the dynamics of doors for robotic manipulation

• Felix Endres
• Jeffrey C. Trinkle
• Wolfram Burgard

Opening doors is a fundamental skill for mobile robots operating in human environments. In this paper we present an approach to learn a dynamic model of a door from sensor observations and utilize it for effectively swinging the door open to a desired angle. The learned model enables the realization of dynamic door-opening strategies and reduces the complexity of the door opening task. For example, the robot does not need to maintain a grasp of the handle, which would form a closed kinematic chain. Accordingly, it reduces the degrees of freedom required of the manipulator and facilitates motion planning. Additionally, execution is faster, because the robot merely needs to push the door long enough to achieve the right combination of position and speed such that the door stops at the desired state. Our approach applies Gaussian process regression to learn the deceleration of the door with respect to position and velocity of the door. This model of the dynamics can be easily learned from observing a human teacher or by interactive experimentation.

IROS Conference 2013 Conference Paper

Learning to guide random tree planners in high dimensional spaces

• Jörg Röwekämper
• Gian Diego Tipaldi
• Wolfram Burgard

In this paper we present the projection and bias heuristic (PBH), a motion planning algorithm that makes use of low-dimensional projections to improve sampling-based planning algorithms. In contrast to other state-of-the-art methods, we do not assume that projections are either random or given by an expert user. Rather, our goal is to learn projections such that planning on them improves the efficiency and the quality of solutions. We present both, a method to learn effective projections and a sampling algorithm that makes use of them. We show that our approach can be easily integrated into popular sampling-based planners. Extensive experiments performed in simulated environments demonstrate that our approach produces paths that are in general shorter than those obtained with state-of-the-art algorithms. Moreover, it generally requires less computation time.

IROS Conference 2013 Conference Paper

Lidar-based teach-and-repeat of mobile robot trajectories

• Christoph Sprunk
• Gian Diego Tipaldi
• Andrea Cherubini
• Wolfram Burgard

Automation of logistics tasks for small lot sizes and flexible production processes requires intuitive and easy-to-use systems that allow non-expert shop floor workers to naturally instruct transportation systems. To this end, we present a novel laser-based scheme for teach-and-repeat of mobile robot trajectories that relies on scan matching to localize the robot relative to a taught trajectory, which is represented by a sequence of raw odometry and 2D laser data. This approach has two advantages. First, it does not require to build a globally consistent metrical map of the environment, which reduces setup time. Second, the direct use of raw sensor data avoids additional errors that might be introduced by the fact that grid maps only provide an approximation of the environment. Real-world experiments carried out with a holonomic and a differential drive platform demonstrate that our approach repeats trajectories with an accuracy of a few millimeters. A comparison with a standard Monte Carlo localization approach on grid maps furthermore reveals that our method yields lower tracking errors for teach-and-repeat tasks.

ICRA Conference 2013 Conference Paper

Poisson-driven dirt maps for efficient robot cleaning

• Jürgen Hess 0001
• Maximilian Beinhofer
• Daniel Kuhner
• Philipp Ruchti
• Wolfram Burgard

Being able to estimate the dirt distribution in an environment makes it possible to compute efficient paths for robotic cleaners. In this paper, we present a novel approach for modeling and estimating the dynamics of the generation of dirt in an environment. Our model uses cell-wise Poisson processes on a regular grid to estimate the distribution of dirt in the environment. It allows for an effective estimation of the dynamics of the generation of dirt and for making predictions about the absolute dirt values. We propose two efficient cleaning policies that are based on the estimated dirt distributions and can easily be adapted to different needs of potential users. Through extensive experiments carried out with a modified iRobot Roomba vacuum cleaning robot and in simulation we demonstrate the effectiveness of our approach.

RLDM Conference 2013 Conference Abstract

Predicting Human Navigation Behavior via Inverse Reinforcement Learning

• Henrik Kretzschmar
• Markus Kuderer
• Wolfram Burgard

We present an approach that allows a mobile robot to learn the behavior of pedestrians from observed trajectories. Our method maintains probability distributions over composite trajectories of all the pedestrians and represents these distributions as a mixture model. The upper level of this model repre- sents a discrete distribution over classes of trajectories that are equivalent according to a set of features, such as passing on the left or passing on the right side. The lower level comprises continuous probability distributions over trajectories for each of these classes and captures physical features of the trajectories, such as velocities and accelerations. For each level, our method learns maximum entropy distributions that match the feature values of the observations. To estimate the expected feature values with respect to the high-dimensional probability distributions over the composite trajectories, our approach applies Hamiltoni- an Markov Chain Monte Carlo sampling. The extensive experimental evaluation suggests that our method models human navigation behavior more accurately than state-of-the-art techniques.

IROS Conference 2013 Conference Paper

Robust landmark selection for mobile robot navigation

• Maximilian Beinhofer
• Jörg Müller 0004
• Andreas Krause 0001
• Wolfram Burgard

Precise navigation is a key capability of autonomous mobile robots and required for many tasks including transportation or docking. To guarantee a robust and accurate localization and navigation performance, many practical approaches rely on observations of artificial landmarks. This raises the question of where to place the landmarks along the desired trajectory of the robot. In this paper, we present a novel approach to landmark selection, which aims at selecting the minimal set of landmarks that bounds the uncertainty about the deviation of the robot from its desired trajectory. At the same time the selected landmark sets are robust against the fact that a certain number of landmarks can be obscured from view during operation. Our algorithm is highly efficient due to a linearization of the whole navigation cycle and employs submodular optimization, for which strong formal bounds on the approximation quality are known. In extensive experiments, also carried out with a real robot, we demonstrate that our approach outperforms several other methods and that it enables robust autonomous robot navigation in practice.

ICRA Conference 2013 Conference Paper

Robust map optimization using dynamic covariance scaling

• Pratik Agarwal
• Gian Diego Tipaldi
• Luciano Spinello
• Cyrill Stachniss
• Wolfram Burgard

Developing the perfect SLAM front-end that produces graphs which are free of outliers is generally impossible due to perceptual aliasing. Therefore, optimization back-ends need to be able to deal with outliers resulting from an imperfect front-end. In this paper, we introduce dynamic covariance scaling, a novel approach for effective optimization of constraint networks under the presence of outliers. The key idea is to use a robust function that generalizes classical gating and dynamically rejects outliers without compromising convergence speed. We implemented and thoroughly evaluated our method on publicly available datasets. Compared to recently published state-of-the-art methods, we obtain a substantial speed up without increasing the number of variables in the optimization process. Our method can be easily integrated in almost any SLAM back-end.

IROS Conference 2013 Conference Paper

Teaching mobile robots to cooperatively navigate in populated environments

• Markus Kuderer
• Henrik Kretzschmar
• Wolfram Burgard

Mobile service robots are envisioned to operate in environments that are populated by humans and therefore ought to navigate in a socially compliant way. Since the desired behavior of the robots highly depends on the application, we need flexible means for teaching a robot a certain navigation policy. We present an approach that allows a mobile robot to learn how to navigate in the presence of humans while it is being teleoperated in its designated environment. Our method applies feature-based maximum entropy learning to derive a navigation policy from the interactions with the humans. The resulting policy maintains a probability distribution over the trajectories of all the agents that allows the robot to cooperatively avoid collisions with humans. In particular, our method reasons about multiple homotopy classes of the agents' trajectories, i. e. , on which sides the agents pass each other. We implemented our approach on a real mobile robot and demonstrate that it is able to successfully navigate in an office environment in the presence of humans relying only on on-board sensors.

IROS Conference 2012 Conference Paper

A benchmark for the evaluation of RGB-D SLAM systems

• Jürgen Sturm
• Nikolas Engelhard
• Felix Endres
• Wolfram Burgard
• Daniel Cremers

In this paper, we present a novel benchmark for the evaluation of RGB-D SLAM systems. We recorded a large set of image sequences from a Microsoft Kinect with highly accurate and time-synchronized ground truth camera poses from a motion capture system. The sequences contain both the color and depth images in full sensor resolution (640 × 480) at video frame rate (30 Hz). The ground-truth trajectory was obtained from a motion-capture system with eight high-speed tracking cameras (100 Hz). The dataset consists of 39 sequences that were recorded in an office environment and an industrial hall. The dataset covers a large variety of scenes and camera motions. We provide sequences for debugging with slow motions as well as longer trajectories with and without loop closures. Most sequences were recorded from a handheld Kinect with unconstrained 6-DOF motions but we also provide sequences from a Kinect mounted on a Pioneer 3 robot that was manually navigated through a cluttered indoor environment. To stimulate the comparison of different approaches, we provide automatic evaluation tools both for the evaluation of drift of visual odometry systems and the global pose error of SLAM systems. The benchmark website [1] contains all data, detailed descriptions of the scenes, specifications of the data formats, sample code, and evaluation tools.

ICRA Conference 2012 Conference Paper

An evaluation of the RGB-D SLAM system

• Felix Endres
• Jürgen Hess 0001
• Nikolas Engelhard
• Jürgen Sturm
• Daniel Cremers
• Wolfram Burgard

We present an approach to simultaneous localization and mapping (SLAM) for RGB-D cameras like the Microsoft Kinect. Our system concurrently estimates the trajectory of a hand-held Kinect and generates a dense 3D model of the environment. We present the key features of our approach and evaluate its performance thoroughly on a recently published dataset, including a large set of sequences of different scenes with varying camera speeds and illumination conditions. In particular, we evaluate the accuracy, robustness, and processing time for three different feature descriptors (SIFT, SURF, and ORB). The experiments demonstrate that our system can robustly deal with difficult data in common indoor scenarios while being fast enough for online operation. Our system is fully available as open-source.

ICRA Conference 2012 Conference Paper

Fully distributed scalable smoothing and mapping with robust multi-robot data association

• Alexander Cunningham
• Kai M. Wurm
• Wolfram Burgard
• Frank Dellaert

In this paper we focus on the multi-robot perception problem, and present an experimentally validated end-to-end multi-robot mapping framework, enabling individual robots in a team to see beyond their individual sensor horizons. The inference part of our system is the DDF-SAM algorithm [1], which provides a decentralized communication and inference scheme, but did not address the crucial issue of data association. One key contribution is a novel, RANSAC-based, approach for performing the between-robot data associations and initialization of relative frames of reference. We demonstrate this system with both data collected from real robot experiments, as well as in a large scale simulated experiment demonstrating the scalability of the proposed approach.

ICRA Conference 2012 Conference Paper

Highly accurate 3D surface models by sparse surface adjustment

• Michael Ruhnke
• Rainer Kümmerle
• Giorgio Grisetti
• Wolfram Burgard

In this paper, we propose an approach to obtain highly accurate 3D models from range data. The key idea of our method is to jointly optimize the poses of the sensor and the positions of the surface points measured with a range scanning device. Our approach applies a physical model of the underlying range sensor. To solve the optimization task it employs a state-of-the-art graph-based optimizer and iteratively refines the structure of the error function by recomputing the data associations after each optimization. We present our approach and evaluate it on data recorded in different real world environments with a RGBD camera and a laser range scanner. The experimental results demonstrate that our method is able to substantially improve the accuracy of SLAM results and that it compares favorable over the moving least squares method.

ICRA Conference 2012 Conference Paper

Improved non-linear spline fitting for teaching trajectories to mobile robots

• Christoph Sprunk
• Boris Lau
• Wolfram Burgard

In this paper, we present improved spline fitting techniques with the application of trajectory teaching for mobile robots. Given a recorded reference trajectory, we apply non-linear least-squares optimization to accurately approximate the trajectory using a parametric spline. The fitting process is carried out without fixed correspondences between data points and points along the spline, which improves the fit especially in sharp curves. By using a specific path model, our approach requires substantially fewer free parameters than standard approaches to achieve similar residual errors. Thus, the generated paths are ideal for subsequent optimization to reduce the time of travel or for the combination with autonomous planning to evade obstacles blocking the path. Our experiments on real-world data demonstrate the advantages of our method in comparison with standard approaches.

IROS Conference 2012 Conference Paper

Null space optimization for effective coverage of 3D surfaces using redundant manipulators

• Jürgen Hess 0001
• Gian Diego Tipaldi
• Wolfram Burgard

In this paper we consider the problem of null space minimization in coverage path planning of 3D surfaces for redundant manipulators. Existing coverage solutions only focus on Euclidean cost functions and often return suboptimal paths with respect to the joint space. In the approach described here, we explicitly consider the null space by treating different inverse kinematics solutions as individual nodes in a graph and model the problem as a generalized traveling salesman problem (GTSP). The GTSP is a generalization of the TSP where the nodes of the graph are subdivided into clusters and at least one node in each cluster needs to be visited. We evaluate our approach using a PR2 robot and complex objects. Our results demonstrate that our method outperforms Euclidean coverage algorithms in terms of manipulation effort and completion time.

AAAI Conference 2012 Conference Paper

Occupancy Grid Models for Robot Mapping in Changing Environments

• Daniel Meyer-Delius
• Maximilian Beinhofer
• Wolfram Burgard

The majority of existing approaches to mobile robot mapping assumes that the world is static, which is generally not justified in real-world applications. However, in many navigation tasks including trajectory planning, surveillance, and coverage, accurate maps are essential for the effective behavior of the robot. In this paper we present a probabilistic grid-based approach for modeling changing environments. Our method represents both, the occupancy and its changes in the corresponding area where the dynamics are characterized by the state transition probabilities of a Hidden Markov Model. We apply an offline and an online technique to learn the parameters from observed data. The advantage of the online approach is that it can dynamically adapt the parameters and at the same time does not require storing the complete observation sequences. Experimental results obtained with data acquired by real robots demonstrate that our model is well-suited for representing changing environments. Further results show that our technique can be used to substantially improve the effectiveness of path planning procedures.

IROS Conference 2012 Conference Paper

On the position accuracy of mobile robot localization based on particle filters combined with scan matching

• Jörg Röwekämper
• Christoph Sprunk
• Gian Diego Tipaldi
• Cyrill Stachniss
• Patrick Pfaff
• Wolfram Burgard

Many applications in mobile robotics and especially industrial applications require that the robot has a precise estimate about its pose. In this paper, we analyze the accuracy of an integrated laser-based robot pose estimation and positioning system for mobile platforms. For our analysis, we used a highly accurate motion capture system to precisely determine the error in the robot's pose. We are able to show that by combining standard components such as Monte-Carlo localization, KLD sampling, and scan matching, an accuracy of a few millimeters at taught-in reference locations can be achieved. We believe that this is an important analysis for developers of robotic applications in which pose accuracy matters.

ECAI Conference 2012 Invited Paper

Probabilistic Techniques for Mobile Robot Navigation

• Wolfram Burgard

Probabilistic approaches have been discovered as one of the most powerful approaches to highly relevant problems in mobile robotics including perception and robot state estimation. Major challenges in the context of probabilistic algorithms for mobile robot navigation lie in the questions of how to deal with highly complex state estimation problems and how to control the robot so that it efficiently carries out its task. In this talk, I will present recently developed techniques for efficiently learning a map of an unknown environment with a mobile robot. I will also describe how this state estimation problem can be solved more effectively by actively controlling the robot. For all algorithms I will present experimental results that have been obtained with mobile robots in real-world environments.

ICRA Conference 2012 Conference Paper

Probabilistic velocity estimation for autonomous miniature airships using thermal air flow sensors

• Jörg Müller 0004
• Oliver Paul
• Wolfram Burgard

Recently, autonomous miniature airships have become a growing research field. Whereas airships are attractive as they can move freely in the three-dimensional space, their high-dimensional state space and the restriction to small and lightweight sensors are demanding constraints with respect to self-localization. Furthermore, their complex second-order kinematics makes the estimation of their pose and velocity through dead reckoning odometry difficult and imprecise. In this paper, we consider the problem of estimating the velocity of a miniature blimp with lightweight air flow sensors. We present a probabilistic sensor model that accurately models the uncertainty of the flow sensors and thus allows for robust state estimation using a particle filter. In experiments carried out with a real airship we demonstrate that our method precisely estimates the velocity of the blimp and outperforms the standard velocity estimates of the motion model as applied in many existent autonomous blimp navigation systems.

IROS Conference 2011 Conference Paper

Accurate human motion capture in large areas by combining IMU- and laser-based people tracking

• Jakob Ziegler
• Henrik Kretzschmar
• Cyrill Stachniss
• Giorgio Grisetti
• Wolfram Burgard

A large number of applications use motion capture systems to track the location and the body posture of people. For instance, the movie industry captures actors to animate virtual characters that perform stunts. Today's tracking systems either operate with statically mounted cameras and thus can be used in confined areas only or rely on inertial sensors that allow for free and large-scale motion but suffer from drift in the pose estimate. This paper presents a novel tracking approach that aims to provide globally aligned full body posture estimates by combining a mobile robot and an inertial motion capture system. In our approach, a mobile robot equipped with a laser scanner is used to anchor the pose estimates of a person given a map of the environment. It uses a particle filter to globally localize a person wearing a motion capture suit and to robustly track the person's position. To obtain a smooth and globally aligned trajectory of the person, we solve a least squares optimization problem formulated from the motion capture suite and tracking data. Our approach has been implemented on a real robot and exhaustively tested. As the experimental evaluation shows, our system is able to provide locally precise and globally aligned estimates of the person's full body posture.

IROS Conference 2011 Conference Paper

Autonomous miniature blimp navigation with online motion planning and re-planning

• Jörg Müller 0004
• Norman Kohler
• Wolfram Burgard

In recent years, there has been an increasing interest in autonomous navigation for lightweight flying robots in indoor environments. Miniature airships, which are an instance of such robots, are especially challenging since they behave nonlinearly, typically are under-actuated, and also are subject to drift. These aspects, paired with their high-dimensional state space, demand efficient planning and control techniques. In this paper, we present a highly effective approach to autonomous navigation of miniature blimps in mapped environments which applies a multi-stage algorithm to accomplish strongly goal-directed tree-based kinodynamic planning. It performs path-guided sampling and optimally selects actions leading the robot towards sampled subgoals. Based on this, our approach can quickly provide a partial trajectory, which is extended and refined in the consecutive planning steps. The navigation system has been implemented and is able to reliably operate a robotic blimp in a real-world setting. Further experiments demonstrate that our approach outperforms a standard tree planner.

IROS Conference 2011 Conference Paper

Efficient motion planning for manipulation robots in environments with deformable objects

• Barbara Frank
• Cyrill Stachniss
• Nichola Abdo
• Wolfram Burgard

The ability to plan their own motions and to reliably execute them is an important precondition for autonomous robots. In this paper, we consider the problem of planning the motion of a mobile manipulation robot in the presence of deformable objects. Our approach combines probabilistic roadmap planning with a physical deformation simulation system. Since the physical deformation simulation is computationally demanding, we use efficient Gaussian process regression to estimate the deformation cost for individual objects based on training examples. We generate the training data by employing a simulation system in a preprocessing step. Consequently, no simulations are needed during runtime. We implemented and tested our approach on a mobile manipulation robot. Our experiments show that the robot is able to accurately predict and thus consider the deformation cost its manipulator introduces to the environment during motion planning. Simultaneously, the computation time is substantially reduced compared to a system that employs physical simulations online.

ICRA Conference 2011 Conference Paper

G 2 o: A general framework for graph optimization

• Rainer Kümmerle
• Giorgio Grisetti
• Hauke Strasdat
• Kurt Konolige
• Wolfram Burgard

Many popular problems in robotics and computer vision including various types of simultaneous localization and mapping (SLAM) or bundle adjustment (BA) can be phrased as least squares optimization of an error function that can be represented by a graph. This paper describes the general structure of such problems and presents g 2 o, an open-source C++ framework for optimizing graph-based nonlinear error functions. Our system has been designed to be easily extensible to a wide range of problems and a new problem typically can be specified in a few lines of code. The current implementation provides solutions to several variants of SLAM and BA. We provide evaluations on a wide range of real-world and simulated datasets. The results demonstrate that while being general g 2 o offers a performance comparable to implementations of state of-the-art approaches for the specific problems.

IROS Conference 2011 Conference Paper

Hierarchies of octrees for efficient 3D mapping

• Kai M. Wurm
• Daniel Hennes
• Dirk Holz
• Radu Bogdan Rusu
• Cyrill Stachniss
• Kurt Konolige
• Wolfram Burgard

The on-chip fabrication and manipulation of microstructures are expected to be applied for single cell analysis system such as cell manipulation and measurement tools. In this paper, we previously present a methodology for fabricating and assembling microstructures inside a microfluidic channel. By the illumination of patterned UV-ray through the mask under a microscope, microstructures with arbitrary shape are made of the photo-crosslinkable resin inside microfluidic device. The microstructures are fabricated at the desired place inside microfluidic channel and manipulated by optical tweezers. Based on the technique which can manipulate multiple points simultaneously by high-speed scanning of a single laser with galvanometer mirror, a rotational microstructure made of a microgear and a rotation axis is assembled and rotated. We also report two methods of solution replacement inside microfluidic channel which reduces viscosity of solvent in order to improve manipulation performance. By adjusting the concentration of photo-crosslinkable resin and replacing solution components, the viscosity of solvent inside channel can be changed. The manipulation speed of the rotational microstructure increases when the viscosity of solvent decreases, because the viscosity resistance for the movement of microstructure is weaker inside lower viscosity solvent. We fabricate rotational microstructures inside lower viscosity solvent and evaluate the movement efficiency compared with microstructures inside former high viscosity solvent.

ICRA Conference 2011 Conference Paper

Highly accurate maximum likelihood laser mapping by jointly optimizing laser points and robot poses

• Michael Ruhnke
• Rainer Kümmerle
• Giorgio Grisetti
• Wolfram Burgard

In this paper we describe an algorithm for learning highly accurate laser-based maps that treats the overall mapping problem as a joint optimization problem over robot poses and laser points. We assume that a laser range finder senses points sampled from a regular surface and we utilize an improved likelihood function that accounts for two phenomena affecting the laser measurements that are often neglected: the conic shape of the laser beam and the incidence angle. To solve the entire problem we apply an optimization procedure that jointly adjusts the position of all the robot poses and all points in the scans. As a result, we obtain highly accurate maps. We evaluated our approach using simulated and real-world data and we show that utilizing the estimated maps greatly improves the localization accuracy of robots. The results furthermore suggest that the accuracy of the resulting map can exceed the resolution of the laser sensors used.

ICRA Conference 2011 Conference Paper

Near-optimal landmark selection for mobile robot navigation

• Maximilian Beinhofer
• Jörg Müller 0004
• Wolfram Burgard

The ability to accurately localize themselves is a fundamental pre-condition for service robots designed to carry out navigation and transportation tasks. Because of the high degree of dynamics in populated and real-world environments, often artificial landmarks are used to achieve the desired accuracy in localization. In this paper we consider the problem of optimally placing landmarks for robots navigating frequently on similar trajectories. Our method maximizes conditional mutual information of the states of the robot given the landmark observations. It uses a greedy algorithm which approximates the solution of the NP-hard maximization problem. For this algorithm, we derive a tight constant-factor bound on the approximation error. We furthermore evaluate the selected landmark sets in extensive experiments carried out both in simulation and with a real robot.

ICRA Conference 2011 Conference Paper

Online generation of kinodynamic trajectories for non-circular omnidirectional robots

• Christoph Sprunk
• Boris Lau
• Patrick Pfaff
• Wolfram Burgard

This paper presents a novel approach to kino-dynamic trajectory generation for non-circular omnidirectional platforms that can be combined with existing path planners. We use quintic Bézier splines to specify position and orientation of the holonomic robot for every point in time. To fully exploit the capabilities of the holonomic robot we propose a novel path representation. It allows for continuous variation of path shapes in the spectrum between straight-line paths with turns on the spot and smooth paths with independent rotations and translations. Using this representation our method optimizes trajectories according to a user-defined cost function, considering the constraints of the platform. This way, it generates fast and efficient trajectories in an anytime fashion. The experiments carried out on an industrial robot show that our approach generates highly efficient and smooth motion trajectories that can be tracked with high precision and predictability. Furthermore, the system operates in real-world environments containing unmapped obstacles and narrow passages.

IROS Conference 2011 Conference Paper

Place recognition in 3D scans using a combination of bag of words and point feature based relative pose estimation

• Bastian Steder
• Michael Ruhnke
• Slawomir Grzonka
• Wolfram Burgard

Place recognition, i. e. , the ability to recognize previously seen parts of the environment, is one of the fundamental tasks in mobile robotics. The wide range of applications of place recognition includes localization (determine the initial pose), SLAM (detect loop closures), and change detection in dynamic environments. In the past, only relatively little work has been carried out to attack this problem using 3D range data and the majority of approaches focuses on detecting similar structures without estimating relative poses. In this paper, we present an algorithm based on 3D range data that is able to reliably detect previously seen parts of the environment and at the same time calculates an accurate transformation between the corresponding scan-pairs. Our system uses the estimated transformation to evaluate a candidate and in this way to more robustly reject false positives for place recognition. We present an extensive set of experiments using publicly available datasets in which we compare our system to other state-of-the-art approaches.

ICRA Conference 2011 Conference Paper

Point feature extraction on 3D range scans taking into account object boundaries

• Bastian Steder
• Radu Bogdan Rusu
• Kurt Konolige
• Wolfram Burgard

In this paper we address the topic of feature extraction in 3D point cloud data for object recognition and pose identification. We present a novel interest keypoint extraction method that operates on range images generated from arbitrary 3D point clouds, which explicitly considers the borders of the objects identified by transitions from foreground to background. We furthermore present a feature descriptor that takes the same information into account. We have implemented our approach and present rigorous experiments in which we analyze the individual components with respect to their repeatability and matching capabilities and evaluate the usefulness for point feature based object detection methods.

IROS Conference 2011 Conference Paper

Simultaneous calibration, localization, and mapping

• Rainer Kümmerle
• Giorgio Grisetti
• Wolfram Burgard

The calibration parameters of a mobile robot play a substantial role in navigation tasks. Often these parameters are subject to variations that depend either on environmental changes or on the wear of the devices. In this paper, we propose an approach to simultaneously estimate a map of the environment, the position of the on-board sensors of the robot, and its kinematic parameters. Our method requires no prior knowledge about the environment and relies only on a rough initial guess of the platform parameters. The proposed approach performs on-line estimation of the parameters and it is able to adapt to non-stationary changes of the configuration. We tested our approach in simulated environments and on a wide range of real world data using different types of robotic platforms.

ICRA Conference 2011 Conference Paper

Using artificial landmarks to reduce the ambiguity in the environment of a mobile robot

• Daniel Meyer-Delius
• Maximilian Beinhofer
• Alexander Kleiner
• Wolfram Burgard

Robust and reliable localization is a fundamental prerequisite for many applications of mobile robots. Although there exist many solutions to the localization problem, structurally symmetrical or featureless environments can prevent different locations from being distinguishable given the data obtained with the robot's sensors. Such ambiguities typically make localization approaches more likely to fail. In this paper, we investigate how artificial landmarks can be utilized to reduce the ambiguity in the environment. We present a practical approach to compute a configuration of indistinguishable landmarks that decreases the overall ambiguity and thus increases the robustness of the localization process. We evaluate our approach in different environments based on real data and in simulation. Our results demonstrate that our approach improves the localization performance of the robot and outperforms other landmark selection approaches.

ICRA Conference 2010 Conference Paper

Consistent mapping of multistory buildings by introducing global constraints to graph-based SLAM

• Michael Karg
• Kai M. Wurm
• Cyrill Stachniss
• Klaus Dietmayer
• Wolfram Burgard

In the past, there has been a tremendous advance in the area of simultaneous localization and mapping (SLAM). However, there are relatively few approaches for incorporating prior information or knowledge about structural similarities into the mapping process. Consider, for example, office buildings in which most of the offices have an identical geometric layout. The same typically holds for the individual stories of buildings. In this paper, we propose an approach for generating alignment constraints between different floors of the same building in the context of graph-based SLAM. This is done under the assumption that the individual floors of a building share at least some structural properties. To identify such areas, we apply a particle filter-based localization approach using maps and observations from different floors. We evaluate our system using several real datasets as well as in simulation. The results demonstrate that our approach is able to correctly align multiple floors and allows the robot to generate consistent models of multi-story buildings.

IROS Conference 2010 Conference Paper

Coordinated exploration with marsupial teams of robots using temporal symbolic planning

• Kai M. Wurm
• Christian Dornhege
• Patrick Eyerich
• Cyrill Stachniss
• Bernhard Nebel
• Wolfram Burgard

The problem of autonomously exploring an environment with a team of robots received considerable attention in the past. However, there are relatively few approaches to coordinate teams of robots that are able to deploy and retrieve other robots. Efficiently coordinating the exploration with such marsupial robots requires advanced planning mechanisms that are able to consider symbolic deployment and retrieval actions. In this paper, we propose a novel approach for coordinating the exploration with marsupial robot teams. Our method integrates a temporal symbolic planner that explicitly considers deployment and retrieval actions with a traditional cost-based assignment procedure. Our approach has been implemented and evaluated in several simulated environments and with varying team sizes. The results demonstrate that our proposed method is able to coordinate marsupial teams of robots to efficiently explore unknown environments.

IROS Conference 2010 Conference Paper

Efficient Sparse Pose Adjustment for 2D mapping

• Kurt Konolige
• Giorgio Grisetti
• Rainer Kümmerle
• Wolfram Burgard
• Benson Limketkai
• Régis Vincent

Pose graphs have become a popular representation for solving the simultaneous localization and mapping (SLAM) problem. A pose graph is a set of robot poses connected by nonlinear constraints obtained from observations of features common to nearby poses. Optimizing large pose graphs has been a bottleneck for mobile robots, since the computation time of direct nonlinear optimization can grow cubically with the size of the graph. In this paper, we propose an efficient method for constructing and solving the linear subproblem, which is the bottleneck of these direct methods. We compare our method, called Sparse Pose Adjustment (SPA), with competing indirect methods, and show that it outperforms them in terms of convergence speed and accuracy. We demonstrate its effectiveness on a large set of indoor real-world maps, and a very large simulated dataset. Open-source implementations in C++, and the datasets, are publicly available.

ICRA Conference 2010 Conference Paper

Improved Monte Carlo localization of autonomous robots through simultaneous estimation of motion model parameters

• Jörg Müller 0004
• Christoph Gonsior
• Wolfram Burgard

In recent years, there has been an increasing interest in autonomous navigation for lightweight flying robots. With regard to self-localization flying robots have several limitations compared to ground vehicles. Due to their limited payload flying vehicles possess only limited computational resources and are restricted to a few and lightweight sensors. Additionally the kinematics of flying robots is rather complex, which requires sophisticated motion models that are typically hard to calibrate. However, as the sensors provide only a limited amount of information, the motion models need to be highly accurate to reduce the potential increase of uncertainty caused by the movements of the vehicle. In this paper, we present a novel approach to simultaneous localization and estimation of motion model parameters and their adaptation in the context of a particle filter. To deal with sudden changes of parameters, our approach utilizes random sampling augmented by additional damping to avoid oscillations caused by the delayed detection of the changes. As we demonstrate in experiments with a real blimp, our method can deal with very sparse and imprecise sensor information and outperforms a standard Monte Carlo localization approach.

IROS Conference 2010 Conference Paper

Improved updating of Euclidean distance maps and Voronoi diagrams

• Boris Lau
• Christoph Sprunk
• Wolfram Burgard

This paper presents novel, highly efficient approaches for updating Euclidean distance maps and Voronoi diagrams represented on grid maps. Our methods employ a dynamic variant of the brushfire algorithm to update only those cells that are actually affected by changes in the environment. In experiments in different environments we show that our update strategies for distance maps and Voronoi diagrams require substantially fewer cell visits and significantly less computation time compared to previous approaches. Furthermore, the dynamic Voronoi diagram also improves on previous work by correctly dealing with non-convex obstacles such as building walls. We also present a dynamic variant of a skeletonization-based approach to Voronoi diagrams that is especially robust to noise. All of our algorithms consider actual Euclidean distances rather than grid steps. An open source implementation is available online.

IROS Conference 2010 Conference Paper

Learning the elasticity parameters of deformable objects with a manipulation robot

• Barbara Frank
• Ruediger Schmedding
• Cyrill Stachniss
• Matthias Teschner
• Wolfram Burgard

In this paper, we consider the problem of determining the elasticity properties of deformable objects with a mobile manipulator equipped with a force sensorb. We learn the parameters by establishing a relation between the applied forces and the corresponding surface deformations. To determine the parameters, we minimize the difference between the observed surface of an object that is deformed by a real manipulator and the deformed surface obtained with a deformation simulator based on finite element methods. To establish the correspondences between the surfaces, our approach applies a 3D registration technique based on point-clouds which is used as the basis for comparing the results of the simulation system with the observations of the real deformations. As we demonstrate in real-world experiments, our system is able to estimate appropriate parameters that can be used to predict future deformations. This information can directly be incorporated into motion planning approaches that are designed for robots operating with deformable objects.

ICRA Conference 2010 Conference Paper

Mapping indoor environments based on human activity

• Slawomir Grzonka
• Frederic Dijoux
• Andreas Karwath
• Wolfram Burgard

We present a novel approach to build approximate maps of structured environments utilizing human motion and activity. Our approach uses data recorded with a data suit which is equipped with several IMUs to detect movements of a person and door opening and closing events. In our approach we interpret the movements as motion constraints and door handling events as landmark detections in a graph-based SLAM framework. As we cannot distinguish between individual doors, we employ a multi-hypothesis approach on top of the SLAM system to deal with the high data-association uncertainty. As a result, our approach is able to accurately and robustly recover the trajectory of the person. We additionally take advantage of the fact that people traverse free space and that doors separate rooms to recover the geometric structure of the environment after the graph optimization. We evaluate our approach in several experiments carried out with different users and in environments of different types.

IROS Conference 2010 Conference Paper

Operating articulated objects based on experience

• Jürgen Sturm
• Advait Jain
• Cyrill Stachniss
• Charles C. Kemp
• Wolfram Burgard

Many tasks that would be of benefit to users in domestic environments require that robots manipulate articulated objects such as doors and drawers. In this paper, we present a novel approach that simultaneously estimates the kinematic model of an articulated object based on the trajectory described by the robot's end effector, and uses this model to predict the future trajectory of the end effector. One advantage of our approach is that the robot can directly use these predictions to generate an equilibrium point control path for operating the mechanism. Additionally, our approach can improve these predictions based on previously learned articulation models. We have implemented and tested our approach on a real mobile manipulator. Through 40 trials, we show that the robot can reliably open various household objects, including cabinet doors, sliding doors, office drawers, and a dishwasher. Furthermore, we demonstrate that using the information from previous interactions as a prior significantly improves the prediction accuracy.

IROS Conference 2010 Conference Paper

Probabilistic Rule Set Joint State Update as approximation to the full joint state estimation applied to multi object scene analysis

• Thilo Grundmann
• Michael Fiegert
• Wolfram Burgard

One essential capability of service robots lies in the identification and localization of objects in the vicinity of the robot. The extreme computational demands of this high-dimensional state estimation problem require approximations of the joint posterior even for small numbers of objects. A common approach to solve this problem is to marginalize the joint state space and to consider object-related state spaces which are estimated individually under the assumption of statistical independence. In practice, however, this independence assumption is often violated, especially when the objects are located close to each other, which leads to a reduced accuracy of this approximation, compared to the full joint estimation. To address this problem, we propose the new method denoted as Rule Set Joint State Update (RSJSU), which features a better approximation of the joint posterior in the presence of dependencies, and thus leads to better estimation results. We present experimental results in which we simultaneously estimate all six degrees of freedom of multiple objects.

IROS Conference 2010 Conference Paper

Recognizing people based on their footsteps using a wearable accelerometer

• Hannes Becker
• Wolfram Burgard

Collaboration of mobile robots and people generate the need for methods allowing the robot to reliable identify a person. The robust identification of the user is especially important in the context of people tracking when there are frequent occlusions. In this paper we present a novel approach for recognizing the user of a mobile robot. Our approach assumes that the user wears a mobile footstep sensor whose data are fused with footstep data extracted from leg movements of people. It relies on a recursive Bayesian estimation scheme to calculate a posterior about the potential associations between the different footstep perceptions. Our approach has been implemented and tested on real data. In simulated experiments, in which we use ground truth leg movement data recorded with a motion capture suite, and with a real robot we demonstrate the robustness of our method even when multiple people are present.

ICRA Conference 2010 Conference Paper

Robust place recognition for 3D range data based on point features

• Bastian Steder
• Giorgio Grisetti
• Wolfram Burgard

The problem of place recognition appears in different mobile robot navigation problems including localization, SLAM, or change detection in dynamic environments. Whereas this problem has been studied intensively in the context of robot vision, relatively few approaches are available for three-dimensional range data. In this paper, we present a novel and robust method for place recognition based on range images. Our algorithm matches a given 3D scan against a database using point features and scores potential transformations by comparing significant points in the scans. A further advantage of our approach is that the features allow for a computation of the relative transformations between scans which is relevant for registration processes. Our approach has been implemented and tested on different 3D data sets obtained outdoors. In several experiments we demonstrate the advantages of our approach also in comparison to existing techniques.

ICRA Conference 2010 Conference Paper

Searching for objects: Combining multiple cues to object locations using a maximum entropy model

• Dominik Joho
• Wolfram Burgard

In this paper, we consider the problem of how background knowledge about usual object arrangements can be utilized by a mobile robot to more efficiently find an object in an unknown environment. We decompose the action selection problem during the search into two parts. First, we compute a belief over the location of the object and subsequently use the belief to select the next target location the robot should visit. For the inference part, we utilize a maximum entropy model which models the conditional distribution over possible locations of the target object given the observations made so far. The model is based on co-occurrences of objects and object attributes in different spatial contexts. The parameters are learned by maximizing the data likelihood using gradient ascent. We evaluate our approach by simulated search runs based on data obtained from different real-world environments. Our results show a significant improvement over a standard search technique which does not employ domain-specific background knowledge.

IROS Conference 2010 Conference Paper

Temporary maps for robust localization in semi-static environments

• Daniel Meyer-Delius
• Jürgen Hess 0001
• Giorgio Grisetti
• Wolfram Burgard

Accurate and robust localization is essential for the successful navigation of autonomous mobile robots. The majority of existing localization approaches, however, is based on the assumption that the environment is static which does not hold for most practical application domains. In this paper, we present a localization framework that can robustly track a robot's pose even in non-static environments. Our approach keeps track of the observations caused by unexpected objects in the environment using temporary local maps. It relies both on these temporary local maps and on a reference map of the environment for estimating the pose of the robot. Experimental results demonstrate that by exploiting the observations caused by unexpected objects our approach outperforms standard localization methods for static environments.

IROS Conference 2010 Conference Paper

Unsupervised learning of compact 3D models based on the detection of recurrent structures

• Michael Ruhnke
• Bastian Steder
• Giorgio Grisetti
• Wolfram Burgard

In this paper we describe a novel algorithm for constructing a compact representation of 3D laser range data. Our approach extracts an alphabet of local scans from the scene. The words of this alphabet are used to replace recurrent local 3D structures, which leads to a substantial compression of the entire point cloud. We optimize our model in terms of complexity and accuracy by minimizing the Bayesian information criterion (BIC). Experimental evaluations on large real-world data show that our method allows robots to accurately reconstruct environments with as few as 70 words.

ICRA Conference 2010 Conference Paper

Vision-based detection for learning articulation models of cabinet doors and drawers in household environments

• Jürgen Sturm
• Kurt Konolige
• Cyrill Stachniss
• Wolfram Burgard

Service robots deployed in domestic environments generally need the capability to deal with articulated objects such as doors and drawers in order to fulfill certain mobile manipulation tasks. This however, requires, that the robots are able to perceive the articulation models of such objects. In this paper, we present an approach for detecting, tracking, and learning articulation models for cabinet doors and drawers without using artificial markers. Our approach uses a highly efficient and sampling-based approach to rectangle detection in depth images obtained from a self-developed active stereo system. The robot can use the generative models learned for the articulated objects to estimate their articulation type, their current configuration, and to make predictions about possible configurations not observed before. We present experiments carried out on real data obtained from our active stereo system. The results demonstrate that our technique is able to learn accurate articulation models. We furthermore provide a detailed error analysis based on ground truth data obtained in a motion capturing studio.

IJCAI Conference 2009 Conference Paper

• Jürgen Sturm
• Vijay Pradeep
• Cyrill Stachniss
• Christian Plagemann
• Kurt Konolige
• Wolfram Burgard

Robots operating in home environments must be able to interact with articulated objects such as doors or drawers. Ideally, robots are able to autonomously infer articulation models by observation. In this paper, we present an approach to learn kinematic models by inferring the connectivity of rigid parts and the articulation models for the corresponding links. Our method uses a mixture of parameterized and parameter-free (Gaussian process) representations and finds low-dimensional manifolds that provide the best explanation of the given observations. Our approach has been implemented and evaluated using real data obtained in various realistic home environment settings.

IROS Conference 2009 Conference Paper

A comparison of SLAM algorithms based on a graph of relations

• Wolfram Burgard
• Cyrill Stachniss
• Giorgio Grisetti
• Bastian Steder
• Rainer Kümmerle
• Christian Dornhege
• Michael Ruhnke
• Alexander Kleiner

In this paper, we address the problem of creating an objective benchmark for comparing SLAM approaches. We propose a framework for analyzing the results of SLAM approaches based on a metric for measuring the error of the corrected trajectory. The metric uses only relative relations between poses and does not rely on a global reference frame. The idea is related to graph-based SLAM approaches in the sense that it considers the energy needed to deform the trajectory estimated by a SLAM approach to the ground truth trajectory. Our method enables us to compare SLAM approaches that use different estimation techniques or different sensor modalities since all computations are made based on the corrected trajectory of the robot. We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the SLAM community. The relations have been obtained by manually matching laser-range observations. We believe that our benchmarking framework allows the user an easy analysis and objective comparisons between different SLAM approaches.

ICRA Conference 2009 Conference Paper

A probabilistic sonar sensor model for robust localization of a small-size blimp in indoor environments using a particle filter

• Jörg Müller 0004
• Axel Rottmann
• Leonhard M. Reindl
• Wolfram Burgard

In recent years, autonomous miniature airships have gained increased interest in the robotics community. This is due to their ability to move safely and hover for extended periods of time. The major constraints of miniature airships come from their limited payload which introduces substantial constraints on their perceptional capabilities. In this paper, we consider the problem of localizing a miniature blimp with lightweight ultrasound sensors. Since the opening angle of the sound cone emitted by a sonar sensor depends on the diameter of the membrane, small-size sonar devices introduce the problem of high uncertainty about which object has been perceived. We present a novel sensor model for ultrasound sensors with large opening angles that allows an autonomous blimp to robustly localize itself in a known environment using Monte Carlo localization. As we demonstrate in experiments with a real blimp, our novel sensor model outperforms a popular sensor model that has in the past been shown to work reliably on wheeled platforms.

ICRA Conference 2009 Conference Paper

A visual odometry framework robust to motion blur

• Alberto Pretto
• Emanuele Menegatti
• Maren Bennewitz
• Wolfram Burgard
• Enrico Pagello

Motion blur is a severe problem in images grabbed by legged robots and, in particular, by small humanoid robots. Standard feature extraction and tracking approaches typically fail when applied to sequences of images strongly affected by motion blur. In this paper, we propose a new feature detection and tracking scheme that is robust even to non-uniform motion blur. Furthermore, we developed a framework for visual odometry based on features extracted out of and matched in monocular image sequences. To reliably extract and track the features, we estimate the point spread function (PSF) of the motion blur individually for image patches obtained via a clustering technique and only consider highly distinctive features during matching. We present experiments performed on standard datasets corrupted with motion blur and on images taken by a camera mounted on walking small humanoid robots to show the effectiveness of our approach. The experiments demonstrate that our technique is able to reliably extract and match features and that it is furthermore able to generate a correct visual odometry, even in presence of strong motion blur effects and without the aid of any inertial measurement sensor.

ICRA Conference 2009 Conference Paper

Adaptive autonomous control using online value iteration with gaussian processes

• Axel Rottmann
• Wolfram Burgard

In this paper, we present a novel approach to controlling a robotic system online from scratch based on the reinforcement learning principle. In contrast to other approaches, our method learns the system dynamics and the value function separately, which permits to identify the individual characteristics and is, therefore, easily adaptable to changing conditions. The major problem in the context of learning control policies lies in high-dimensional state and action spaces, that needs to be explored in order to identify the optimal policy. In this paper, we propose an approach that learns the system dynamics and the value function in an alternating fashion based on Gaussian process models. Additionally, to reduce computation time and to make the system applicable to online learning, we present an efficient sparsification method. In experiments carried out with a real miniature blimp we demonstrate that our approach can learn height control online. Further results obtained with an inverted pendulum show that our method requires less data to achieve the same performance as an off-line learning approach.

ICRA Conference 2009 Conference Paper

Autonomous driving in a multi-level parking structure

• Rainer Kümmerle
• Dirk Hähnel
• Dmitri Dolgov
• Sebastian Thrun
• Wolfram Burgard

Recently, the problem of autonomous navigation of automobiles has gained substantial interest in the robotics community. Especially during the two recent DARPA grand challenges, autonomous cars have been shown to robustly navigate over extended periods of time through complex desert courses or through dynamic urban traffic environments. In these tasks, the robots typically relied on GPS traces to follow pre-defined trajectories so that only local planners were required. In this paper, we present an approach for autonomous navigation of cars in indoor structures such as parking garages. Our approach utilizes multi-level surface maps of the corresponding environments to calculate the path of the vehicle and to localize it based on laser data in the absence of sufficiently accurate GPS information. It furthermore utilizes a local path planner for controlling the vehicle. In a practical experiment carried out with an autonomous car in a real parking garage we demonstrate that our approach allows the car to autonomously park itself in a large-scale multi-level structure.

ICRA Conference 2009 Conference Paper

Imitation learning with generalized task descriptions

• Clemens Eppner
• Jürgen Sturm
• Maren Bennewitz
• Cyrill Stachniss
• Wolfram Burgard

In this paper, we present an approach that allows a robot to observe, generalize, and reproduce tasks observed from multiple demonstrations. Motion capture data is recorded in which a human instructor manipulates a set of objects. In our approach, we learn relations between body parts of the demonstrator and objects in the scene. These relations result in a generalized task description. The problem of learning and reproducing human actions is formulated using a dynamic Bayesian network (DBN). The posteriors corresponding to the nodes of the DBN are estimated by observing objects in the scene and body parts of the demonstrator. To reproduce a task, we seek for the maximum-likelihood action sequence according to the DBN. We additionally show how further constraints can be incorporated online, for example, to robustly deal with unforeseen obstacles. Experiments carried out with a real 6-DoF robotic manipulator as well as in simulation show that our approach enables a robot to reproduce a task carried out by a human demonstrator. Our approach yields a high degree of generalization illustrated by performing a pick-and-place and a whiteboard cleaning task.

IROS Conference 2009 Conference Paper

Improving robot navigation in structured outdoor environments by identifying vegetation from laser data

• Kai M. Wurm
• Rainer Kümmerle
• Cyrill Stachniss
• Wolfram Burgard

This paper addresses the problem of vegetation detection from laser measurements. The ability to detect vegetation is important for robots operating outdoors, since it enables a robot to navigate more efficiently and safely in such environments. In this paper, we propose a novel approach for detecting low, grass-like vegetation using laser remission values. In our algorithm, the laser remission is modeled as a function of distance, incidence angle, and material. We classify surface terrain based on 3D scans of the surroundings of the robot. The model is learned in a self-supervised way using vibration-based terrain classification. In all real world experiments we carried out, our approach yields a classification accuracy of over 99%. We furthermore illustrate how the learned classifier can improve the autonomous navigation capabilities of mobile robots.

IROS Conference 2009 Conference Paper

Kinodynamic motion planning for mobile robots using splines

• Boris Lau
• Christoph Sprunk
• Wolfram Burgard

This paper presents an approach to time-optimal kinodynamic motion planning for a mobile robot. A global path planner is used to generate collision-free straight-line paths from the robot's position to a given goal location. With waypoints of this path, an initial trajectory is generated which defines the planned position of the robot over time. A velocity profile is computed that accounts for constraints on the velocity and acceleration of the robot. The trajectory is refined to minimize the time needed for traversal by an any-time optimization algorithm. An error-feedback controller generates motor commands to execute the planned trajectory. Quintic Be¿zier splines are used to allow for curvature-continuous joins of trajectory segments, which enables the system to replan trajectories in order to react to unmapped obstacles. Experiments on real robots are presented that show our system's capabilities of smooth, precise, and predictive motion.

IROS Conference 2009 Conference Paper

Learning efficient policies for vision-based navigation

• Armin Hornung
• Hauke Strasdat
• Maren Bennewitz
• Wolfram Burgard

Cameras are popular sensors for robot navigation tasks such as localization as they are inexpensive, lightweight, and provide rich data. However, fast movements of a mobile robot typically reduce the performance of vision-based localization systems due to motion blur. In this paper, we present a reinforcement learning approach to choose appropriate velocity profiles for vision-based navigation. The learned policy minimizes the time to reach the destination and implicitly takes the impact of motion blur on observations into account. To reduce the size of the resulting policies, which is desirable in the context of memory-constrained systems, we compress the learned policy via a clustering approach. Extensive simulated and real-world experiments demonstrate that our learned policy significantly outperforms any policy that uses a constant velocity. We furthermore show, that our policy is applicable to different environments. Additional experiments demonstrate that our compressed policies do not result in a performance loss compared to the originally learned policy.

ICRA Conference 2009 Conference Paper

Modeling RFID signal strength and tag detection for localization and mapping

• Dominik Joho
• Christian Plagemann
• Wolfram Burgard

In recent years, there has been an increasing interest within the robotics community in investigating whether Radio Frequency Identification (RFID) technology can be utilized to solve localization and mapping problems in the context of mobile robots. We present a novel sensor model which can be utilized for localizing RFID tags and for tracking a mobile agent moving through an RFID-equipped environment. The proposed probabilistic sensor model characterizes the received signal strength indication (RSSI) information as well as the tag detection events to achieve a higher modeling accuracy compared to state-of-the-art models which deal with one of these aspects only. We furthermore propose a method that is able to bootstrap such a sensor model in a fully unsupervised fashion. Real-world experiments demonstrate the effectiveness of our approach also in comparison to existing techniques.

IROS Conference 2009 Conference Paper

Object identification with tactile sensors using bag-of-features

• Alexander Schneider
• Jürgen Sturm
• Cyrill Stachniss
• Marco Reisert
• Hans Burkhardt
• Wolfram Burgard

In this paper, we present a novel approach for identifying objects using touch sensors installed in the finger tips of a manipulation robot. Our approach operates on low-resolution intensity images that are obtained when the robot grasps an object. We apply a bag-of-words approach for object identification. By means of unsupervised clustering on training data, our approach learns a vocabulary from tactile observations which is used to generate a histogram codebook. The histogram codebook models distributions over the vocabulary and is the core identification mechanism. As the objects are larger than the sensor, the robot typically needs multiple grasp actions at different positions to uniquely identify an object. To reduce the number of required grasp actions, we apply a decision-theoretic framework that minimizes the entropy of the probabilistic belief about the type of the object. In our experiments carried out with various industrial and household objects, we demonstrate that our approach is able to discriminate between a large set of objects. We furthermore show that using our approach, a robot is able to distinguish visually similar objects that have different elasticity properties by using only the information from the touch sensor.

ICRA Conference 2009 Conference Paper

Probabilistic situation recognition for vehicular traffic scenarios

• Daniel Meyer-Delius
• Christian Plagemann
• Wolfram Burgard

To act intelligently in dynamic environments, a system must understand the current situation it is involved in at any given time. This requires dealing with temporal context, handling multiple and ambiguous interpretations, and accounting for various sources of uncertainty. In this paper we propose a probabilistic approach to modeling and recognizing situations. We define a situation as a distribution over sequences of states that have some meaningful interpretation. Each situation is characterized by an individual hidden Markov model that describes the corresponding distribution. In particular, we consider typical traffic scenarios and describe how our framework can be used to model and track different situations while they are evolving. The approach was evaluated experimentally in vehicular traffic scenarios using real and simulated data. The results show that our system is able to recognize and track multiple situation instances in parallel and make sensible decisions between competing hypotheses. Additionally, we show that our models can be used for predicting the position of the tracked vehicles.

ICRA Conference 2009 Conference Paper

Real-world robot navigation amongst deformable obstacles

• Barbara Frank
• Cyrill Stachniss
• Ruediger Schmedding
• Matthias Teschner
• Wolfram Burgard

In this paper, we consider the problem of mobile robots navigating in environments with non-rigid objects. Whereas robots can plan their paths more effectively when they utilize the information about the deformability of objects, they also need to consider the influence of the interaction with the deformable objects on their measurements during the execution of their navigation task. In this paper, we present a probabilistic approach to identify the measurements influenced by the deformable objects. Based on a learned statistics about the influence of the deformable objects on the measurements, the robot is able to perform a sensor-based collision avoidance of unforeseen objects. We present experiments carried out with a real robot that illustrate the practicability of our approach.

IROS Conference 2009 Conference Paper

Regression-based online situation recognition for vehicular traffic scenarios

• Daniel Meyer-Delius
• Jürgen Sturm
• Wolfram Burgard

In this paper, we present an approach for learning generalized models for traffic situations. We formulate the problem using a dynamic Bayesian network (DBN) from which we learn the characteristic dynamics of a situation from labeled trajectories using kernel regression. For a new and unlabeled trajectory, we can then infer the corresponding situation by evaluating the data likelihood for the individual situation models. In experiments carried out on laser range data gathered on a car in real traffic and in simulation, we show that we can robustly recognize different traffic situations even from trajectories corresponding to partial situation instances.

IROS Conference 2009 Conference Paper

Robust on-line model-based object detection from range images

• Bastian Steder
• Giorgio Grisetti
• Mark Van Loock
• Wolfram Burgard

A mobile robot that accomplishes high level tasks needs to be able to classify the objects in the environment and to determine their location. In this paper, we address the problem of online object detection in 3D laser range data. The object classes are represented by 3D point-clouds that can be obtained from a set of range scans. Our method relies on the extraction of point features from range images that are computed from the point-clouds. Compared to techniques that directly operate on a full 3D representation of the environment, our approach requires less computation time while retaining the robustness of full 3D matching. Experiments demonstrate that the proposed approach is even able to deal with partially occluded scenes and to fulfill the runtime requirements of online applications.

ICRA Conference 2009 Conference Paper

Towards a navigation system for autonomous indoor flying

• Slawomir Grzonka
• Giorgio Grisetti
• Wolfram Burgard

Recently there has been increasing research on the development of autonomous flying vehicles. Whereas most of the proposed approaches are suitable for outdoor operation, only a few techniques have been designed for indoor environments. In this paper we present a general system consisting of sensors and algorithms which enables a small sized flying vehicle to operate indoors. This is done by adapting techniques which have been successfully applied on ground robots. We released our system as open-source with the intention to provide the community with a new framework for building applications for indoor flying robots. We present a set of experiments to validate our system on an open source quadrotor.

ICRA Conference 2009 Conference Paper

Tracking groups of people with a multi-model hypothesis tracker

• Boris Lau
• Kai O. Arras
• Wolfram Burgard

People in densely populated environments typically form groups that split and merge. In this paper we track groups of people so as to reflect this formation process and gain efficiency in situations where maintaining the state of individual people would be intractable. We pose the group tracking problem as a recursive multi-hypothesis model selection problem in which we hypothesize over both, the partitioning of tracks into groups (models) and the association of observations to tracks (assignments). Model hypotheses that include split, merge, and continuation events are first generated in a data-driven manner and then validated by means of the assignment probabilities conditioned on the respective model. Observations are found by clustering points from a laser range finder given a background model and associated to existing group tracks using the minimum average Hausdorff distance. Experiments with a stationary and a moving platform show that, in populated environments, tracking groups is clearly more efficient than tracking people separately. Our system runs in real-time on a typical desktop computer.

ICRA Conference 2009 Conference Paper

Unsupervised learning of 3D object models from partial views

• Michael Ruhnke
• Bastian Steder
• Giorgio Grisetti
• Wolfram Burgard

We present an algorithm for learning 3D object models from partial object observations. The input to our algorithm is a sequence of 3D laser range scans. Models learned from the objects are represented as point clouds. Our approach can deal with partial views and it can robustly learn accurate models from complex scenes. It is based on an iterative matching procedure which attempts to recursively merge similar models. The alignment between models is determined using a novel scan registration procedure based on range images. The decision about which models to merge is performed by spectral clustering of a similarity matrix whose entries represent the consistency between different models.

ICRA Conference 2009 Conference Paper

Utilizing reflection properties of surfaces to improve mobile robot localization

• Maren Bennewitz
• Cyrill Stachniss
• Sven Behnke
• Wolfram Burgard

A main difficulty that arises in the context of probabilistic localization is the design of an appropriate observation model, i. e. , determining the likelihood of a sensor measurement given the pose of the robot and a map of the environment. Many successful approaches to localization rely on data provided by range sensors, e. g. , laser range scanners. When using such data one normally has to deal with erroneous maximum-range readings that occur due to poor-reflecting surfaces. In general, these readings cannot be distinguished from readings obtained when no obstacle is within the measurement range of the sensor. Therefore, existing localization techniques treat these readings alike in the observation model. In this paper, we present a novel approach that explicitly considers the reflection properties of surfaces and thus the expectation of valid range measurements. In addition to the expected range measurement, we compute the probability of reflectance for a beam given the relative pose of the robot to the obstacle taking into account the angle of incidence of the beam. We estimate the reflection properties of surfaces using data collected with a mobile robot equipped with a laser range scanner. As we demonstrate in experiments carried out with a real robot, our technique leads to significantly improved localization results compared to a state-of-the-art observation model.

ICRA Conference 2009 Conference Paper

Which landmark is useful? Learning selection policies for navigation in unknown environments

• Hauke Strasdat
• Cyrill Stachniss
• Wolfram Burgard

In general, a mobile robot that operates in unknown environments has to maintain a map and has to determine its own location given the map. This introduces significant computational and memory constraints for most autonomous systems, especially for lightweight robots such as humanoids or flying vehicles. In this paper, we present a novel approach for learning a landmark selection policy that allows a robot to discard landmarks that are not valuable for its current navigation task. This enables the robot to reduce the computational burden and to carry out its task more efficiently by maintaining only the important landmarks. Our approach applies an unscented Kalman filter for addressing the simultaneous localization and mapping problems and uses Monte-Carlo reinforcement learning to obtain the selection policy. Based on real world and simulation experiments, we show that the learned policies allow for efficient robot navigation and outperform handcrafted strategies. We furthermore demonstrate that the learned policies are not only usable in a specific scenario but can also be generalized towards environments with varying properties.

IROS Conference 2008 Conference Paper

Coordinated multi-robot exploration using a segmentation of the environment

• Kai M. Wurm
• Cyrill Stachniss
• Wolfram Burgard

This paper addresses the problem of exploring an unknown environment with a team of mobile robots. The key issue in coordinated multi-robot exploration is how to assign target locations to the individual robots such that the overall mission time is minimized. In this paper, we propose a novel approach to distribute the robots over the environment that takes into account the structure of the environment. To achieve this, it partitions the space into segments, for example, corresponding to individual rooms. Instead of only considering frontiers between unknown and explored areas as target locations, we send the robots to the individual segments with the task to explore the corresponding area. Our approach has been implemented and tested in simulation as well as in real world experiments. The experiments demonstrate that the overall exploration time can be significantly reduced by considering our segmentation method.

ICRA Conference 2008 Conference Paper

Efficient path planning for mobile robots in environments with deformable objects

• Barbara Frank
• Markus Becker 0003
• Cyrill Stachniss
• Wolfram Burgard
• Matthias Teschner

The ability to reliably navigate through the environment is an important prerequisite for truly autonomous robots. In this paper, we consider the problem of path planning in environments with non-rigid obstacles such as curtains or plants. We present an approach that combines probabilistic roadmaps with a physical simulation of object deformations to determine a path that optimizes the trade-off between the deformation cost and the distance to be traveled. We describe how our approach utilizes Finite Element theory for calculating the deformation cost. Since the high computational requirements of the corresponding simulation prevent this method from being applicable online, we present an approximation that uses a preprocessing step to determine a deformation cost function for each object. This cost function allows us to estimate the deformation costs of arbitrary paths through the objects and is used to evaluate the trajectories generated by the roadmap planner online. We present experiments which demonstrate that the resulting algorithm plans nearly identical trajectories compared to the method that relies on computationally intense simulations. At the same time, our approach allows the robot to quickly calculate paths in environments with deformable objects.

ICRA Conference 2008 Conference Paper

Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probabilities

• Kai O. Arras
• Slawomir Grzonka
• Matthias Luber
• Wolfram Burgard

We present an approach to laser-based people tracking using a multi-hypothesis tracker that detects and tracks legs separately with Kalman filters, constant velocity motion models, and a multi-hypothesis data association strategy. People are defined as high-level tracks consisting of two legs that are found with little model knowledge. We extend the data association so that it explicitly handles track occlusions in addition to detections and deletions. Additionally, we adapt the corresponding probabilities in a situation-dependent fashion so as to reflect the fact that legs frequently occlude each other. Experimental results carried out with a mobile robot illustrate that our approach can robustly and efficiently track multiple people even in situations of high levels of occlusion.

IROS Conference 2008 Conference Paper

Efficiently learning high-dimensional observation models for Monte-Carlo localization using Gaussian mixtures

• Patrick Pfaff
• Cyrill Stachniss
• Christian Plagemann
• Wolfram Burgard

Whereas probabilistic approaches are a powerful tool for mobile robot localization, they heavily rely on the proper definition of the so-called observation model which defines the likelihood of an observation given the position and orientation of the robot and the map of the environment. Most of the sensor models for range sensors proposed in the past either consider the individual beam measurements independently or apply uni-modal models to represent the likelihood function. In this paper, we present an approach that learns place-dependent sensor models for entire range scans using Gaussian mixture models. To deal with the high dimensionality of the measurement space, we utilize principle component analysis for dimensionality reduction. In practical experiments carried out with data obtained from a real robot, we demonstrate that our model substantially outperforms existing and popular sensor models.

IROS Conference 2008 Conference Paper

Estimating landmark locations from geo-referenced photographs

• Henrik Kretzschmar
• Cyrill Stachniss
• Christian Plagemann
• Wolfram Burgard

The problem of estimating the positions of landmarks using a mobile robot equipped with a camera has intensively been studied in the past. In this paper, we consider a variant of this problem in which the robot should estimate the locations of observed landmarks based on a sparse set of geo-referenced images for which no heading information is available. Sources for such kind of data are image portals such as Flickr or Google Image Search. We formulate the problem of estimating the landmark locations as an optimization problem and show that it is possible to accurately localize the landmarks in real world settings.

ICRA Conference 2008 Conference Paper

Gaussian mixture models for probabilistic localization

• Patrick Pfaff
• Christian Plagemann
• Wolfram Burgard

One of the key tasks during the realization of probabilistic approaches to localization is the design of a proper sensor model, that calculates the likelihood of a measurement given the current pose of the vehicle and the map of the environment. In the past, range sensors have become popular for mobile robot localization since they directly measure distance. However, in situations in which the robot operates close to edges of obstacles or in highly cluttered environments, small changes in the pose of the robot can lead to large variations in the acquired range scans. If the sensor model used does not appropriately characterize the resulting fluctuations, the performance of probabilistic approaches may substantially degrade. A common solution is to artificially smooth the likelihood function or to only integrate a small fraction of the measurements. In this paper we present a more fundamental and robust approach which uses mixtures of Gaussians to model the likelihood function for single range measurements. In practical experiments we compare our approach to previous methods and demonstrate that it yields a substantially increase in robustness.

ICRA Conference 2008 Conference Paper

How to learn accurate grid maps with a humanoid

• Cyrill Stachniss
• Maren Bennewitz
• Giorgio Grisetti
• Sven Behnke
• Wolfram Burgard

Humanoids have recently become a popular research platform in the robotics community. Such robots offer various fields for new applications. However, they have several drawbacks compared to wheeled vehicles such as stability problems, limited payload capabilities, violation of the flat world assumption, and they typically provide only very rough odometry information, if at all. In this paper, we investigate the problem of learning accurate grid maps with humanoid robots. We present techniques to deal with some of the above-mentioned difficulties. We describe how an existing approach to the simultaneous localization and mapping (SLAM) problem can be adapted to robustly learn accurate maps with a humanoid equipped with a laser range finder. We present an experiment in which our mapping system builds a highly accurate map with a size of around 20 m by 20 m using data acquired with a humanoid in our office environment containing two loops. The resulting maps have a similar accuracy as maps built with a wheeled robot.

IROS Conference 2008 Conference Paper

Learning predictive terrain models for legged robot locomotion

• Christian Plagemann
• Sebastian Mischke
• Sam Prentice
• Kristian Kersting
• Nicholas Roy
• Wolfram Burgard

Legged robots require accurate models of their environment in order to plan and execute paths. We present a probabilistic technique based on Gaussian processes that allows terrain models to be learned and updated efficiently using sparse approximation techniques. The major benefit of our terrain model is its ability to predict elevations at unseen locations more reliably than alternative approaches, while it also yields estimates of the uncertainty in the prediction. In particular, our nonstationary Gaussian process model adapts its covariance to the situation at hand, allowing more accurate inference of terrain height at points that have not been observed directly. We show how a conventional motion planner can use the learned terrain model to plan a path to a goal location, using a terrain-specific cost model to accept or reject candidate footholds. In experiments with a real quadruped robot equipped with a laser range finder, we demonstrate the usefulness of our approach and discuss its benefits compared to simpler terrain models such as elevations grids.

ICRA Conference 2008 Conference Paper

Monocular range sensing: A non-parametric learning approach

• Christian Plagemann
• Felix Endres
• Jürgen Hess 0001
• Cyrill Stachniss
• Wolfram Burgard

Mobile robots rely on the ability to sense the geometry of their local environment in order to avoid obstacles or to explore the surroundings. For this task, dedicated proximity sensors such as laser range finders or sonars are typically employed. Cameras are a cheap and lightweight alternative to such sensors, but do not directly offer proximity information. In this paper, we present a novel approach to learning the relationship between range measurements and visual features extracted from a single monocular camera image. As the learning engine, we apply Gaussian processes, a non-parametric learning technique that not only yields the most likely range prediction corresponding to a certain visual input but also the predictive uncertainty. This information, in turn, can be utilized in an extended grid-based mapping scheme to more accurately update the map. In practical experiments carried out in different environments with a mobile robot equipped with an omnidirectional camera system, we demonstrate that our system is able to produce proximity estimates with an accuracy comparable to that of dedicated sensors such as sonars or infrared range finders.

ICRA Conference 2008 Conference Paper

Online constraint network optimization for efficient maximum likelihood map learning

• Giorgio Grisetti
• Dario Lodi Rizzini
• Cyrill Stachniss
• Edwin Olson
• Wolfram Burgard

In this paper, we address the problem of incrementally optimizing constraint networks for maximum likelihood map learning. Our approach allows a robot to efficiently compute configurations of the network with small errors while the robot moves through the environment. We apply a variant of stochastic gradient descent and use a tree-based parameterization of the nodes in the network. By integrating adaptive learning rates in the parameterization of the network, our algorithm can use previously computed solutions to determine the result of the next optimization run. Additionally, our approach updates only the parts of the network which are affected by the newly incorporated measurements and starts the optimization approach only if the new data reveals inconsistencies with the network constructed so far. These improvements yield an efficient solution for this class of online optimization problems. Our approach has been implemented and tested on simulated and on real data. We present comparisons to recently proposed online and offline methods that address the problem of optimizing constraint network. Experiments illustrate that our approach converges faster to a network configuration with small errors than the previous approaches.

ICRA Conference 2008 Conference Paper

Unsupervised body scheme learning through self-perception

• Jürgen Sturm
• Christian Plagemann
• Wolfram Burgard

In this paper, we present an approach allowing a robot to learn a generative model of its own physical body from scratch using self-perception with a single monocular camera. Our approach yields a compact Bayesian network for the robot’s kinematic structure including the forward and inverse models relating action signals and body pose. We propose to simultaneously learn local action models for all pairs of perceivable body parts from data generated through random “motor babbling. ” From this repertoire of local models, we construct a Bayesian network for the full system using the pose prediction accuracy on a separate cross validation data set as the criterion for model selection. The resulting model can be used to predict the body pose when no perception is available and allows for gradient-based posture control. In experiments with real and simulated manipulator arms, we show that our system is able to quickly learn compact and accurate models and to robustly deal with noisy observations.

IJCAI Conference 2007 Conference Paper

• Rudolph Triebel
• Richard Schmidt
• oacute; scar Mart
• iacute; nez Mozos
• Wolfram Burgard

In this paper, we present an algorithm to identify different types of objects from 2D and 3D laser range data. Our method is a combination of an instance-based feature extraction similar to the Nearest-Neighbor classifier (NN) and a collective classification method that utilizes associative Markov networks (AMNs). Compared to previous approaches, we transform the feature vectors so that they are better separable by linear hyper-planes, which are learned by the AMN classifier. We present results of extensive experiments in which we evaluate the performance of our algorithm on several recorded indoor scenes and compare it to the standard AMN approach as well as the NN classifier. The classification rate obtained with our algorithm substantially exceeds those of the AMN and the NN.

IJCAI Conference 2007 Conference Paper

• Christian Plagemann
• Dieter Fox
• Wolfram Burgard

The ability to detect failures and to analyze their causes is one of the preconditions of truly autonomous mobile robots. Especially online failure detection is a complex task, since the effects of failures are typically difficult to model and often resemble the noisy system behavior in a fault-free operational mode. The extremely low a priori likelihood of failures poses additional challenges for detection algorithms. In this paper, we present an approach that applies Gaussian process classification and regression techniques for learning highly effective proposal distributions of a particle filter that is applied to track the state of the system. As a result, the efficiency and robustness of the state estimation process is substantially improved. In practical experiments carried out with a real robot we demonstrate that our system is capable of detecting collisions with unseen obstacles while at the same time estimating the changing point of contact with the obstacle.

IROS Conference 2007 Conference Paper

Analyzing gaussian proposal distributions for mapping with rao-blackwellized particle filters

• Cyrill Stachniss
• Giorgio Grisetti
• Wolfram Burgard
• Nicholas Roy

Particle filters are a frequently used filtering technique in the robotics community. They have been successfully applied to problems such as localization, mapping, or tracking. The particle filter framework allows the designer to freely choose the proposal distribution which is used to obtain the next generation of particles in estimating dynamical processes. This choice greatly influences the performance of the filter. Many approaches have achieved good performance through informed proposals which explicitly take into account the current observation. A popular approach is to approximate the desired proposal distribution by a Gaussian. This paper presents a statistical analysis of the quality of such Gaussian approximations. We also propose a way to obtain the optimal proposal in a non-parametric way and then identify the error introduced by the Gaussian approximation. Furthermore, we present an alternative sampling strategy that better deals with situations in which the target distribution is multi-modal. Experimental results indicate that our alternative sampling strategy leads to accurate maps more frequently that the Gaussian approach while requiring only minimal additional computational overhead.

IROS Conference 2007 Conference Paper

Approximate covariance estimation in graphical approaches to SLAM

• Gian Diego Tipaldi
• Giorgio Grisetti
• Wolfram Burgard

Smoothing and optimization approaches are an effective means for solving the simultaneous localization and mapping (SLAM) problem. Most of the existing techniques focus mainly on determining the most likely map and leave open how to efficiently compute the marginal covariances. These marginal covariances, however, are essential for solving the data association problem. In this paper we present a novel algorithm for computing an approximation of the marginal. In experiments we demonstrate that our approach outperforms two commonly used techniques, namely loopy belief propagation and belief propagation on a spanning tree. Compared to these approaches, our algorithm yields better estimates while preserving the same time complexity.

IROS Conference 2007 Conference Paper

Autonomous blimp control using model-free reinforcement learning in a continuous state and action space

• Axel Rottmann
• Christian Plagemann
• Peter Hilgers
• Wolfram Burgard

In this paper, we present an approach that applies the reinforcement learning principle to the problem of learning height control policies for aerial blimps. In contrast to previous approaches, our method does not require sophisticated hand- tuned models, but rather learns the policy online, which makes the system easily adaptable to changing conditions. The blimp we apply our approach to is a small-scale vehicle equipped with an ultrasound sensor that measures its elevation relative to the ground. The major problem in the context of learning control policies lies in the high-dimensional state-action space that needs to be explored in order to identify the values of all state-action pairs. In this paper, we propose a solution to learning continuous control policies based on the Gaussian process model. In practical experiments carried out on a real robot we demonstrate that the system is able to learn a policy online within a few minutes only.

IROS Conference 2007 Conference Paper

Efficient estimation of accurate maximum likelihood maps in 3D

• Giorgio Grisetti
• Slawomir Grzonka
• Cyrill Stachniss
• Patrick Pfaff
• Wolfram Burgard

Learning maps is one of the fundamental tasks of mobile robots. In the past, numerous efficient approaches to map learning have been proposed. Most of them, however, assume that the robot lives on a plane. In this paper, we consider the problem of learning maps with mobile robots that operate in non-flat environments and apply maximum likelihood techniques to solve the graph-based SLAM problem. Due to the non-commutativity of the rotational angles in 3D, major problems arise when applying approaches designed for the two-dimensional world. The non-commutativity introduces serious difficulties when distributing a rotational error over a sequence of poses. In this paper, we present an efficient solution to the SLAM problem that is able to distribute a rotational error over a sequence of nodes. Our approach applies a variant of gradient descent to solve the error minimization problem. We implemented our technique and tested it on large simulated and real world datasets. We furthermore compared our approach to solving the problem by LU-decomposition. As the experiments illustrate, our technique converges significantly faster to an accurate map with low error and is able to correct maps with bigger noise than existing methods.

IROS Conference 2007 Conference Paper

Improved likelihood models for probabilistic localization based on range scans

• Patrick Pfaff
• Christian Plagemann
• Wolfram Burgard

Range sensors are popular for localization since they directly measure the geometry of the local environment. Another distinct benefit is their typically high accuracy and spatial resolution. It is a well-known problem, however, that the high precision of these sensors leads to practical problems in probabilistic localization approaches such as Monte Carlo localization (MCL), because the likelihood function becomes extremely peaked if no means of regularization are applied. In practice, one therefore artificially smoothes the likelihood function or only integrates a small fraction of the measurements. In this paper we present a more fundamental and robust approach, that provides a smooth likelihood model for entire range scans. Additionally, it is location-dependent. In practical experiments we compare our approach to previous methods and demonstrate that it leads to a more robust localization.

IROS Conference 2007 Conference Paper

Learning maps in 3D using attitude and noisy vision sensors

• Bastian Steder
• Giorgio Grisetti
• Slawomir Grzonka
• Cyrill Stachniss
• Axel Rottmann
• Wolfram Burgard

In this paper, we address the problem of learning 3D maps of the environment using a cheap sensor setup which consists of two standard web cams and a low cost inertial measurement unit. This setup is designed for lightweight or flying robots. Our technique uses visual features extracted from the web cams and estimates the 3D location of the landmarks via stereo vision. Feature correspondences are estimated using a variant of the PROSAC algorithm. Our mapping technique constructs a graph of spatial constraints and applies an efficient gradient descent-based optimization approach to estimate the most likely map of the environment. Our approach has been evaluated in comparably large outdoor and indoor environments. We furthermore present experiments in which our technique is applied to build a map with a blimp.

ICML Conference 2007 Conference Paper

Most likely heteroscedastic Gaussian process regression

• Kristian Kersting
• Christian Plagemann
• Patrick Pfaff
• Wolfram Burgard

This paper presents a novel Gaussian process (GP) approach to regression with input-dependent noise rates. We follow Goldberg et al.'s approach and model the noise variance using a second GP in addition to the GP governing the noise-free output value. In contrast to Goldberg et al., however, we do not use a Markov chain Monte Carlo method to approximate the posterior noise variance but a most likely noise approach. The resulting model is easy to implement and can directly be used in combination with various existing extensions of the standard GPs such as sparse approximations. Extensive experiments on both synthetic and real-world data, including a challenging perception problem in robotics, show the effectiveness of most likely heteroscedastic GP regression.

ICRA Conference 2007 Conference Paper

Towards Mapping of Cities

• Patrick Pfaff
• Rudolph Triebel
• Cyrill Stachniss
• Pierre Lamon
• Wolfram Burgard
• Roland Siegwart

Map learning is a fundamental task in mobile robotics because maps are required for a series of high level applications. In this paper, we address the problem of building maps of large-scale areas like villages or small cities. We present our modified car-like robot which we use to acquire the data about the environment. We introduce our localization system which is based on an information filter and is able to merge the information obtained by different sensors. We furthermore describe out mapping technique that is able to compactly model three-dimensional scenes and allows us efficient and accurate incremental map learning. We additionally apply a global optimization techniques in order to accurately close loops in the environment. Our approach has been implemented and deeply tested on a real car equipped with a series of sensors. Experiments described in this paper illustrate the accuracy and efficiency of the presented techniques.

ICRA Conference 2007 Conference Paper

Using Boosted Features for the Detection of People in 2D Range Data

• Kai O. Arras
• Óscar Martínez Mozos
• Wolfram Burgard

This paper addresses the problem of detecting people in two dimensional range scans. Previous approaches have mostly used pre-defined features for the detection and tracking of people. We propose an approach that utilizes a supervised learning technique to create a classifier that facilitates the detection of people. In particular, our approach applies AdaBoost to train a strong classifier from simple features of groups of neighboring beams corresponding to legs in range data. Experimental results carried out with laser range data illustrate the robustness of our approach even in cluttered office environments

IROS Conference 2006 Conference Paper

Improving Data Association in Vision-based SLAM

• Arturo Gil
• Óscar Reinoso
• Óscar Martínez Mozos
• Cyrill Stachniss
• Wolfram Burgard

This paper presents an approach to vision-based simultaneous localization and mapping (SLAM). Our approach uses the scale invariant feature transform (SIFT) as features and applies a rejection technique to concentrate on a reduced set of distinguishable, stable features. We track detected SIFT features over consecutive frames obtained by a stereo camera and select only those features that appear to be stable from different views. Whenever a feature is selected, we compute a representative feature given the previous observations. This approach is applied within a Rao-Blackwellized particle filter to make the data association easier and furthermore to reduce the number of landmarks that need to be maintained in the map. Our system has been implemented and tested on data gathered with a mobile robot in a typical office environment. Experiments presented in this paper demonstrate that our method improves the data association and in this way leads to more accurate maps

IROS Conference 2006 Conference Paper

Learning Relational Navigation Policies

• Alexandru Cocora
• Kristian Kersting
• Christian Plagemann
• Wolfram Burgard
• Luc De Raedt

Navigation is one of the fundamental tasks for a mobile robot. The majority of path planning approaches has been designed to entirely solve the given problem from scratch given the current and goal configurations of the robot. Although these approaches yield highly efficient plans, the computed policies typically do not transfer to other, similar tasks. We propose to learn relational decision trees as abstract navigation strategies from example paths. Relational abstraction has several interesting and important properties. First, it allows a mobile robot to generalize navigation plans from specific examples provided by users or exploration. Second, the navigation policy learned in one environment can be transferred to unknown environments. In several experiments with real robots in a real environment and in simulated runs, we demonstrate the usefulness of our approach

IROS Conference 2006 Conference Paper

Multi-Level Surface Maps for Outdoor Terrain Mapping and Loop Closing

• Rudolph Triebel
• Patrick Pfaff
• Wolfram Burgard

To operate outdoors or on non-flat surfaces, mobile robots need appropriate data structures that provide a compact representation of the environment and at the same time support important tasks such as path planning and localization. One such representation that has been frequently used in the past are elevation maps which store in each cell of a discrete grid the height of the surface in the corresponding area. Whereas elevation maps provide a compact representation, they lack the ability to represent vertical structures or even multiple levels. In this paper, we propose a new representation denoted as multi-level surface maps (MLS maps). Our approach allows to store multiple surfaces in each cell of the grid. This enables a mobile robot to model environments with structures like bridges, underpasses, buildings or mines. Additionally, they allow to represent vertical structures. Throughout this paper we present algorithms for updating these maps based on sensory input, to match maps calculated from two different scans, and to solve the loop-closing problem given such maps. Experiments carried out with a real robot in an outdoor environment demonstrate that our approach is well-suited for representing large-scale outdoor environments

IROS Conference 2006 Conference Paper

Multiple Hypothesis Tracking of Clusters of People

• Manuel Mucientes
• Wolfram Burgard

Mobile robots operating in populated environments typically can improve their service and navigation behavior when they know where people are in their vicinity and in which direction they are heading. In this paper we present an algorithm for tracking clusters of people using multiple hypothesis tracking (MHT). The motivation for our approach is that tracking clusters of objects instead of the individual objects enhances the reliability and robustness of the tracking especially when the objects move in groups. To efficiently keep track of multiple objects and clusters, our approach uses MHT in combination with Murty's algorithm. The set of hypothesis for each iteration is constructed in two consecutive steps: one for solving the data association problem, taking also into account the frequent occlusions between the objects, and the second one for considering the joining of different clusters. Our approach has been implemented and tested on a real robot and in a typical hallway environment. Experimental results demonstrate that our approach can robustly deal with several groups of people and is able to reliably manage the splits and joins of clusters

ICRA Conference 2006 Conference Paper

Robust 3D Scan Point Classification using Associative Markov Networks

• Rudolph Triebel
• Kristian Kersting
• Wolfram Burgard

In this paper we present an efficient technique to learn associative Markov networks (AMNs) for the segmentation of 3D scan data. Our technique is an extension of the work recently presented by Anguelov et al. (2005), in which AMNs are applied and the learning is done using max-margin optimization. In this paper we show that by adaptively reducing the training data, the training process can be performed much more efficiently while still achieving good classification results. The reduction is obtained by utilizing kd-trees and pruning them appropriately. Our algorithm does not require any additional parameters and yields an abstraction of the training data. In experiments with real data collected from a mobile outdoor robot we demonstrate that our approach yields accurate segmentations

ICRA Conference 2006 Conference Paper

Speeding-up Multi-robot Exploration by Considering Semantic Place Information

• Cyrill Stachniss
• Óscar Martínez Mozos
• Wolfram Burgard

In this paper, we consider the problem of exploring an unknown environment with a team of mobile robots. One of the key issues in multi-robot exploration is how to assign target locations to the individual robots. To better distribute the robots over the environment and to avoid redundant work, we take into account the type of place a potential target is located in (e. g. , a corridor or a room). To determine the type of a place, we apply a classifier learned with AdaBoost which additionally considers spatial dependencies between nearby locations. Our approach to incorporate the type of places in the coordination of the robots has been implemented and tested in different environments. The experiments demonstrate that our system effectively distributes the robots over the environment and allows them to accomplish their mission faster compared to approaches that ignore the semantic place labels

ICRA Conference 2006 Conference Paper

Speeding-up Rao-blackwellized SLAM

• Giorgio Grisetti
• Gian Diego Tipaldi
• Cyrill Stachniss
• Wolfram Burgard
• Daniele Nardi

Recently, Rao-Blackwellized particle filters have become a popular tool to solve the simultaneous localization and mapping problem. This technique applies a particle filter in which each particle carries an individual map of the environment. Accordingly, a key issue is to reduce the number of particles and/or to make use of compact map representations. This paper presents an approximative but highly efficient approach to mapping with Rao-Blackwellized particle filters. Moreover, it provides a compact map model. A key advantage is that the individual particles can share large parts of the model of the environment. Furthermore, they are able to re-use an already computed proposal distribution. Both techniques substantially speed up the overall process and reduce the memory requirements. Experimental results obtained with mobile robots in large-scale indoor environments and based on published, standard datasets illustrate the advantages of our methods over previous Rao-Blackwellized mapping approaches

IROS Conference 2006 Conference Paper

Supervised Learning of Topological Maps using Semantic Information Extracted from Range Data

• Óscar Martínez Mozos
• Wolfram Burgard

This paper presents an approach to create topological maps from geometric maps obtained with a mobile robot in an indoor-environment using range data. Our approach utilizes AdaBoost, a supervised learning algorithm, to classify each point of the geometric map into semantic classes. We then apply a segmentation procedure based on probabilistic relaxation labeling on the resulting classifications to eliminate errors. The topological graph is then extracted from the individual different regions and their connections. In this way, we obtain a topological map in the form of a graph, in which each node indicates a region in the environment with its corresponding semantic class (e. g. , corridor, or room) and the edges indicate the connections between them. Experimental results obtained with data from different real-world environments demonstrate the effectiveness of our approach

ICRA Conference 2005 Conference Paper

Autonomous Terrain Mapping and Classification Using Hidden Markov Models

• Denis Fernando Wolf
• Gaurav S. Sukhatme
• Dieter Fox
• Wolfram Burgard

This paper presents a new approach for terrain mapping and classification using mobile robots with 2D laser range finders. Our algorithm generates 3D terrain maps and classifies navigable and non-navigable regions on those maps using Hidden Markov models. The maps generated by our approach can be used for path planning, navigation, local obstacle avoidance, detection of changes in the terrain, and object recognition. We propose a map segmentation algorithm based on Markov Random Fields, which removes small errors in the classification. In order to validate our algorithms, we present experimental results using two robotic platforms.

ICRA Conference 2005 Conference Paper

Improving Grid-based SLAM with Rao-Blackwellized Particle Filters by Adaptive Proposals and Selective Resampling

• Giorgio Grisetti
• Cyrill Stachniss
• Wolfram Burgard

Recently Rao-Blackwellized particle filters have been introduced as effective means to solve the simultaneous localization and mapping (SLAM) problem. This approach uses a particle filter in which each particle carries an individual map of the environment. Accordingly, a key question is how to reduce the number of particles. In this paper we present adaptive techniques to reduce the number of particles in a Rao-Blackwellized particle filter for learning grid maps. We propose an approach to compute an accurate proposal distribution taking into account not only the movement of the robot but also the most recent observation. This drastically decrease the uncertainty about the robot's pose in the prediction step of the filter. Furthermore, we present an approach to selectively carry out re-sampling operations which seriously reduces the problem of particle depletion. Experimental results carried out with mobile robots in large-scale indoor as well as in outdoor environments illustrate the advantages of our methods over previous approaches.

IROS Conference 2005 Conference Paper

Improving plane extraction from 3D data by fusing laser data and vision

• Henrik Andreasson
• Rudolph Triebel
• Wolfram Burgard

The problem of extracting three-dimensional structures from data acquired with mobile robots has received considerable attention over the past years. Robots that are able to perceive their three-dimensional environment are envisioned to more robustly perform tasks like navigation, rescue, and manipulation. In this paper we present an approach that simultaneously uses color and range information to cluster 3D points into planar structures. Our current system also is able to calibrate the camera and the laser based on the remission values provided by the range scanner and the brightness of the pixels in the image. It has been implemented on a mobile robot equipped with a manipulator that carries a range scanner and a camera for acquiring colored range scans. Several experiments carried out on real data and in simulations demonstrate that our approach yields highly accurate results also in comparison with previous approaches.

ICRA Conference 2005 Conference Paper

Recovering Particle Diversity in a Rao-Blackwellized Particle Filter for SLAM After Actively Closing Loops

• Cyrill Stachniss
• Giorgio Grisetti
• Wolfram Burgard

Acquiring models of the environment belongs to the fundamental tasks of mobile robots. Approaches addressing the problem of simultaneous localization and mapping (SLAM) typically process the perceived sensor data and do not influence the motion of the mobile robot. In this paper, we present an approach to actively closing loops during exploration. It applies a Rao-Blackwellized particle filter to maintain multiple hypotheses about potential trajectories of the robot and corresponding maps. To prevent the particle filter from becoming overly confident, we present a technique to recover the particle diversity after successfully closing a loop. This way the particle depletion problem is avoided. The combination of our approach with the active loop closing strategy allows to deal with multiple nested loops. Experimental results presented in this paper illustrate the advantage of our method over pervious approaches to mapping with Rao-Blackwellized particle filters.

ICRA Conference 2005 Conference Paper

Supervised Learning of Places from Range Data using AdaBoost

• Óscar Martínez Mozos
• Cyrill Stachniss
• Wolfram Burgard

This paper addresses the problem of classifying places in the environment of a mobile robot into semantic categories. We believe that semantic information about the type of place improves the capabilities of a mobile robot in various domains including localization, path-planning, or human-robot interaction. Our approach uses AdaBoost, a supervised learning algorithm, to train a set of classifiers for place recognition based on laser range data. In this paper we describe how this approach can be applied to distinguish between rooms, corridors, doorways, and hallways. Experimental results obtained in simulation and with real robots demonstrate the effectiveness of our approach in various environments.

ICRA Conference 2005 Conference Paper

Using Hierarchical EM to Extract Planes from 3D Range Scans

• Rudolph Triebel
• Wolfram Burgard
• Frank Dellaert

Recently, the acquisition of three-dimensional maps has become more and more popular. This is motivated by the fact that robots act in the three-dimensional world and several tasks such as path planning or localizing objects can be carried out more reliable using three-dimensional representations. In this paper we consider the problem of extracting planes from three-dimensional range data. In contrast to previous approaches our algorithm uses a hierarchical variant of the popular Expectation Maximization (EM) algorithm [1] to simultaneously learn the main directions of the planar structures. These main directions are then used to correct the position and orientation of planes. In practical experiments carried out with real data and in simulations we demonstrate that our algorithm can accurately extract planes and their orientation from range data.

IROS Conference 2004 Conference Paper

Exploration with active loop-closing for FastSLAM

• Cyrill Stachniss
• Dirk Hähnel
• Wolfram Burgard

Acquiring models of the environment belongs to the fundamental tasks of mobile robots. In the last few years several researchers have focused on the problem of simultaneous localization and mapping (SLAM). Classic SLAM approaches are passive in the sense that they only process the perceived sensor data and do not influence the motion of the mobile robot. In this paper we present a novel and integrated approach that combines autonomous exploration with simultaneous localization and mapping. Our method uses a grid-based version of the FastSLAM algorithm and at each point in time considers actions to actively close loops during exploration. By re-entering already visited areas the robot reduces its localization error and this way learns more accurate maps. Experimental results presented in this paper illustrate the advantage of our method over pervious approaches lacking the ability to actively close loops.

ICRA Conference 2004 Conference Paper

Mapping and Localization with RFID Technology

• Dirk Hähnel
• Wolfram Burgard
• Dieter Fox
• Kenneth P. Fishkin
• Matthai Philipose

We analyze whether radio frequency identification (RFID) technology can be used to improve the localization of mobile robots and persons in their environment. In particular we study the problem of localizing RFID tags with a mobile platform that is equipped with a pair of RFID antennas. We present a probabilistic measurement model for RFID readers that allow us to accurately localize RFID tags in the environment. We also demonstrate how such maps can be used to localize a robot and persons in their environment. Finally, we present experiments illustrating that the computational requirements for global robot localization can be reduced strongly by fusing RFID information with laser data.

IROS Conference 2004 Conference Paper

Relative localization for pairs of robots based on unidentifiable moving features

• Luis Montesano
• Luis Montano
• Wolfram Burgard

This paper presents a new method for relative localization of a pair of robots based on the trajectories described by unidentifiable moving objects. Our approach uses a Rao-Blackwellized particle filter to estimate both the relative location of the robots and the data associations between the moving objects around the robots. We describe our implementation on real robots and present experiments illustrating the robustness of our algorithm.

ICRA Conference 2003 Conference Paper

A system for volumetric robotic mapping of abandoned mines

• Sebastian Thrun
• Dirk Hähnel
• Dave Ferguson 0001
• Michael Montemerlo
• Rudolph Triebel
• Wolfram Burgard
• Christopher R. Baker
• Zachary Omohundro

This paper describes two robotic systems developed for acquiring accurate volumetric maps of underground mines. One system is based on a cart instrumented by laser range finders, pushed through a mine by people. Another is a remotely controlled mobile robot equipped with laser range finders. To build consistent maps of large mines with many cycles, we describe an algorithm for estimating global correspondences and aligning robot paths. This algorithm enables us to recover consistent maps several hundreds of meters in diameter, without odometric information. We report results obtained in two mines, a research mine in Bruceton, PA, and an abandoned coal mine in Burgettstown, PA.

ICRA Conference 2003 Conference Paper

Adapting navigation strategies using motions patterns of people

• Maren Bennewitz
• Wolfram Burgard
• Sebastian Thrun

As people move through their environments, they do not move randomly. Instead, they are often engaged in typical motion patterns, related to specific locations they might be interested in approaching. In this paper we propose a method for adapting the behavior of a mobile robot according to the activities of the people in its surrounding. Our approach uses learned models of people's motion behaviors. Whenever the robot detects a person it computes a probabilistic estimate about which motion pattern the person might be engaged in. During the path planning it then uses this belief to improve its navigation behavior. In different practical experiments carried out on a real robot we demonstrate that our approach allows a robot to quickly adapt its navigation plans according to the activities of the persons in its surrounding. We also present experiments illustrating that our approach provides a better behavior than a standard reactive collision avoidance system.

NeurIPS Conference 2003 Conference Paper

An Autonomous Robotic System for Mapping Abandoned Mines

• David Ferguson
• Aaron Morris
• Dirk Hähnel
• Christopher Baker
• Zachary Omohundro
• Carlos Reverte
• Scott Thayer
• Charles Whittaker

We present the software architecture of a robotic system for mapping abandoned mines. The software is capable of acquiring consistent 2D maps of large mines with many cycles, represented as Markov random £elds. 3D C-space maps are acquired from local 3D range scans, which are used to identify navigable paths using A* search. Our system has been deployed in three abandoned mines, two of which inaccessible to people, where it has acquired maps of unprecedented detail and accuracy.

IROS Conference 2003 Conference Paper

An efficient fastSLAM algorithm for generating maps of large-scale cyclic environments from raw laser range measurements

• Dirk Hähnel
• Wolfram Burgard
• Dieter Fox
• Sebastian Thrun

The ability to learn a consistent model of its environment is a prerequisite for autonomous mobile robots. A particularly challenging problem in acquiring environment maps is that of closing loops; loops in the environment create challenging data association problems [J. -S. Gutman et al. , 1999]. This paper presents a novel algorithm that combines Rao-Blackwellized particle filtering and scan matching. In our approach scan matching is used for minimizing odometric errors during mapping. A probabilistic model of the residual errors of scan matching process is then used for the resampling steps. This way the number of samples required is seriously reduced. Simultaneously we reduce the particle depletion problem that typically prevents the robot from closing large loops. We present extensive experiments that illustrate the superior performance of our approach compared to previous approaches.

IJCAI Conference 2003 Conference Paper

An Extension of the ICP Algorithm for Modeling Nonrigid Objects with Mobile Robots

• Dirk H
• hnel
• Sebastian Thrun
• Wolfram Burgard

The iterative closest point (ICP) algorithm [2] is a popular method for modeling 3D objects from range data. The classical ICP algorithm rests on a rigid surface assumption. Building on recent work on nonrigid object models [5; 16; 9], this paper presents an ICP algorithm capable of modeling nonrigid objects, where individual scans may be subject to local deformations. We describe an integrated mathematical framework for simultaneously registering scans and recovering the surface configuration. To tackle the resulting high-dimensional optimization problems, we introduce a hierarchical method that first matches a coarse skeleton of scan points, then adapts local scan patches. The approach is implemented for a mobile robot capable of acquiring 3D models of objects.

IJCAI Conference 2003 Conference Paper

Exploring Unknown Environments with Mobile Robots using Coverage Maps

• Cyrill Stachniss
• Wolfram Burgard

In this paper we introduce coverage maps as a new way of representing the environment of a mobile robot. Coverage maps store for each cell of a given grid a posterior about the amount the corresponding cell is covered by an obstacle. Using this representation a mobile robot can more accurately reason about its uncertainty in the map of the environment than with standard occupancy grids. We present a model for proximity sensors designed to update coverage maps upon sensory input. We also describe how coverage maps can be used to formulate a decision-theoretic approach for mobile robot exploration. We present experiments carried out with real robots in which accurate maps are build from noisy ultrasound data. Finally, we present a comparison of different view-point selection strategies for mobile robot exploration.

ICRA Conference 2003 Conference Paper

Map building with mobile robots in dynamic environments

• Dirk Hähnel
• Rudolph Triebel
• Wolfram Burgard
• Sebastian Thrun

The problem of generating maps with mobile robots has received considerable attention over the past years. Most of the techniques developed so far have been designed for situations in which the environment is static during the mapping process. Dynamic objects, however, can lead to serious errors in the resulting maps such as spurious objects or misalignments due to localization errors. In this paper we consider the problem of creating maps with mobile robots in dynamic environments. We present a new approach that interleaves mapping and localization with a probabilistic technique to identify spurious measurements. In several experiments we demonstrate that our algorithm generates accurate 2D and 3D in different kinds of dynamic indoor and outdoor environments. We also use our algorithm to isolate the dynamic objects and generate 3D representation of them.

IROS Conference 2003 Conference Paper

Mapping and exploration with mobile robots using coverage maps

• Cyrill Stachniss
• Wolfram Burgard

Exploration and mapping belongs to the fundamental tasks of mobile robots. In the past, many approaches have used occupancy grid maps to represent the environment during the map building process. Occupancy grids, however, are based on the assumption that each cell is either occupied or free. In this paper we introduce coverage maps as an alternative way of representing the environment of a robot. Coverage maps store for each cell of a given grid a posterior about the amount the corresponding cell is covered by an obstacle. We also present a model that allows us to update coverage maps upon input obtained from proximity sensors. We furthermore describe how to use coverage maps for a decision theoretic approach to exploration. Finally we present experimental results illustrating that coverage maps can be used to efficiently learn highly accurate models even if noisy sensors such as ultrasounds are used.

IJCAI Conference 2003 Conference Paper

Where is. ? Learning and Utilizing Motion Patterns of Persons with Mobile Robots

• Grzegorz Cielniak
• Maren Bennewitz
• Wolfram Burgard

Whenever people move through their environments they do not move randomly. Instead, they usually follow specific trajectories or motion patterns corresponding to their intentions. Knowledge about such patterns may enable a mobile robot to robustly keep track of persons in its environment or to improve its obstacle avoidance behavior. This paper proposes a technique for learning collections of trajectories that characterize typical motion patterns of persons. Data recorded with laser-range finders is clustered using the expectation maximization algorithm. Based on the result of the clustering process we derive a Hidden Markov Model (HMM). This HMM is able to estimate the current and future positions of multiple persons given knowledge about their intentions. Experimental results obtained with a mobile robot using laser and vision data collected in a typical office building with several persons illustrate the reliability and robustness of the approach. We also demonstrate that our model provides better estimates than an HMM directly learned from the data.

IROS Conference 2002 Conference Paper

An integrated approach to goal-directed obstacle avoidance under dynamic constraints for dynamic environments

• Cyrill Stachniss
• Wolfram Burgard

Whenever robots are installed in populated environments, they need appropriate techniques to avoid collisions with unexpected obstacles. Over the past years several reactive techniques have been developed that use heuristic evaluation functions to choose appropriate actions whenever a robot encounters an unforeseen obstacle. Whereas the majority of these approaches determines only the next steering command, some additionally consider sequences of possible poses. However, they generally do not consider sequences of actions in the velocity space. Accordingly, these methods are not able to slow down the robot early enough before it has to enter a narrow passage. In this paper we present a new approach that integrates path planning with sensor-based collision avoidance. Our algorithm simultaneously considers the robot's pose and velocities during the planning process. We employ different strategies to deal with the huge state space that has to be explored. Our method has been implemented and tested on real robots and in simulation runs. Extensive experiments demonstrate that our technique can reliably control mobile robots moving at high speeds.

ICRA Conference 2002 Conference Paper

Learning Motion Patterns of Persons for Mobile Service Robots

• Maren Bennewitz
• Wolfram Burgard
• Sebastian Thrun

We propose a method for learning models of people's motion behaviors in an indoor environment. As people move through their environments, they do not move randomly. Instead, they often engage in typical motion patterns, related to specific locations that they might be interested in approaching and specific trajectories that they might follow in doing so. Knowledge about such patterns may enable a mobile robot to develop improved people following and obstacle avoidance skills. This paper proposes an algorithm that learns collections of typical trajectories that characterize a person's motion patterns. Data, recorded by mobile robots equipped with laser range finders, is clustered into different types of motion using the popular expectation maximization algorithm, while simultaneously learning multiple motion patterns. Experimental results, obtained using data collected in a domestic residence and in an office building, illustrate that highly predictive models of human motion patterns can be learned.

IROS Conference 2002 Conference Paper

Map building with mobile robots in populated environments

• Dirk Hähnel
• Dirk Schulz 0001
• Wolfram Burgard

The problem of generating maps with mobile robots has received considerable attention over the past years. However, most of the approaches assume that the environment is static during the data-acquisition phase. In this paper we consider the problem of creating maps with mobile robots in populated environments. Our approach uses a probabilistic method to track multiple people and to incorporate the results of the tracking technique into the mapping process. The resulting maps are more accurate since corrupted readings are treated accordingly during the matching phase and since the number of spurious objects in the resulting maps is reduced. Our approach has been implemented and tested on real robot systems in indoor and outdoor scenarios. We present several experiments illustrating the capabilities of our approach to generate accurate 2D and 3D maps.

ICRA Conference 2002 Conference Paper

Robust Vision-Based Localization for Mobile Robots using an Image Retrieval System Based on Invariant Features

• Jürgen Wolf 0001
• Wolfram Burgard
• Hans Burkhardt

We present a vision-based approach to mobile robot localization, that integrates an image retrieval system with Monte-Carlo localization. The image retrieval process is based on features that are invariant with respect to image translations, rotations, and limited scale. Using the local features the system is robust against distortion and occlusions, which is especially important in populated environments. By using the sample-based Monte-Carlo localization technique our robot is able to globally localize itself to reliably keep tracking of its position, and to recover from localization failures. Both techniques are combined by extracting for each image a set of possible view-points using a two-dimensional map of the environment. Our technique was implemented and tested extensively. We present several experiments demonstrating the reliability and robustness of our approach even in the context of dynamics in the environment and larger errors in the odometry.

IROS Conference 2002 Conference Paper

Using EM to learn motion behaviors of persons with mobile robots

• Maren Bennewitz
• Wolfram Burgard
• Sebastian Thrun

We propose a method for learning models of people's motion behaviors in indoor environments. As people move through their environments, they do not move randomly. Instead, they often engage in typical motion patterns, related to specific locations that they might be interested in approaching and specific trajectories that they might follow in doing so. Knowledge about such patterns may enable a mobile robot to develop improved people following and obstacle avoidance skills. This paper proposes an algorithm that learns collections of typical trajectories that characterize a person's motion patterns. Data, recorded by mobile robots equipped with laser-range finders, is clustered into different types of motion using the popular expectation maximization algorithm, while simultaneously learning multiple motion patterns. Experimental results, obtained using data collected in a domestic residence and in an office building, illustrate that highly predictive models of human motion patterns can be learned.

IROS Conference 2001 Conference Paper

Exploiting constraints during prioritized path planning for teams of mobile robots

• Maren Bennewitz
• Wolfram Burgard
• Sebastian Thrun

Coordinating the motion of multiple mobile robots is one of the fundamental problems in robotics. The predominant algorithms for coordinating teams of robots are decoupled and prioritized, thereby avoiding combinatorially hard planning problems typically faced by centralized approaches. We present a method for finding solvable priority schemes for such prioritized and decoupled planning techniques. Existing approaches apply a single priority scheme which makes them overly prone to failure in cases where valid solutions exists. By searching in the space of priorization schemes, our approach overcomes this limitation. To focus the search, our algorithm is guided by constraints generated from the task specification. To illustrate the appropriateness of this approach, the paper discusses experimental results obtained with real robots and through systematic robot simulation. The experimental results demonstrate that our approach successfully solves many more coordination problems than previous decoupled and prioritized techniques.

ICRA Conference 2001 Conference Paper

Optimizing Schedules for Prioritized Path Planning of Multi-Robot Systems

• Maren Bennewitz
• Wolfram Burgard
• Sebastian Thrun

The coordination of robot motions is one of the fundamental problems for multi-robot systems. A popular approach to avoid planning in the high-dimensional composite configuration space is the prioritized and decoupled technique. In this paper we present a method for optimizing priority schemes for such prioritized and decoupled planning technique. Our approach performs a randomized search with hill-climbing to find solutions and to minimize the overall path lengths. The technique has been implemented and tested on real robots and in extensive simulation runs. The experimental results demonstrate that our method is able to greatly reduce the number of failures and to significantly reduce the overall path length for different prioritized and decoupled path planning techniques and even for large teams of robots.

AIJ Journal 2001 Journal Article

Robust Monte Carlo localization for mobile robots

• Sebastian Thrun
• Dieter Fox
• Wolfram Burgard
• Frank Dellaert

Mobile robot localization is the problem of determining a robot's pose from sensor data. This article presents a family of probabilistic localization algorithms known as Monte Carlo Localization (MCL). MCL algorithms represent a robot's belief by a set of weighted hypotheses (samples), which approximate the posterior under a common Bayesian formulation of the localization problem. Building on the basic MCL algorithm, this article develops a more robust algorithm called Mixture-MCL, which integrates two complimentary ways of generating samples in the estimation. To apply this algorithm to mobile robots equipped with range finders, a kernel density tree is learned that permits fast sampling. Systematic empirical results illustrate the robustness and computational efficiency of the approach.

ICRA Conference 2001 Conference Paper

Tracking Multiple Moving Targets with a Mobile Robot using Particle Filters and Statistical Data Association

• Dirk Schulz 0001
• Wolfram Burgard
• Dieter Fox
• Armin B. Cremers

One of the goals in the field of mobile robotics is the development of mobile platforms which operate in populated environments and offer various services to humans. For many tasks it is highly desirable that a robot can determine the positions of the humans in its surrounding. In this paper we present a method for tracking multiple moving objects with a mobile robot. We introduce a sample-based variant of joint probabilistic data association filters to track features originating from individual objects and to solve the correspondence problem between the detected features and the filters. In contrast to standard methods, occlusions are handled explicitly during data association. The technique has been implemented and tested on a real robot. Experiments carried out in a typical office environment show that the method is able to track multiple persons even when the trajectories of two people are crossing each other.

ICML Conference 2001 Conference Paper

Using EM to Learn 3D Models of Indoor Environments with Mobile Robots

• Yufeng Liu
• Rosemary Emery
• Deepayan Chakrabarti
• Wolfram Burgard
• Sebastian Thrun

ICRA Conference 2000 Conference Paper

A Real-Time Algorithm for Mobile Robot Mapping With Applications to Multi-Robot and 3D Mapping

• Sebastian Thrun
• Wolfram Burgard
• Dieter Fox

We present an incremental method for concurrent mapping and localization for mobile robots equipped with 2D laser range finders. The approach uses a fast implementation of scan-matching for mapping, paired with a sample-based probabilistic method for localization. Compact 3D maps are generated using a multi-resolution approach adopted from the computer graphics literature, fed by data from a dual laser system. Our approach builds 3D maps of large, cyclic environments in real-time, and it is robust. Experimental results illustrate that accurate maps of large, cyclic environments can be generated even in the absence of any odometric data.

ICRA Conference 2000 Conference Paper

Collaborative Multi-Robot Exploration

• Wolfram Burgard
• Mark Moors
• Dieter Fox
• Reid G. Simmons
• Sebastian Thrun

In this paper we consider the problem of exploring an unknown environment by a team of robots. As in single-robot exploration the goal is to minimize the overall exploration time. The key problem to be solved therefore is to choose appropriate target points for the individual robots so that they simultaneously explore different regions of their environment. We present a probabilistic approach for the coordination of multiple robots which, in contrast to previous approaches, simultaneously takes into account the costs of reaching a target point and the utility of target points. The utility of target points is given by the size of the unexplored area that a robot can cover with its sensors upon reaching a target position. Whenever a target point is assigned to a specific robot, the utility of the unexplored area visible from this target position is reduced for the other robots. This way, a team of multiple robots assigns different target points to the individual robots. The technique has been implemented and tested extensively in real-world experiments and simulation runs. The results given in this paper demonstrate that our coordination technique significantly reduces the exploration time compared to previous approaches.

ICRA Conference 1999 Conference Paper

Coastal Navigation: Mobile Robot Navigation with Uncertainty in Dynamic Environments

• Nicholas Roy
• Wolfram Burgard
• Dieter Fox
• Sebastian Thrun

Ships often use the coasts of continents for navigation in the absence of better tools such as GPS, since being close to land allows sailors to determine with high accuracy where they are. Similarly for mobile robots, in many environments global and accurate localization is not always feasible. Environments can lack features, and dynamic obstacles such as people can confuse and block sensors. We demonstrate a technique for generating trajectories that take into account both the information content of the environment, and the density of the people in the environment. These trajectories reduce the average positional certainty as the robot moves, reducing the likelihood the robot will become lost at any point. Our method was successfully implemented and used by the mobile robot Minerva, a museum tourguide robot, for a 2 week period in the Smithsonian National Museum of American History.

AIJ Journal 1999 Journal Article

Experiences with an interactive museum tour-guide robot

• Wolfram Burgard
• Armin B. Cremers
• Dieter Fox
• Dirk Hähnel
• Gerhard Lakemeyer
• Dirk Schulz
• Walter Steiner
• Sebastian Thrun

This article describes the software architecture of an autonomous, interactive tour-guide robot. It presents a modular and distributed software architecture, which integrates localization, mapping, collision avoidance, planning, and various modules concerned with user interaction and Web-based telepresence. At its heart, the software approach relies on probabilistic computation, on-line learning, and any-time algorithms. It enables robots to operate safely, reliably, and at high speeds in highly dynamic environments, and does not require any modifications of the environment to aid the robot's operation. Special emphasis is placed on the design of interactive capabilities that appeal to people's intuition. The interface provides new means for human-robot interaction with crowds of people in public places, and it also provides people all around the world with the ability to establish a “virtual telepresence” using the Web. To illustrate our approach, results are reported obtained in mid-1997, when our robot “RHINO” was deployed for a period of six days in a densely populated museum. The empirical results demonstrate reliable operation in public environments. The robot successfully raised the museum's attendance by more than 50%. In addition, thousands of people all over the world controlled the robot through the Web. We conjecture that these innovations transcend to a much larger range of application domains for service robots.

ICRA Conference 1999 Conference Paper

MINERVA: A Second-Generation Museum Tour-Guide Robot

• Sebastian Thrun
• Maren Bennewitz
• Wolfram Burgard
• Armin B. Cremers
• Frank Dellaert
• Dieter Fox
• Dirk Hähnel
• Charles R. Rosenberg

This paper describes an interactive tour-guide robot, which was successfully exhibited in a Smithsonian museum. During its two weeks of operation, the robot interacted with thousands of people, traversing more than 44 km at speeds of up to 163 cm/sec. Our approach specifically addresses issues such as safe navigation in unmodified and dynamic environments, and short-term human-robot interaction. It uses learning pervasively at all levels of the software architecture.

ICRA Conference 1999 Conference Paper

Monte Carlo Localization for Mobile Robots

• Frank Dellaert
• Dieter Fox
• Wolfram Burgard
• Sebastian Thrun

To navigate reliably in indoor environments, a mobile robot must know where it is. Thus, reliable position estimation is a key problem in mobile robotics. We believe that probabilistic approaches are among the most promising candidates to providing a comprehensive and real-time solution to the robot localization problem. However, current methods still face considerable hurdles. In particular the problems encountered are closely related to the type of representation used to represent probability densities over the robot's state space. Earlier work on Bayesian filtering with particle-based density representations opened up a new approach for mobile robot localization based on these principles. We introduce the Monte Carlo localization method, where we represent the probability density involved by maintaining a set of samples that are randomly drawn from it. By using a sampling-based representation we obtain a localization method that can represent arbitrary distributions. We show experimentally that the resulting method is able to efficiently localize a mobile robot without knowledge of its starting location. It is faster, more accurate and less memory-intensive than earlier grid-based methods, .

IROS Conference 1999 Conference Paper

Probabilistic mapping of unexpected objects by a mobile robot

• Frank Schönherr
• Joachim Hertzberg
• Wolfram Burgard

We present a technique for extending a given metric map of the environment by objects of a known type, where localization and perception of the robot is allowed to be uncertain. The advantage of our approach is that it allows the robot to estimate its own position in the given outline of the environment and thus to estimate the position of the objects not contained in the map. The method relies on partially observable Markov decision processes as well as on the Baum-Welch algorithm. It has been implemented and evaluated in several simulation experiments and also in a real-world sewage pipe network. The experimental results demonstrate that our approach can efficiently and accurately estimate the position of unexpected objects. Due to the probabilistic nature of the underlying techniques, our method can deal with noisy sensors as well as with large odometry errors which generally occur when deploying a robot in a sewerage pipe system.

IROS Conference 1999 Conference Paper

Robust visualization of navigation experiments with mobile robots over the Internet

• Dirk Schulz 0001
• Wolfram Burgard
• Armin B. Cremers

Visualization is an important precondition for successful teleoperation of instructable mobile robots. Data connections with varying and limited bandwidth such as the Internet, however prohibit the continuous transmission of video signals. In this paper we propose a predictive simulation technique which is designed to permit the reliable visualization of the robot's actions over the Internet. It differs from previous approaches in that it includes an odometry and sensor simulation. This simulation of the robot allows the integration of a complete robot control system to reliably predict complex actions of the robot even if large transmission gaps of several seconds occur. We describe an application of the predictive simulation technique to navigation experiments with mobile robots. We present different experiments carried out with a real robot illustrating that the predictive simulation technique provides accurate visualizations of the robot's actions even if transmission gaps of more than ten seconds occur.

ICML Conference 1999 Conference Paper

Sonar-Based Mapping of Large-Scale Mobile Robot Environments using EM

• Wolfram Burgard
• Dieter Fox
• Hauke Jans
• Christian Matenar
• Sebastian Thrun

ICRA Conference 1998 Conference Paper

A Hybrid Collision Avoidance Method for Mobile Robots

• Dieter Fox
• Wolfram Burgard
• Sebastian Thrun
• Armin B. Cremers

Proposes a hybrid approach to the problem of collision avoidance for indoor mobile robots. The /spl mu/DWA (model-based dynamic window approach) integrates sensor data from various sensors with information extracted from a map of the environment, to generate collision-free motion. A novel integration rule ensures that with high likelihood, the robot avoids collisions with obstacles not detectable with its sensors, even if it is uncertain about its position. The approach was implemented and tested extensively as part of an installation, in which a mobile robot gave interactive tours to visitors of the "Deutsches Museum Bonn. " Here our approach was essential for the success of the entire mission, because a large number of ill-shaped obstacles prohibited the use of purely sensor-based methods for collision avoidance.

IROS Conference 1998 Conference Paper

An experimental comparison of localization methods

• Jens-Steffen Gutmann
• Wolfram Burgard
• Dieter Fox
• Kurt Konolige

Localization is the process of updating the pose of a robot in an environment, based on sensor readings. In this experimental study, we compare two methods for localization of indoor mobile robots: Markov localization, which uses a probability distribution across a grid of robot poses; and scan matching, which uses Kalman filtering techniques based on matching sensor scans. Both these techniques are dense matching methods, that is, they match dense sets of environment features to an a priori map. To arrive at results for a range of situations, we utilize several different types of environments, and add noise to both the dead-reckoning and the sensors. Analysis shows that, roughly, the scan-matching techniques are more efficient and accurate, but Markov localization is better able to cope with large amounts of noise. These results suggest hybrid methods that are efficient, accurate and robust to noise.

IROS Conference 1998 Conference Paper

Integrating global position estimation and position tracking for mobile robots: the dynamic Markov localization approach

• Wolfram Burgard
• Andrcas Derr
• Dieter Fox
• Armin B. Cremers

Localization is one of the fundamental problems of mobile robots. In order to efficiently perform useful tasks such as office delivery, mobile robots must know their position in their environment. Existing approaches can be distinguished according to the type of localization problem they are designed to solve. Tracking techniques aim at monitoring the robot's position. They assume that the position is initially known and cannot recover from situations in which they lost track of the robot's position. Global localization techniques on the other hand, are able to estimate the robot's position under complete uncertainty. We present the dynamic Markov localization technique as a uniform approach to position estimation, which is able (1) to globally estimate the position of the robot, (2) to efficiently track its position whenever the robot's certainty is high, and (3) to detect and recover from localization failures. The approach has been implemented and intensively tested in real-world environments. We present several experiments illustrating the strength of our method.

ICRA Conference 1998 Conference Paper

Probabilistic Mapping of an Environment by a Mobile Robot

• Sebastian Thrun
• Dieter Fox
• Wolfram Burgard

This paper addresses the problem of building large-scale maps of indoor environments with mobile robots. It proposes a statistical approach that describes the map building problem as a constrained maximum-likelihood estimation problem, for which it devises a practical algorithm. Experimental results in large, cyclic environments illustrate the appropriateness of the approach.

AAAI Conference 1998 Conference Paper

The Interactive Museum Tour-Guide Robot

• Wolfram Burgard
• Dieter Fox
• Gerhard Lakemeyer
• Walter Steiner

This paper describes the software architecture of an autonomous tour-guide/tutor robot. This robot was recently deployed in the “Deutsches Museum Bonn, ” were it guided hundreds of visitors through the museum during a six-day deployment period. The robot’s control software integrates low-level probabilistic reasoning with high-level problem solving embedded in first order logic. A collection of software innovations, described in this paper, enabled the robot to navigate at high speeds through dense crowds, while reliably avoiding collisions with obstacles—some of which could not even be perceived. Also described in this paper is a user interface tailored towards non-expert users, which was essential for the robot’s success in the museum. Based on these experiences, this paper argues that time is ripe for the development of AI-based commercial service robots that assist people in everyday life.

IJCAI Conference 1997 Conference Paper

Active Mobile Robot Localization

• Wolfram Burgard
• Dieter Fox
• Sebastian Thrun

Localization is the problem of determining the position of a mobile robot from sensor data. Most existing localization approaches are passive, i. e. , they do not exploit the opportunity to control the robot's effectors during localization. This paper proposes an active localization approach. The approach provides rational criteria for (1) setting the robot's motion direction (exploration), and (2) determining the pointing direction of the sensors so as to most efficiently localize the robot. Furthermore, it is able to deal with noisy sensors and approximative world models. The appropriateness of our approach is demonstrated empirically using a mobile robot in a structured office environment.

IROS Conference 1996 Conference Paper

Controlling synchro-drive robots with the dynamic window approach to collision avoidance

• Dieter Fox
• Wolfram Burgard
• Sebastian Thrun

This paper proposes the dynamic window approach to reactive collision avoidance for mobile robots equipped with synchro-drives. The approach is derived directly from the motion dynamics of the robot and is therefore particularly well-suited for robots operating at high speed. It differs from previous approaches in that the search for commands controlling the translational and rotational velocity of the robot is carried out directly in the space of velocities. The advantage of our approach is that it correctly and in a rigorous way incorporates the dynamics of the robot. This is done by reducing the search space to the dynamic window, which consists of the velocities reachable within a short time interval. Within the dynamic window the approach only considers admissible velocities yielding a trajectory on which the robot is able to stop safely. Among these velocities the combination of translational and rotational velocity is chosen by maximizing an objective function. The objective function includes a measure of progress towards a goal location, the forward velocity of the robot, and the distance to the next obstacle on the trajectory. In extensive experiments the approach presented here has been found to safely control our mobile robot RHINO with speeds of up to 95 cm/sec, in populated and dynamic environments.

AAAI Conference 1996 Conference Paper

Estimating the Absolute Position of a Mobile Robot Using Position Probability Grids

• Wolfram Burgard
• Daniel Hennig

In order to re-use existing models of the environment mobile robots must be able to estimate their position and orientation in such models. Most of the existing methods for position estimation are based on special purpose sensors or aim at tracking the robot’ s position relative to the known starting point. This paper describes the position probability grid approach to estimating the robot’ s absolute position and orientation in a metric model of the environment. Our method is designed to work with standard sensors and is independent of any knowledge about the starting point. It is a Bayesian approach based on certainty grids. In each cell of such a grid we store the probability that this cell refers to the current position of the robot. These probabilities are obtained by integrating the likelihoods of sensor readings over time. Results described in this paper show that our technique is able to reliably estimate the position of a robot in complex environments. Our approach has proven to be robust with respect to inaccurate environmental models, noisy sensors, and ambiguous situations.

CSL Conference 1991 Conference Paper

Efficiency Considerations on Goal-Directed Chaining for Logic Programs

• Wolfram Burgard

Abstract This paper presents a linear resolution strategy (GDFC-resolution) for definite logic programs in which goal-directed forward chaining is used to find refutations. GDFC-resolution focuses on relevant facts and rules by means of query independent link clauses. One problem is, that the number of link clauses may be infinite if the program contains recursive rules defined over recursive data structures. We present an approach to generate a more general but finite set of link clauses. We show that GDFC-resolution is sound and complete for definite programs. We discuss the efficiency of GDFC-resolution comparing it with SLD-resolution and present some experimental results.