Rudolph Triebel

ICRA Conference 2025 Conference Paper

Single-Shot Metric Depth from Focused Plenoptic Cameras

• Blanca Lasheras-Hernandez
• Klaus H. Strobl
• Sergio Izquierdo
• Tim Bodenmüller
• Rudolph Triebel
• Javier Civera 0001

Metric depth estimation from visual sensors is crucial for robots to perceive, navigate, and interact with their environment. Traditional range imaging setups, such as stereo or structured light cameras, face hassles including calibration, occlusions, and hardware demands, with accuracy limited by the baseline between cameras. Single- and multi-view monocular depth offers a more compact alternative, but is constrained by the unobservability of the metric scale. Light field imaging provides a promising solution for estimating metric depth by using a unique lens configuration through a single device. However, its application to single-view dense metric depth is under-addressed mainly due to the technology's high cost, the lack of public benchmarks, and proprietary geometrical models and software. Our work explores the potential of focused plenoptic cameras for dense metric depth. We propose a novel pipeline that predicts metric depth from a single plenoptic camera shot by first generating a sparse metric point cloud using a neural network, which is then used to scale and align a dense relative depth map regressed by a foundation depth model, resulting in a dense metric depth. To validate it, we curated the Light Field & Stereo Image Dataset 1 1 Dataset available at https://zenodo.org/records/14224205. (LFS) of real-world light field images with stereo depth labels, filling a current gap in existing resources. Experimental results show that our pipeline produces accurate metric depth predictions, laying a solid groundwork for future research in this field. 2 2 Work partially supported by the DLR Impulse Project SaiNSOR.

ICRA Conference 2025 Conference Paper

Towards Autonomous Data Annotation and System-Agnostic Robotic Grasping Benchmarking with 3D-Printed Fixtures

• Wout Boerdijk
• Maximilian Durner
• Ryo Sakagami
• Peter Lehner
• Rudolph Triebel

The interaction of robots with their environment requires robust object-centric perception capabilities, typically achieved using learning-based methods trained on synthetic data. However, real-world deployment demands evaluating these capabilities in relevant environments, often involving extensive manual annotation for a quantitative analysis. Additionally, standardized evaluations for robotic tasks, such as grasping, need reproducible object scene configurations and performance benchmarks. We propose a solution to both problems by temporarily employing 3D-printed components, socalled fixtures, which can be designed for any rigid object. Once the scene is set up and object poses are extracted, the fixtures are removed, leaving the natural scene without any artificial distractions. The presented approach is seemingly applicable for pre-determined configurations of multiple objects, which enables precise re-building of scenes with consistent object-toobject relations. Our suggested annotation procedure achieves strong pose accuracy solely on RGB images without any manual involvement. We evaluate and show the usability of the proposed fixtures for automated real-world data annotation to fine-tune a detector and for benchmarking object pose estimation algorithms for robotic grasping. Code and fixture meshes for 3D printing are available at https://github.com/DLRRM/fixture_generation.

IROS Conference 2024 Conference Paper

A Learning-based Controller for Multi-Contact Grasps on Unknown Objects with a Dexterous Hand

• Dominik Winkelbauer
• Rudolph Triebel
• Berthold Bäuml

Existing grasp controllers usually either only support finger-tip grasps or need explicit configuration of the inner forces. We propose a novel grasp controller that supports arbitrary grasp types, including power grasps with multi-contacts, while operating self-contained on before unseen objects. No detailed contact information is needed, but only a rough 3D model, e. g. , reconstructed from a single depth image. First, the external wrench being applied to the object is estimated by using the measured torques at the joints. Then, the torques necessary to counteract the estimated wrench while keeping the object at its initial pose are predicted. The torques are commanded via desired joint angles to an underlying joint-level impedance controller. To reach real-time performance, we propose a learning-based approach that is based on a wrench estimator- and a torque predictor neural network. Both networks are trained in a supervised fashion using data generated via the analytical formulation of the controller. In an extensive simulation-based evaluation, we show that our controller is able to keep 83. 1% of the tested grasps stable when applying external wrenches with up to 10 N. At the same time, we outperform the two tested baselines by being more efficient and inducing less involuntary object movement. Finally, we show that the controller also works on the real DLR-Hand II, reaching a cycle time of 6 ms. Website: aidx-lab.org/grasping

ICRA Conference 2024 Conference Paper

A Safety-Adapted Loss for Pedestrian Detection in Autonomous Driving

• Maria Lyssenko
• Piyush Pimplikar
• Maarten Bieshaar
• Farzad Nozarian
• Rudolph Triebel

In safety-critical domains like autonomous driving (AD), errors by the object detector may endanger pedestrians and other vulnerable road users (VRU). As raw evaluation metrics are not an adequate safety indicator, recent works leverage domain knowledge to identify safety-relevant VRU, and to back-annotate the criticality of the interaction to the object detector. However, those approaches do not consider the safety factor in the deep neural network (DNN) training process. Thus, state-of-the-art DNN penalize all misdetections equally irrespective of their importance for the safe driving task. Hence, to mitigate the occurrence of safety-critical failure cases like false negatives, a safety-aware training strategy is needed to enhance the detection performance for critical pedestrians. In this paper, we propose a novel, safety-adapted loss variation that leverages the estimated per-pedestrian criticality during training. Therefore, we exploit the reachable set-based time-to-collision (TTC RSB ) metric from the motion domain along with distance information to account for the worst-case threat. Our evaluation results using RetinaNet and FCOS on the nuScenes dataset demonstrate that training the models with our safety-adapted loss function mitigates the misdetection of safety-critical pedestrians with robust performance for the general case, i. e. , safety-irrelevant pedestrians.

IROS Conference 2024 Conference Paper

Making the Flow Glow - Robot Perception under Severe Lighting Conditions using Normalizing Flow Gradients

• Simon Kristoffersson Lind
• Rudolph Triebel
• Volker Krüger

Modern robotic perception is highly dependent on neural networks. It is well known that neural network-based perception can be unreliable in real-world deployment, especially in difficult imaging conditions. Out-of-distribution detection is commonly proposed as a solution for ensuring reliability in real-world deployment. Previous work has shown that normalizing flow models can be used for out-of-distribution detection to improve reliability of robotic perception tasks. Specifically, camera parameters can be optimized with respect to the likelihood output from a normalizing flow, which allows a perception system to adapt to difficult vision scenarios. With this work we propose to use the absolute gradient values from a normalizing flow, which allows the perception system to optimize local regions rather than the whole image. By setting up a table top picking experiment with exceptionally difficult lighting conditions, we show that our method achieves a 60% higher success rate for an object detection task compared to previous methods.

ICRA Conference 2024 Conference Paper

Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments

• Alberto García-Hernández
• Riccardo Giubilato
• Klaus H. Strobl
• Javier Civera 0001
• Rudolph Triebel

Perceptual aliasing and weak textures pose significant challenges to the task of place recognition, hindering the performance of Simultaneous Localization and Mapping (SLAM) systems. This paper presents a novel model, called UMF (standing for Unifying Local and Global Multimodal Features) that 1) leverages multi-modality by cross-attention blocks between vision and LiDAR features, and 2) includes a re-ranking stage that re-orders based on local feature matching the top-k candidates retrieved using a global representation. Our experiments, particularly on sequences captured on a planetary-analogous environment, show that UMF outperforms significantly previous baselines in those challenging aliased environments. Since our work aims to enhance the reliability of SLAM in all situations, we also explore its performance on the widely used RobotCar dataset, for broader applicability. Code and models are available at https://github.com/DLR-RM/UMF.

IROS Conference 2023 Conference Paper

6D Object Pose Estimation from Approximate 3D Models for Orbital Robotics

• Maximilian Ulmer
• Maximilian Durner
• Martin Sundermeyer
• Manuel Stoiber
• Rudolph Triebel

We present a novel technique to estimate the 6D pose of objects from single images where the 3D geometry of the object is only given approximately and not as a precise 3D model. To achieve this, we employ a dense 2D-to-3D correspondence predictor that regresses 3D model coordinates for every pixel. In addition to the 3D coordinates, our model also estimates the pixel-wise coordinate error to discard correspondences that are likely wrong. This allows us to generate multiple 6D pose hypotheses of the object, which we then refine iteratively using a highly efficient region-based approach. We also introduce a novel pixel-wise posterior formulation by which we can estimate the probability for each hypothesis and select the most likely one. As we show in experiments, our approach is capable of dealing with extreme visual conditions including overexposure, high contrast, or low signal-to-noise ratio. This makes it a powerful technique for the particularly challenging task of estimating the pose of tumbling satellites for in-orbit robotic applications. Our method achieves state-of-the-art performance on the SPEED+ dataset and has won the SPEC2021 post-mortem competition. Code, trained models, and the used satellite model will be made publicly available.

IROS Conference 2023 Conference Paper

Efficient and Feasible Robotic Assembly Sequence Planning via Graph Representation Learning

• Matan Atad
• Jianxiang Feng
• Ismael Rodríguez
• Maximilian Durner
• Rudolph Triebel

Automatic Robotic Assembly Sequence Planning (RASP) can significantly improve productivity and resilience in modern manufacturing along with the growing need for greater product customization. One of the main challenges in realizing such automation resides in efficiently finding solutions from a growing number of potential sequences for increasingly complex assemblies. Besides, costly feasibility checks are always required for the robotic system. To address this, we propose a holistic graphical approach including a graph representation called Assembly Graph for product assemblies and a policy architecture, Graph Assembly Processing Network, dubbed GRACE for assembly sequence generation. With GRACE, we are able to extract meaningful information from the graph input and predict assembly sequences in a step-by-step manner. In experiments, we show that our approach can predict feasible assembly sequences across product variants of aluminum profiles based on data collected in simulation of a dual-armed robotic system. We further demonstrate that our method is capable of detecting infeasible assemblies, substantially alleviating the undesirable impacts from false predictions, and hence facilitating real-world deployment soon. Code and training data are available at https://github.com/DLR-RM/GRACE.

IROS Conference 2023 Conference Paper

Fusing Visual Appearance and Geometry for Multi-Modality 6DoF Object Tracking

• Manuel Stoiber
• Mariam Elsayed
• Anne E. Reichert
• Florian Steidle
• Dongheui Lee
• Rudolph Triebel

In many applications of advanced robotic manipulation, six degrees of freedom (6DoF) object pose estimates are continuously required. In this work, we develop a multi-modality tracker that fuses information from visual appearance and geometry to estimate object poses. The algorithm extends our previous method ICG, which uses geometry, to additionally consider surface appearance. In general, object surfaces contain local characteristics from text, graphics, and patterns, as well as global differences from distinct materials and colors. To incorporate this visual information, two modalities are developed. For local characteristics, keypoint features are used to minimize distances between points from keyframes and the current image. For global differences, a novel region approach is developed that considers multiple regions on the object surface. In addition, it allows the modeling of external geometries. Experiments on the YCB-Video and OPT datasets demonstrate that our approach ICG+ performs best on both datasets, outperforming both conventional and deep learning-based methods. At the same time, the algorithm is highly efficient and runs at more than 300 Hz. The source code of our tracker is publicly available.

ICML Conference 2023 Conference Paper

Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks

• Dominik Schnaus
• Jongseok Lee
• Daniel Cremers
• Rudolph Triebel

In this work, we propose a novel prior learning method for advancing generalization and uncertainty estimation in deep neural networks. The key idea is to exploit scalable and structured posteriors of neural networks as informative priors with generalization guarantees. Our learned priors provide expressive probabilistic representations at large scale, like Bayesian counterparts of pre-trained models on ImageNet, and further produce non-vacuous generalization bounds. We also extend this idea to a continual learning framework, where the favorable properties of our priors are desirable. Major enablers are our technical contributions: (1) the sums-of-Kronecker-product computations, and (2) the derivations and optimizations of tractable objectives that lead to improved generalization bounds. Empirically, we exhaustively show the effectiveness of this method for uncertainty estimation and generalization.

IROS Conference 2023 Conference Paper

Learning-Based Real-Time Torque Prediction for Grasping Unknown Objects with a Multi-Fingered Hand

• Dominik Winkelbauer
• Berthold Bäuml
• Rudolph Triebel

When grasping objects with a multi-finger hand, it is crucial for the grasp stability to apply the correct torques at each joint so that external forces are countered. Most current systems use simple heuristics instead of modeling the required torque correctly. Instead, we propose a learning-based approach that is able to predict torques for grasps on unknown objects in real-time. The neural network, trained end-to-end using supervised learning, is shown to predict torques that are more efficient, and the objects are held with less involuntary movement compared to all tested heuristic baselines. Specifically, for 90 % of the grasps the translational deviation of the object is below 2. 9 mm and the rotational below 3. 1°. To generate training data, we formulate the analytical computation of torques as an optimization problem and handle the indeterminacy of multi-contacts using an elastic model. We further show that the network generalizes to predict torques for unknown objects on the real robot system with an inference time of 1. 5 ms. Website: dlr-alr.github.io/grasping/

IROS Conference 2022 Conference Paper

A Two-stage Learning Architecture that Generates High-Quality Grasps for a Multi-Fingered Hand

• Dominik Winkelbauer
• Berthold Bäuml
• Matthias Humt
• Nils Thuerey
• Rudolph Triebel

We investigate the problem of planning stable grasps for object manipulations using an 18-DOF robotic hand with four fingers. The main challenge here is the high-dimensional search space, and we address this problem using a novel two-stage learning process. In the first stage, we train an autoregressive network called the hand-pose-generator, which learns to generate a distribution of valid 6D poses of the palm for a given volumetric object representation. In the second stage, we employ a network that regresses 12D finger joint configurations and a scalar grasp quality from given object representations and palm poses. To train our networks, we use synthetic training data generated by a novel grasp planning algorithm, which also proceeds stage-wise: first the palm pose, then the finger positions. Here, we devise a Bayesian Optimization scheme for the palm pose and a physics-based grasp pose metric to rate stable grasps. In experiments on the YCB benchmark data set, we show a grasp success rate of over 83%, as well as qualitative results grasping unknown objects on a real robot system.

IROS Conference 2022 Conference Paper

Bayesian Active Learning for Sim-to-Real Robotic Perception

• Jianxiang Feng
• Jongseok Lee
• Maximilian Durner
• Rudolph Triebel

While learning from synthetic training data has recently gained an increased attention, in real-world robotic applications, there are still performance deficiencies due to the so-called Sim-to-Real gap. In practice, this gap is hard to resolve with only synthetic data. Therefore, we focus on an efficient acquisition of real data within a Sim-to-Real learning pipeline. Concretely, we employ deep Bayesian active learning to minimize manual annotation efforts and devise an autonomous learning paradigm to select the data that is considered useful for the human expert to annotate. To achieve this, a Bayesian Neural Network (BNN) object detector providing reliable un-certainty estimates is adapted to infer the informativeness of the unlabeled data. Furthermore, to cope with misalignments of the label distribution in uncertainty-based sampling, we develop an effective randomized sampling strategy that performs favorably compared to other complex alternatives. In our experiments on object classification and detection, we show benefits of our approach and provide evidence that labeling efforts can be reduced significantly. Finally, we demonstrate the practical effectiveness of this idea in a grasping task on an assistive robot.

IROS Conference 2022 Conference Paper

RECALL: Rehearsal-free Continual Learning for Object Classification

• Markus Knauer
• Maximilian Denninger
• Rudolph Triebel

Convolutional neural networks show remarkable results in classification but struggle with learning new things on the fly. We present a novel rehearsal-free approach, where a deep neural network is continually learning new unseen object categories without saving any data of prior sequences. Our approach is called RECALL, as the network recalls categories by calculating logits for old categories before training new ones. These are then used during training to avoid changing the old categories. For each new sequence, a new head is added to accommodate the new categories. To mitigate forgetting, we present a regularization strategy where we replace the classification with a regression. Moreover, for the known categories, we propose a Mahalanobis loss that includes the variances to account for the changing densities between known and unknown categories. Finally, we present a novel dataset for continual learning (HOWS-CL-25), especially suited for object recognition on a mobile robot, including 150, 795 synthetic images of 25 household object categories. Our approach RECALL outperforms the current state of the art on CORe50 and iCIFAR-100 and reaches the best performance on HOWS-CL-25.

ICRA Conference 2022 Conference Paper

Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning

• Elie Aljalbout
• Maximilian Ulmer
• Rudolph Triebel

Vision-based reinforcement learning (RL) is a promising approach to solve control tasks involving images as the main observation. State-of-the-art RL algorithms still struggle in terms of sample efficiency, especially when using image observations. This has led to increased attention on integrating state representation learning (SRL) techniques into the RL pipeline. Work in this field demonstrates a substantial improvement in sample efficiency among other benefits. However, to take full advantage of this paradigm, the quality of samples used for training plays a crucial role. More importantly, the diversity of these samples could affect the sample efficiency of vision-based RL, but also its generalization capability. In this work, we present an approach to improve sample diversity for state representation learning. Our method enhances the exploration capability of RL algorithms, by taking advantage of the SRL setup. Our experiments show that our proposed approach boosts the visitation of problematic states, improves the learned state representation, and outperforms the baselines for all tested environments. These results are most apparent for environments where the baseline methods struggle. In simple environments, our method contributes to stabilizing the training, reducing the reward variance, and improving sample efficiency.

IROS Conference 2022 Conference Paper

The Probabilistic Robot Kinematics Model and its Application to Sensor Fusion

• Lukas Meyer
• Klaus H. Strobl
• Rudolph Triebel

Robots with elasticity in structural components can suffer from undesired end-effector positioning imprecision, which exceeds the accuracy requirements for successful manipulation. We present the Probabilistic-Product-Of-Exponentials robot model, a novel approach for kinematic modeling of robots. It does not only consider the robot's deterministic geometry but additionally models time-varying and configuration-dependent errors in a probabilistic way. Our robot model allows to propagate the errors along the kinematic chain and to compute their influence on the end-effector pose. We apply this model in the context of sensor fusion for manipulator pose correction for two different robotic systems. The results of a simulation study, as well as of an experiment, demonstrate that probabilistic configuration-dependent error modeling of the robot kinematics is crucial in improving pose estimation results.

IROS Conference 2022 Conference Paper

Towards Safety-Aware Pedestrian Detection in Autonomous Systems

• Maria Lyssenko
• Christoph Gladisch
• Christian Heinzemann
• Matthias Woehrle
• Rudolph Triebel

In this paper, we present a framework to assess the quality of a pedestrian detector in an autonomous driving scenario. To do this, we exploit performance metrics from the domain of computer vision on one side and so-called threat metrics from the motion planning domain on the other side. Based on a reachability analysis that accounts for the uncertainty in future motions of other traffic participants, we can determine the worst-case threat from the planning domain and relate it to the corresponding detection from the visual input. Our evaluation results for a RetinaNet on the Argoverse 1. 1 [1] dataset show that already a rather simple threat metric such as time-to-collision (TTC) allows to select potentially dangerous interactions between the ego vehicle and a pedestrian when purely vision-based detections fail, even if they are passed to a subsequent object tracker. In addition, our results show that two different DNNs (Deep Neural Networks) with comparable performance differ significantly in the number of critical scenarios that we can identify with our method.

ICRA Conference 2021 Conference Paper

"What's This? " - Learning to Segment Unknown Objects from Manipulation Sequences

• Wout Boerdijk
• Martin Sundermeyer
• Maximilian Durner
• Rudolph Triebel

We present a novel framework for self-supervised grasped object segmentation with a robotic manipulator. Our method successively learns an agnostic foreground segmentation followed by a distinction between manipulator and object solely by observing the motion between consecutive RGB frames. In contrast to previous approaches, we propose a single, end-to-end trainable architecture which jointly incorporates motion cues and semantic knowledge. Furthermore, while the motion of the manipulator and the object are substantial cues for our algorithm, we present means to robustly deal with distraction objects moving in the background, as well as with completely static scenes. Our method neither depends on any visual registration of a kinematic robot or 3D object models, nor on precise hand-eye calibration or any additional sensor data. By extensive experimental evaluation we demonstrate the superiority of our framework and provide detailed insights on its capability of dealing with the aforementioned extreme cases of motion. We also show that training a semantic segmentation network with the automatically labeled data achieves results on par with manually annotated training data. Code and pretrained model are available at https://github.com/DLR-RM/DistinctNet.

IROS Conference 2021 Conference Paper

A Photorealistic Terrain Simulation Pipeline for Unstructured Outdoor Environments

• Marcus Gerhard Müller
• Maximilian Durner
• Abel Gawel
• Wolfgang Stürzl
• Rudolph Triebel
• Roland Siegwart

Suitable datasets are an integral part of robotics research, especially for training neural networks in robot perception. However, in many domains, suitable real-world data are scarce and cannot be easily obtained. This problem is especially prevalent for unstructured outdoor environments, in particular, planetary ones. Recent advances in photorealistic simulations help researchers to simulate close-to-real data in many domains. Yet, there exists no high-quality synthetic data for planetary exploration tasks. Also, existing simulators lack the fidelity required for generating planetary data, which is inherently less structured than human environments. Synthetic planetary data requires careful modeling and annotation of many different terrain aspect and details, such as textures and distributions of rocks, to become a valuable test-bed for robotics. To fill this gap, we present a novel simulator specifically designed for the needs of planetary robotics visual tasks, but also applicable for other outdoor environments. Our simulator is capable of generating large varieties of (planetary) outdoor scenes with rich generation of meta data, such as multilevel semantic and instance annotations. To demonstrate the wide applicability of this new simulator, we evaluate its performance on typical robotics applications, i. e. semantic segmentation, instance segmentation, and visual SLAM. Our simulator is accessible under https://github.com/DLR-RM/oaisys.

ICRA Conference 2021 Conference Paper

Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes

• Martin Sundermeyer
• Arsalan Mousavian
• Rudolph Triebel
• Dieter Fox

Grasping unseen objects in unconstrained, cluttered environments is an essential skill for autonomous robotic manipulation. Despite recent progress in full 6-DoF grasp learning, existing approaches often consist of complex sequential pipelines that possess several potential failure points and run-times unsuitable for closed-loop grasping. Therefore, we propose an end-to-end network that efficiently generates a distribution of 6-DoF parallel-jaw grasps directly from a depth recording of a scene. Our novel grasp representation treats 3D points of the recorded point cloud as potential grasp contacts. By rooting the full 6-DoF grasp pose and width in the observed point cloud, we can reduce the dimensionality of our grasp representation to 4-DoF which greatly facilitates the learning process. Our class-agnostic approach is trained on 17 million simulated grasps and generalizes well to real world sensor data. In a robotic grasping study of unseen objects in structured clutter we achieve over 90% success rate, cutting the failure rate in half compared to a recent state-of-the-art method. Video of the real world experiments and code are available at https://research.nvidia.com/publication/2021-03_Contact-GraspNet%3A--Efficient.

ICRA Conference 2021 Conference Paper

DOT: Dynamic Object Tracking for Visual SLAM

• Irene Ballester
• Alejandro Fontán
• Javier Civera 0001
• Klaus H. Strobl
• Rudolph Triebel

In this paper we present DOT (Dynamic Object Tracking), a front-end that added to existing SLAM systems can significantly improve their robustness and accuracy in highly dynamic environments. DOT combines instance segmentation and multi-view geometry to generate masks for dynamic objects in order to allow SLAM systems based on rigid scene models to avoid such image areas in their optimizations. To determine which objects are actually moving, DOT segments first instances of potentially dynamic objects and then, with the estimated camera motion, tracks such objects by minimizing the photometric reprojection error. This short-term tracking improves the accuracy of the segmentation with respect to other approaches. In the end, only actually dynamicmasks are generated. We have evaluated DOT with ORB-SLAM 2 [1] in three public datasets. Our results show that our approach improves significantly the accuracy and robustness of ORB-SLAM2, especially in highly dynamic scenes.

ICRA Conference 2021 Conference Paper

Exploration of Large Outdoor Environments Using Multi-Criteria Decision Making

• Hannah Lehner
• Martin J. Schuster
• Tim Bodenmüller
• Rudolph Triebel

We present a Multi-Criteria Decision Making (MCDM) framework specifically designed for planetary exploration. Our work is based on PROMETHEE II, which allows operators to add task-specific criteria and conditions. We extended this algorithm to improve its resource usage by reducing the number of candidate exploration goals that have to be evaluated and compared. This is crucial when given a large number of goals, as is typical for outdoor environments. In addition, we identified five different criteria for planetary exploration, including a novel criterion that we call Direction of Interest (DOI), and use a categorization of these for further resource optimizations. We thereby ensure that the CPU usage of our decision making method can meet the limited budget of a space rover. We present simulated and real-world experiments with the Lightweight Rover Unit (LRU) and show a reduction of the processing time for decision making of approx. 70%.

ICRA Conference 2021 Conference Paper

Learning to Localize in New Environments from Synthetic Training Data

• Dominik Winkelbauer
• Maximilian Denninger
• Rudolph Triebel

Most existing approaches for visual localization either need a detailed 3D model of the environment or, in the case of learning-based methods, must be retrained for each new scene. This can either be very expensive or simply impossible for large, unknown environments, for example in search-and-rescue scenarios. Although there are learning-based approaches that operate scene-agnostically, the generalization capability of these methods is still outperformed by classical approaches. In this paper, we present an approach that can generalize to new scenes by applying specific changes to the model architecture, including an extended regression part, the use of hierarchical correlation layers, and the exploitation of scale and uncertainty information. Our approach outperforms the 5-point algorithm using SIFT features on equally big images and additionally surpasses all previous learning-based approaches that were trained on different data. It is also superior to most of the approaches that were specifically trained on the respective scenes. We also evaluate our approach in a scenario with only very few reference images, showing that under such more realistic conditions our learning-based approach considerably exceeds both existing learning-based and classical methods.

IROS Conference 2021 Conference Paper

Multi-Modal Loop Closing in Unstructured Planetary Environments with Visually Enriched Submaps

• Riccardo Giubilato
• Mallikarjuna Vayugundla
• Wolfgang Stürzl
• Martin J. Schuster
• Armin Wedler
• Rudolph Triebel

Future planetary missions will rely on rovers that can autonomously explore and navigate in unstructured environments. An essential element is the ability to recognize places that were already visited or mapped. In this work, we leverage the ability of stereo cameras to provide both visual and depth information, guiding the search and validation of loop closures from a multi-modal perspective. We propose to augment submaps that are created by aggregating stereo point clouds, with visual keyframes. Point clouds matches are found by comparing CSHOT descriptors and validated by clustering, while visual matches are established by comparing keyframes using Bag-of-Words (BoW) and ORB descriptors. The relative transformations resulting from both keyframe and point cloud matches are then fused to provide pose constraints between submaps in our graph-based SLAM framework. Using the LRU rover, we performed several tests in both an indoor laboratory environment as well as a challenging planetary analog environment on Mount Etna, Italy, consisting of areas where either keyframes or point clouds alone failed to provide adequate matches demonstrating the benefit of the proposed multi-modal approach.

IROS Conference 2021 Conference Paper

Towards Robust Monocular Visual Odometry for Flying Robots on Planetary Missions

• Martin Wudenka
• Marcus Gerhard Müller
• Nikolaus Demmel
• Armin Wedler
• Rudolph Triebel
• Daniel Cremers
• Wolfgang Stürzl

In the future, extraterrestrial expeditions will not only be conducted by rovers but also by flying robots. The technical demonstration drone Ingenuity, that just landed on Mars, will mark the beginning of a new era of exploration unhindered by terrain traversability. Robust self-localization is crucial for that. Cameras that are lightweight, cheap and information-rich sensors are already used to estimate the ego-motion of vehicles. However, methods proven to work in man-made environments cannot simply be deployed on other planets. The highly repetitive textures present in the wastelands of Mars pose a huge challenge to descriptor matching based approaches. In this paper, we present an advanced robust monocular odometry algorithm that uses efficient optical flow tracking to obtain feature correspondences between images and a refined keyframe selection criterion. In contrast to most other approaches, our framework can also handle rotation-only motions that are particularly challenging for monocular odometry systems. Furthermore, we present a novel approach to estimate the current risk of scale drift based on a principal component analysis of the relative translation information matrix. This way we obtain an implicit measure of uncertainty. We evaluate the validity of our approach on all sequences of a challenging real-world dataset captured in a Mars-like environment and show that it outperforms state-of-the-art approaches. The source code is publicly available at: https://github.com/DLR-RM/granit.

IROS Conference 2021 Conference Paper

Unknown Object Segmentation from Stereo Images

• Maximilian Durner
• Wout Boerdijk
• Martin Sundermeyer
• Werner Friedl
• Zoltán-Csaba Márton
• Rudolph Triebel

Although instance-aware perception is a key prerequisite for many autonomous robotic applications, most of the methods only partially solve the problem by focusing solely on known object categories. However, for robots interacting in dynamic and cluttered environments, this is not realistic and severely limits the range of potential applications. Therefore, we propose a novel object instance segmentation approach that does not require any semantic or geometric information of the objects beforehand. In contrast to existing works, we do not explicitly use depth data as input, but rely on the insight that slight viewpoint changes, which for example are provided by stereo image pairs, are often sufficient to determine object boundaries and thus to segment objects. Focusing on the versatility of stereo sensors, we employ a transformer-based architecture that maps directly from the pair of input images to the object instances. This has the major advantage that instead of a noisy, and potentially incomplete depth map as an input, on which the segmentation is computed, we use the original image pair to infer the object instances and a dense depth map. In experiments in several different application domains, we show that our Instance Stereo Transformer (INSTR) algorithm outperforms current state-of-the-art methods that are based on depth maps. Training code and pretrained models are available at https://github.com/DLR-RM/instr.

ICML Conference 2020 Conference Paper

Estimating Model Uncertainty of Neural Networks in Sparse Information Form

• Jongseok Lee
• Matthias Humt
• Jianxiang Feng
• Rudolph Triebel

We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulation of the Multivariate Normal Distribution (MND), also known as the information form. The key insight of our work is that the information matrix, i. e. the inverse of the covariance matrix tends to be sparse in its spectrum. Therefore, dimensionality reduction techniques such as low rank approximations (LRA) can be effectively exploited. To achieve this, we develop a novel sparsification algorithm and derive a cost-effective analytical sampler. As a result, we show that the information form can be scalably applied to represent model uncertainty in DNNs. Our exhaustive theoretical analysis and empirical evaluations on various benchmarks show the competitiveness of our approach over the current methods.

IROS Conference 2020 Conference Paper

Gaussian Process Gradient Maps for Loop-Closure Detection in Unstructured Planetary Environments

• Cedric Le Gentil
• Mallikarjuna Vayugundla
• Riccardo Giubilato
• Wolfgang Stürzl
• Teresa A. Vidal-Calleja
• Rudolph Triebel

The ability to recognize previously mapped locations is an essential feature for autonomous systems. Unstructured planetary-like environments pose a major challenge to these systems due to the similarity of the terrain. As a result, the ambiguity of the visual appearance makes state-of-the-art visual place recognition approaches less effective than in urban or man-made environments. This paper presents a method to solve the loop closure problem using only spatial information. The key idea is to use a novel continuous and probabilistic representations of terrain elevation maps. Given 3D point clouds of the environment, the proposed approach exploits Gaussian Process (GP) regression with linear operators to generate continuous gradient maps of the terrain elevation information. Traditional image registration techniques are then used to search for potential matches. Loop closures are verified by leveraging both the spatial characteristic of the elevation maps (SE (2) registration) and the probabilistic nature of the GP representation. A submap-based localization and mapping framework is used to demonstrate the validity of the proposed approach. The performance of this pipeline is evaluated and benchmarked using real data from a rover that is equipped with a stereo camera and navigates in challenging, unstructured planetary-like environments in Morocco and on Mt. Etna.

IROS Conference 2020 Conference Paper

Robust MUSIC-Based Sound Source Localization in Reverberant and Echoic Environments

• Marco Sewtz
• Tim Bodenmüller
• Rudolph Triebel

Intuitive human robot interfaces like speech or gesture recognition are essential for gaining acceptance for robots in daily life. However, such interaction requires that the robot detects the human's intention to interact, tracks his position and keeps its sensor systems in an optimal configuration. Audio is a suitable modality for such task as it allows for detecting a speaker in arbitrary positions around the robot. In this paper, we present a novel approach for localization of sound sources by analyzing the frequency spectrum of the received signal and applying a motion model to the estimation process. We use an improved version of the Generalized Singular Value Decomposition (GSVD) based MUltiple SIgnal Classification (MUSIC) algorithm as a direction of arrival (DoA) estimator. Further, we introduce a motion model to enable robust localization in reverberant and echoic environments. We evaluate the system under real conditions in an experimental setup. Our experiments show that our approach outperforms current state-of-the-art algorithm and demonstrate the robustness against the previously mentioned disruptive factors.

ICRA Conference 2020 Conference Paper

Visual-Inertial Telepresence for Aerial Manipulation

• Jongseok Lee
• Ribin Balachandran
• Yuri S. Sarkisov
• Marco De Stefano
• Andre Coelho
• Kashmira Shinde
• Min Jun Kim 0003
• Rudolph Triebel

This paper presents a novel telepresence system for enhancing aerial manipulation capabilities. It involves not only a haptic device, but also a virtual reality that provides a 3D visual feedback to a remotely-located teleoperator in real-time. We achieve this by utilizing onboard visual and inertial sensors, an object tracking algorithm and a pregenerated object database. As the virtual reality has to closely match the real remote scene, we propose an extension of a marker tracking algorithm with visual-inertial odometry. Both indoor and outdoor experiments show benefits of our proposed system in achieving advanced aerial manipulation tasks, namely grasping, placing, force exertion and peg-in-hole insertion.

ICRA Conference 2019 Conference Paper

Visual Repetition Sampling for Robot Manipulation Planning

• En Yen Puang
• Peter Lehner
• Zoltán-Csaba Márton
• Maximilian Durner
• Rudolph Triebel
• Alin Albu-Schäffer

One of the main challenges in sampling-based motion planners is to find an efficient sampling strategy. While methods such as Rapidly-exploring Random Tree (RRT) have shown to be more reliable in complex environments than optimization-based methods, they often require longer planning times, which reduces their usability for real-time applications. Recently, biased sampling methods have shown to remedy this issue. For example Gaussian Mixture Models (GMMs) have been used to sample more efficiently in feasible regions of the configuration space. Once the GMM is learned, however, this approach does not adapt its biases to individual planning scene during inference. Hence, we propose in this work a more efficient sampling strategy to further bias the GMM based on visual input upon query. We employ an autoencoder trained entirely in simulation to extract features from depth images and use the latent representation to adjust the weights of each mixture components in the GMM. We show empirically that this improves the sampling efficiency of an RRT motion planner in both real and simulated scenes.

IROS Conference 2018 Conference Paper

Appearance-Based Along-Route Localization for Planetary Missions

• Iris Lynne Grixa
• Philipp Schulz
• Wolfgang Stürzl
• Rudolph Triebel

We propose an appearance-based along-route localization algorithm that relies on robust place recognition by matching image sequences instead of individual frames. Our approach extends state of the art place recognition framework SeqSLAM in several aspects to realize real-time localization along routes for autonomous navigation. First, our method is online in that we only rely on the recently observed image frames. Second, we provide a homing mechanism based on rotations computed from frame matches. And third, we use a more flexible mechanism to search for matching locations, not restricting the search to straight lines in the cost matrix as in SeqSLAM, but allowing for a wide variety of route traversal conditions such as varying velocities or rotational and translational viewpoint differences. We investigate different image preprocessing steps as well as image similarity metrics wrt. their influence on illumination and viewpoint invariance for a more robust place recognition. On a new challenging dataset, recorded in real world experiments with a planetary rover, in the course of a Moon-analogue mission on Sicily's Mount Etna, we show the feasibility of our direct, sequence-based approach to along-route localization.

IROS Conference 2018 Conference Paper

Incremental Semi-Supervised Learning from Streams for Object Classification

• Ioannis Chiotellis
• Franziska Zimmermann
• Daniel Cremers
• Rudolph Triebel

The Label Propagation (LP) algorithm, first introduced by Zhu and Ghahramani [1], is a semi-supervised method used in transductive learning scenarios, where all data are available already in the beginning. In this work, we present a novel extension of the LP algorithm for applications where data samples are observed sequentially - as is the case in autonomous driving. Specifically, our “Incremental Label Propagation” algorithm efficiently approximates the so called harmonic solution on a nearest-neighbor graph that is regularly updated by new labeled and unlabeled nodes. We achieve this by reformulating the original algorithm based on an active set of nodes and by introducing a threshold to decide whether the label of a given node should be updated or not. Our method can also deal with graphs that are not fully connected, and we give a formal convergence proof for this general case. In experiments on the challenging KITTI benchmark data stream, we show superior performance in terms of both test accuracy and number of required training labels compared to state-of-the-art online learning methods.

IROS Conference 2018 Conference Paper

Persistent Anytime Learning of Objects from Unseen Classes

• Maximilian Denninger
• Rudolph Triebel

We present a fast and very effective method for object classification that is particularly suited for robotic applications such as grasping and semantic mapping. Our approach is based on a Random Forest classifier that can be trained incrementally. This has the major benefit that semantic information from new data samples can be incorporated without retraining the entire model. Even if new samples from a previously unseen class are presented, our method is able to perform efficient updates and learn a sustainable representation for this new class. Further features of our method include a very fast and memory-efficient implementation, as well as the ability to interrupt the learning process at any time without a significant performance degradation. Experiments on benchmark data for robotic applications show the clear benefits of our incremental approach and its competitiveness with standard offline methods in terms of classification accuracy.

ICRA Conference 2018 Conference Paper

Semantic Labeling of Indoor Environments from 3D RGB Maps

• Manuel Brucker
• Maximilian Durner
• Rares Ambrus
• Zoltán-Csaba Márton
• Axel Wendt
• Patric Jensfelt
• Kai O. Arras
• Rudolph Triebel

We present an approach to automatically assign semantic labels to rooms reconstructed from 3D RGB maps of apartments. Evidence for the room types is generated using state-of-the-art deep-learning techniques for scene classification and object detection based on automatically generated virtual RGB views, as well as from a geometric analysis of the map's 3D structure. The evidence is merged in a conditional random field, using statistics mined from different datasets of indoor environments. We evaluate our approach qualitatively and quantitatively and compare it to related methods.

IROS Conference 2017 Conference Paper

A method for hand-eye and camera-to-camera calibration for limited fields of view

• Christian Nissler
• Zoltán-Csaba Márton
• Hannes Kisner
• Ulrike Thomas
• Rudolph Triebel

In classical robot-camera calibration, a 6D transformation between the camera frame and the local frame of a robot is estimated by first observing a known calibration object from a number of different view points and then finding transformation parameters that minimize the reprojection error. The disadvantage with this is that often not all configurations can be reached by the end-effector, which leads to an inaccurate parameter estimation. Therefore, we propose a more versatile method based on the detection of oriented visual features, in our case AprilTags. From a collected number of such detections during a defined rotation of a joint, we fit a Bingham distribution by maximizing the observation likelihood of the detected orientations. After a tilt and a second rotation, a camera-to-joint transformation can be determined. In experiments with accurate ground truth available, we evaluate our approach in terms of precision and robustness, both for hand-eye/robot-camera and for camera-camera calibration, with classical solutions serving as a baseline.

IROS Conference 2017 Conference Paper

Selecting CNN features for online learning of 3D objects

• Monika Ullrich
• Haider Ali
• Maximilian Durner
• Zoltán-Csaba Márton
• Rudolph Triebel

We present a novel method for classifying 3D objects that is particularly tailored for the requirements in robotic applications. The major challenges here are the comparably small amount of available training data and the fact that often data is perceived in streams and not in fixed-size pools. Traditional state-of-the-art learning methods, however, require a large amount of training data, and their online learning capabilities are usually limited. Therefore, we propose a modality-specific selection of convolutional neural networks (CNN), pre-trained or fine-tuned, in combination with a classifier that is designed particularly for online learning from data streams, namely the Mondrian Forest (MF). We show that this combination of trained features obtained from a CNN can be improved further if a feature selection algorithm is applied. In our experiments, we use the resulting features both with a MF and a linear Support Vector Machine (SVM). With SVM we beat the state of the art on an RGB-D dataset, while with MF a strong result for active learning is achieved.

ICRA Conference 2016 Conference Paper

Stream-based Active Learning for efficient and adaptive classification of 3D objects

• Alexander Narr
• Rudolph Triebel
• Daniel Cremers

We present a new Active Learning approach for classifying objects from streams of 3D point cloud data. The major problems here are the non-uniform occurrence of class instances and the unbalanced numbers of samples per class. We show that standard online learning methods based on decision trees perform comparably bad for such data streams, which are however particularly relevant for mobile robots that need to learn semantics persistently. To address this, we use Mondrian forests (MF), a recent online learning algorithm that is independent on the data order. We present an extension of that algorithm and show that MF are less overconfident than standard Random Forests. In experiments on the KITTI benchmark, we show that this leads to a substantially improved classification performance for data streams, rendering our approach very attractive for lifelong robot learning applications.

ICRA Conference 2015 Conference Paper

Active online confidence boosting for efficient object classification

• Dennis Mund
• Rudolph Triebel
• Daniel Cremers

We present a novel efficient algorithm for object classification. Our method is based on the active learning framework, in which training and classification are performed in loops, and new ground truth labels are queried from the supervisor in each loop. Our underlying classifier is from the family of boosting methods, but in contrast to earlier methods, our Confidence Boosting particularly focusses on misclassified samples that have a high classification confidence associated. We show that weighting these samples more than others leads to a decrease of overconfidence, for which we give a formal definition. As a result, our classifier is better suited for active learning, leading to steeper learning curves and less required label queries. We show the benefits of our approach on standard data sets from machine learning and robotics.

IROS Conference 2015 Conference Paper

Semi-supervised online learning for efficient classification of objects in 3D data streams

• Ye Tao
• Rudolph Triebel
• Daniel Cremers

We present a novel learning algorithm especially designed for challenging, large-scale classification problems in mobile robotics. Our method addresses two important aims: first it reduces the required amount of interaction with a human supervisor, which increases the level of autonomy of the learning process. And second, it has the capability to update its internal representation online with every new observed data sample, which makes it adaptive to new environments. The proposed method is based on a combination of two established methods, namely Online Star Clustering and Label Propagation, but it extends and modifies these in such a way that significant shortcomings such as classification inaccuracy and run time inefficiency can be resolved. In experiments on large benchmark data sets, we show that our approach can quickly learn to classify 3D objects with a significantly reduced amount of required ground truth labels for training.

ICRA Conference 2013 Conference Paper

Knowing when we don't know: Introspective classification for mission-critical decision making

• Hugo Grimmett
• Rohan Paul
• Rudolph Triebel
• Ingmar Posner

Classification precision and recall have been widely adopted by roboticists as canonical metrics to quantify the performance of learning algorithms. This paper advocates that for robotics applications, which often involve mission-critical decision making, good performance according to these standard metrics is desirable but insufficient to appropriately characterise system performance. We introduce and motivate the importance of a classifier's introspective capacity: the ability to mitigate potentially overconfident classifications by an appropriate assessment of how qualified the system is to make a judgement on the current test datum. We provide an intuition as to how this introspective capacity can be achieved and systematically investigate it in a selection of classification frameworks commonly used in robotics: support vector machines, LogitBoost classifiers and Gaussian Process classifiers (GPCs). Our experiments demonstrate that for common robotics tasks a framework such as a GPC exhibits a superior introspective capacity while maintaining commensurate classification performance to more popular, alternative approaches.

AAAI Conference 2012 Conference Paper

Parsing Outdoor Scenes from Streamed 3D Laser Data Using Online Clustering and Incremental Belief Updates

• Rudolph Triebel
• Rohan Paul
• Daniela Rus
• Paul Newman

In this paper, we address the problem of continually parsing a stream of 3D point cloud data acquired from a laser sensor mounted on a road vehicle. We leverage an online star clustering algorithm coupled with an incremental belief update in an evolving undirected graphical model. The fusion of these techniques allows the robot to parse streamed data and to continually improve its understanding of the world. The core competency produced is an ability to infer object classes from similarities based on appearance and shape features, and to concurrently combine that with a spatial smoothing algorithm incorporating geometric consistency. This formulation of feature-space star clustering modulating the potentials of a spatial graphical model is entirely novel. In our method, the two sources of information: feature similarity and geometrical consistency are fed continually into the system, improving the belief over the class distributions as new data arrives. The algorithm obviates the need for hand-labeled training data and makes no apriori assumptions on the number or characteristics of object categories. Rather, they are learnt incrementally over time from streamed input data. In experiments performed on real 3D laser data from an outdoor scene, we show that our approach is capable of obtaining an everimproving unsupervised scene categorization.

IROS Conference 2012 Conference Paper

Semantic categorization of outdoor scenes with uncertainty estimates using multi-class gaussian process classification

• Rohan Paul
• Rudolph Triebel
• Daniela Rus
• Paul Newman 0001

This paper presents a novel semantic categorization method for 3D point cloud data using supervised, multiclass Gaussian Process (GP) classification. In contrast to other approaches, and particularly Support Vector Machines, which probably are the most used method for this task to date, GPs have the major advantage of providing informative uncertainty estimates about the resulting class labels. As we show in experiments, these uncertainty estimates can either be used to improve the classification by neglecting uncertain class labels or - more importantly - they can serve as an indication of the under-representation of certain classes in the training data. This means that GP classifiers are much better suited in a lifelong learning framework, where not all classes are represented initially, but instead new training data arrives during the operation of the robot.

ICRA Conference 2011 Conference Paper

Bayesian on-line learning of driving behaviors

• Jérôme Maye
• Rudolph Triebel
• Luciano Spinello
• Roland Siegwart

This paper presents a novel self-supervised on-line learning method to discover driving behaviors from data acquired with an inertial measurement unit (IMU) and a camera. Both sensors where mounted in a car that was driven by a human through a typical city environment with intersections, pedestrian crossings and traffic lights. The presented system extracts motion segments from the IMU data and relates them to visual cues obtained from camera data. It employs a Bayesian on-line estimation method to discover the motion segments based on change-point detection and uses a Dirichlet Compound Multinomial (DCM) model to represent the visual features extracted from the camera images. By incorporating these visual cues into the on-line estimation process, labels are computed that are equal for similar motion segments. As a result, typical traffic situations such as braking maneuvers in front of a red light can be identified automatically. Furthermore, appropriate actions in form of observed motion changes are associated to the discovered traffic situations. The approach is evaluated on a real data set acquired in the center of Zurich.

AAAI Conference 2010 Conference Paper

A Layered Approach to People Detection in 3D Range Data

• Luciano Spinello
• Kai Arras
• Rudolph Triebel
• Roland Siegwart

People tracking is a key technology for autonomous systems, intelligent cars and social robots operating in populated environments. What makes the task di cult is that the appearance of humans in range data can change drastically as a function of body pose, distance to the sensor, self-occlusion and occlusion by other objects. In this paper we propose a novel approach to pedestrian detection in 3D range data based on supervised learning techniques to create a bank of classifiers for di erent height levels of the human body. In particular, our approach applies AdaBoost to train a strong classifier from geometrical and statistical features of groups of neighboring points at the same height. In a second step, the AdaBoost classifiers mutually enforce their evidence across di erent heights by voting into a continuous space. Pedestrians are finally found e ciently by mean-shift search for local maxima in the voting space. Experimental results carried out with 3D laser range data illustrate the robustness and e ciency of our approach even in cluttered urban environments. The learned people detector reaches a classification rate up to 96% from a single 3D scan.

ICRA Conference 2010 Conference Paper

Inferring the semantics of direction signs in public places

• Jérôme Maye
• Luciano Spinello
• Rudolph Triebel
• Roland Siegwart

Most large-scale public environments provide direction signs to facilitate the orientation for humans and to find their way to a goal location in the environment. Thus, for a robot operating in the same environment, it would be beneficial to interpret such signs correctly for a safe and efficient navigation. In this work, we propose a novel approach to infer the meaning of direction signs and to use that for navigation, i. e. , to find a mapping of a detected sign to a motion direction. Our method uses a hierarchical extension of the Implicit Shape Model framework called HISM that does not require any hand-labeled training data to detect the signs. On the lower level of this two-stage hierarchy, ISM is applied to image descriptors as in the standard approach. On the higher level, ISM operates on subparts of signs called tokens, using weights learned from data. The interpretation of the signs is inferred by associating navigation data to direction instructions. We conducted experiments from image data acquired in an airport terminal, aiming towards the implementation of a robotic guide, with promising results.

ICRA Conference 2010 Conference Paper

Unsupervised discovery of repetitive objects

• Jiwon Shin
• Rudolph Triebel
• Roland Siegwart

We present a novel approach for unsupervised discovery of repetitive objects from 3D point clouds. Our method assumes that objects are non-deformable and uses multiple occurrences of an object as the evidence for its existence. We segment input range data by superpixel segmentation and extract features for each segment. We search for a group of segments where each segment matches a segment in another group using a joint compatibility test. The discovered objects are then verified by the Iterative Closest Point algorithm to remove false matches. The presented method was tested on real data of complex objects. The experiments demonstrate that the proposed approach is capable of finding objects that occur multiple times in a scene and distinguish apart those objects of different types.

IROS Conference 2009 Conference Paper

Detecting pedestrians at very small scales

• Luciano Spinello
• A. Macho
• Rudolph Triebel
• Roland Siegwart

This paper presents a novel image based detection method for pedestrians at very small scales (between 16 × 20 and 32 × 40). We propose a set of new distinctive image features based on collections of local image gradients grouped by a superpixel segmentation. Features are collected and classified using AdaBoost. The positive classified features then vote for potential hypotheses that are collected using a mean shift mode estimation approach. The presented method overcomes the common limitations of a sliding window approach as well as those of standard voting approaches based on interest points. Extensive tests have been produced on a dataset with more than 20000 images showing the potential of this approach.

IROS Conference 2008 Conference Paper

Multimodal detection and tracking of pedestrians in urban environments with explicit ground plane extraction

• Luciano Spinello
• Rudolph Triebel
• Roland Siegwart

This paper presents a novel people detection and tracking method based on a combined multimodal sensor approach that utilizes 2D and 3D laser range and camera data. Laser data points are clustered and classified with a set of geometrical features using an SVM AdaBoost method. The clusters define a region of interest in the image that is adjusted using the ground plane information extracted from the 3D laser. In this areas a novel vision based people detector based on Implicit Shape Model (ISM) is applied. Each detected person is tracked using a greedy data association technique and multiple Extended Kalman Filters that use different motion models. This way, the filter can cope with a variety of different motion patterns. The tracker is asynchronously updated by the detections from the laser and the camera data. Experiments conducted in real-world outdoor scenarios with crowds of pedestrians demonstrate the usefulness of our approach.

IJCAI Conference 2007 Conference Paper

• Rudolph Triebel
• Richard Schmidt
• oacute; scar Mart
• iacute; nez Mozos
• Wolfram Burgard

In this paper, we present an algorithm to identify different types of objects from 2D and 3D laser range data. Our method is a combination of an instance-based feature extraction similar to the Nearest-Neighbor classifier (NN) and a collective classification method that utilizes associative Markov networks (AMNs). Compared to previous approaches, we transform the feature vectors so that they are better separable by linear hyper-planes, which are learned by the AMN classifier. We present results of extensive experiments in which we evaluate the performance of our algorithm on several recorded indoor scenes and compare it to the standard AMN approach as well as the NN classifier. The classification rate obtained with our algorithm substantially exceeds those of the AMN and the NN.

ICRA Conference 2007 Conference Paper

Towards Mapping of Cities

• Patrick Pfaff
• Rudolph Triebel
• Cyrill Stachniss
• Pierre Lamon
• Wolfram Burgard
• Roland Siegwart

Map learning is a fundamental task in mobile robotics because maps are required for a series of high level applications. In this paper, we address the problem of building maps of large-scale areas like villages or small cities. We present our modified car-like robot which we use to acquire the data about the environment. We introduce our localization system which is based on an information filter and is able to merge the information obtained by different sensors. We furthermore describe out mapping technique that is able to compactly model three-dimensional scenes and allows us efficient and accurate incremental map learning. We additionally apply a global optimization techniques in order to accurately close loops in the environment. Our approach has been implemented and deeply tested on a real car equipped with a series of sensors. Experiments described in this paper illustrate the accuracy and efficiency of the presented techniques.

IROS Conference 2006 Conference Paper

Multi-Level Surface Maps for Outdoor Terrain Mapping and Loop Closing

• Rudolph Triebel
• Patrick Pfaff
• Wolfram Burgard

To operate outdoors or on non-flat surfaces, mobile robots need appropriate data structures that provide a compact representation of the environment and at the same time support important tasks such as path planning and localization. One such representation that has been frequently used in the past are elevation maps which store in each cell of a discrete grid the height of the surface in the corresponding area. Whereas elevation maps provide a compact representation, they lack the ability to represent vertical structures or even multiple levels. In this paper, we propose a new representation denoted as multi-level surface maps (MLS maps). Our approach allows to store multiple surfaces in each cell of the grid. This enables a mobile robot to model environments with structures like bridges, underpasses, buildings or mines. Additionally, they allow to represent vertical structures. Throughout this paper we present algorithms for updating these maps based on sensory input, to match maps calculated from two different scans, and to solve the loop-closing problem given such maps. Experiments carried out with a real robot in an outdoor environment demonstrate that our approach is well-suited for representing large-scale outdoor environments

ICRA Conference 2006 Conference Paper

Robust 3D Scan Point Classification using Associative Markov Networks

• Rudolph Triebel
• Kristian Kersting
• Wolfram Burgard

In this paper we present an efficient technique to learn associative Markov networks (AMNs) for the segmentation of 3D scan data. Our technique is an extension of the work recently presented by Anguelov et al. (2005), in which AMNs are applied and the learning is done using max-margin optimization. In this paper we show that by adaptively reducing the training data, the training process can be performed much more efficiently while still achieving good classification results. The reduction is obtained by utilizing kd-trees and pruning them appropriately. Our algorithm does not require any additional parameters and yields an abstraction of the training data. In experiments with real data collected from a mobile outdoor robot we demonstrate that our approach yields accurate segmentations

IROS Conference 2005 Conference Paper

Improving plane extraction from 3D data by fusing laser data and vision

• Henrik Andreasson
• Rudolph Triebel
• Wolfram Burgard

The problem of extracting three-dimensional structures from data acquired with mobile robots has received considerable attention over the past years. Robots that are able to perceive their three-dimensional environment are envisioned to more robustly perform tasks like navigation, rescue, and manipulation. In this paper we present an approach that simultaneously uses color and range information to cluster 3D points into planar structures. Our current system also is able to calibrate the camera and the laser based on the remission values provided by the range scanner and the brightness of the pixels in the image. It has been implemented on a mobile robot equipped with a manipulator that carries a range scanner and a camera for acquiring colored range scans. Several experiments carried out on real data and in simulations demonstrate that our approach yields highly accurate results also in comparison with previous approaches.

ICRA Conference 2005 Conference Paper

Using Hierarchical EM to Extract Planes from 3D Range Scans

• Rudolph Triebel
• Wolfram Burgard
• Frank Dellaert

Recently, the acquisition of three-dimensional maps has become more and more popular. This is motivated by the fact that robots act in the three-dimensional world and several tasks such as path planning or localizing objects can be carried out more reliable using three-dimensional representations. In this paper we consider the problem of extracting planes from three-dimensional range data. In contrast to previous approaches our algorithm uses a hierarchical variant of the popular Expectation Maximization (EM) algorithm [1] to simultaneously learn the main directions of the planar structures. These main directions are then used to correct the position and orientation of planes. In practical experiments carried out with real data and in simulations we demonstrate that our algorithm can accurately extract planes and their orientation from range data.

ICRA Conference 2003 Conference Paper

A system for volumetric robotic mapping of abandoned mines

• Sebastian Thrun
• Dirk Hähnel
• Dave Ferguson 0001
• Michael Montemerlo
• Rudolph Triebel
• Wolfram Burgard
• Christopher R. Baker
• Zachary Omohundro

This paper describes two robotic systems developed for acquiring accurate volumetric maps of underground mines. One system is based on a cart instrumented by laser range finders, pushed through a mine by people. Another is a remotely controlled mobile robot equipped with laser range finders. To build consistent maps of large mines with many cycles, we describe an algorithm for estimating global correspondences and aligning robot paths. This algorithm enables us to recover consistent maps several hundreds of meters in diameter, without odometric information. We report results obtained in two mines, a research mine in Bruceton, PA, and an abandoned coal mine in Burgettstown, PA.

ICRA Conference 2003 Conference Paper

Map building with mobile robots in dynamic environments

• Dirk Hähnel
• Rudolph Triebel
• Wolfram Burgard
• Sebastian Thrun

The problem of generating maps with mobile robots has received considerable attention over the past years. Most of the techniques developed so far have been designed for situations in which the environment is static during the mapping process. Dynamic objects, however, can lead to serious errors in the resulting maps such as spurious objects or misalignments due to localization errors. In this paper we consider the problem of creating maps with mobile robots in dynamic environments. We present a new approach that interleaves mapping and localization with a probabilistic technique to identify spurious measurements. In several experiments we demonstrate that our algorithm generates accurate 2D and 3D in different kinds of dynamic indoor and outdoor environments. We also use our algorithm to isolate the dynamic objects and generate 3D representation of them.