Author name cluster

Stephan Weiss 0002

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

53 papers

1 author row

IROS Conference 2025 Conference Paper

Consistent Pose Estimation of Unmanned Ground Vehicles through Terrain-Aided Multi-Sensor Fusion on Geometric Manifolds

Alexander Raab
Stephan Weiss 0002
Alessandro Fornasier
Christian Brommer
Abdalrahman Ibrahim

Aiming to enhance the consistency and thus long-term accuracy of Extended Kalman Filters for terrestrial vehicle localization, this paper introduces the Manifold Error State Extended Kalman Filter (M-ESEKF). By representing the robot’s pose in a space with reduced dimensionality, the approach ensures feasible estimates on generic smooth surfaces, without introducing artificial constraints or simplifications that may degrade a filter’s performance. The accompanying measurement models are compatible with common loosely- and tightly-coupled sensor modalities and also implicitly account for the ground geometry. We extend the formulation by introducing a novel correction scheme that embeds additional domain knowledge into the sensor data, giving more accurate uncertainty approximations and further enhancing filter consistency. The proposed estimator is seamlessly integrated into a validated modular state estimation framework, demonstrating compatibility with existing implementations. Extensive Monte Carlo simulations across diverse scenarios and dynamic sensor configurations show that the M-ESEKF outperforms classical filter formulations in terms of consistency and stability. Moreover, it eliminates the need for scenario-specific parameter tuning, enabling its application in a variety of real-world settings.

IROS Conference 2025 Conference Paper

Learning Point Correspondences In Radar 3D Point Clouds For Radar-Inertial Odometry

Jan Michalczyk
Stephan Weiss 0002
Jan Steinbrener

Using 3D point clouds in odometry estimation in robotics often requires finding a set of correspondences between points in subsequent scans. While there are established methods for point clouds of sufficient quality, state-of-the-art still struggles when this quality drops. Thus, this paper presents a novel learning-based framework for predicting robust point correspondences between pairs of noisy, sparse and unstructured 3D point clouds from a light-weight, low-power, inexpensive, consumer-grade System-on-Chip (SoC) Frequency Modulated Continuous Wave (FMCW) radar sensor. Our network is based on the transformer architecture which allows leveraging the attention mechanism to discover pairs of points in consecutive scans with the greatest mutual affinity. The proposed network is trained in a self-supervised way using set-based multi-label classification cross-entropy loss, where the ground-truth set of matches is found by solving the Linear Sum Assignment (LSA) optimization problem, which avoids tedious hand annotation of the training data. Additionally, posing the loss calculation as multi-label classification permits supervising on point correspondences directly instead of on odometry error, which is not feasible for sparse and noisy data from the SoC radar we use. We evaluate our method with an open-source state-of-the-art Radar-Inertial Odometry (RIO) framework in real-world Unmanned Aerial Vehicle (UAV) flights and with the widely used public Coloradar dataset. Evaluation shows that the proposed method improves the position estimation accuracy by over 14 % and 19 % on average, respectively. The open source code and datasets can be found here: https://github.com/aau-cns/radar_transformer.

IROS Conference 2025 Conference Paper

Real-Time Initialization of Unknown Anchors for UWB-aided Navigation

Giulio Delama
Igor Borowski
Roland Jung
Stephan Weiss 0002

This paper presents a framework for the real-time initialization of unknown Ultra-Wideband (UWB) anchors in UWB-aided navigation systems. The method is designed for localization solutions where UWB modules act as supplementary sensors. Our approach enables the automatic detection and calibration of previously unknown anchors during operation, removing the need for manual setup. By combining an online Positional Dilution of Precision (PDOP) estimation, a lightweight outlier detection method, and an adaptive robust kernel for non-linear optimization, our approach significantly improves robustness and suitability for real-world applications compared to state-of-the-art. In particular, we show that our metric which triggers an initialization decision is more conservative than current ones commonly based on initial linear or non-linear initialization guesses. This allows for better initialization geometry and subsequently lower initialization errors. We demonstrate the proposed approach on two different mobile robots: an autonomous forklift and a quadcopter equipped with a UWB-aided Visual-Inertial Odometry (VIO) framework. The results highlight the effectiveness of the proposed method with robust initialization and low positioning error. We open-source our code in a C ++ library including a ROS wrapper.

IROS Conference 2024 Conference Paper

A Graph-Based Self-Calibration Technique for Cable-Driven Robots with Sagging Cable

M. R. Dindarloo
A. S. Mirjalili
S. A. Khalilpour
Rooholla Khorrambakht
Stephan Weiss 0002
Hamid D. Taghirad

The efficient operation of large-scale Cable-Driven Parallel Robots (CDPRs) relies on precise calibration of kinematic parameters and the simplicity of the calibration process. This paper presents a graph-based self-calibration framework that explicitly addresses cable sag effects and facilitates the calibration procedure for large-scale CDPRs by only relying on internal sensors. A unified factor graph is proposed, incorporating a catenary cable model to capture cable sagging. The factor graph iteratively refines kinematic parameters, including anchor point locations and initial cable length, by considering jointly onboard sensor data and the robot’s kineto-static model. The applicability and accuracy of the proposed technique are demonstrated through Finite Element (FE) simulations, on both large and small-scale CDPRs subjected to significant initialization perturbations.

ICRA Conference 2024 Conference Paper

An Equivariant Approach to Robust State Estimation for the ArduPilot Autopilot System

Alessandro Fornasier
Yixiao Ge
Pieter van Goor
Martin Scheiber
Andrew Tridgell
Robert E. Mahony
Stephan Weiss 0002

The majority of commercial and open-source autopilot software for uncrewed aerial vehicles rely on the tried and tested extended Kalman filter (EKF) to provide the state estimation solution for the inertial navigation system (INS). While modern implementations achieve remarkable robustness, it is often due to the careful implementation of exception code for a multitude of corner cases along with significant skilled tuning effort. In this paper, we use the data wealth of the ArduPilot community to identify and highlight the most common real-world challenges in INS state estimation, including sensor self-calibration, robustness in static conditions, global navigation satellite system (GNSS) outliers and shifts, and robustness to faulty inertial measurement units (IMUs). We propose a novel equivariant filter (EqF) formulation for the INS solution that exploits a Semi-Direct-Bias symmetry group for multi-sensor fusion with self-calibration capabilities and incorporates equivariant velocity-type measurements. We augment the filter with a simple innovation-covariance inflation strategy that seamlessly handles GNSS outliers and shifts without requiring coding of a whole set of exception cases. We use real-world data from the Ardupilot community to demonstrate the performance of the proposed filter on known cases where existing filters fail without careful exception handling or case-specific tuning and benchmark against the ArduPilot’s EKF3, the most sophisticated EKF implementation currently available.

IROS Conference 2024 Conference Paper

Modular Meshed Ultra-Wideband Aided Inertial Navigation with Robust Anchor Calibration

Roland Jung
Luca Santoro
Davide Brunelli
Daniele Fontanelli
Stephan Weiss 0002

This paper introduces a generic filter-based state estimation framework that supports two state-decoupling strategies based on cross-covariance factorization. These strategies reduce the computational complexity and inherently support true modularity – a perquisite for handling and processing meshed range measurements among a time-varying set of devices. In order to utilize these measurements in the estimation framework, positions of newly detected stationary devices (anchors) and the pairwise biases between the ranging devices are required. In this work an autonomous calibration procedure for new anchors is presented, that utilizes range measurements from multiple tags as well as already known anchors. To improve the robustness, an outlier rejection method is introduced. After the calibration is performed, the sensor fusion framework obtains initial beliefs of the anchor positions and dictionaries of pairwise biases, in order to fuse range measurements obtained from new anchors tightly-coupled. The effectiveness of the filter and calibration framework has been validated through evaluations on a recorded dataset and real-world experiments.

IROS Conference 2024 Conference Paper

Tightly-Coupled Factor Graph Formulation For Radar-Inertial Odometry

Jan Michalczyk
Julius Quell
Florian Steidle
Marcus Gerhard Müller
Stephan Weiss 0002

In this paper, we present a Radar-Inertial Odometry (RIO) method based on the nonlinear optimization of factor graphs in a sliding window fashion. Our method makes use of a light-weight, low-power, inexpensive and commonly available hardware enabling easy deployment on small Unmanned Aerial Vehicles (UAV)s. We keep the state estimation problem bounded by employing partial marginalization of the oldest states, rendering the method real-time capable. We compare the implemented approach to the state-of-the-art multi-state Extended Kalman Filter (EKF)-based method in a one-to-one fashion. That is, we implemented in a single custom C++ RIO framework both estimation back-ends with all other parts shared and thus identical for a fair direct comparison. In the real-world flight experiments, we compare the two methods and show that both perform similarly in terms of accuracy when the linearization point is not far from the true state. Upon wrong initialization, the factor graph approach heavily outperforms the EKF approach. We also acknowledge that the influence of undetected outliers can overwhelm the inherent benefits of the nonlinear optimization approach leading to the insight that the estimator front-end has an important (and often underestimated) role in the overall performance. The open source code and datasets can be found here: https://github.com/aau-cns/aaucns_rio.

ICRA Conference 2023 Conference Paper

AI-Based Multi-Object Relative State Estimation with Self-Calibration Capabilities

Thomas Jantos
Christian Brommer
Eren Allak
Stephan Weiss 0002
Jan Steinbrener

The capability to extract task specific, semantic information from raw sensory data is a crucial requirement for many applications of mobile robotics. Autonomous inspection of critical infrastructure with Unmanned Aerial Vehicles (UAVs), for example, requires precise navigation relative to the structure that is to be inspected. Recently, Artificial Intelligence (AI)-based methods have been shown to excel at extracting semantic information such as 6 degree-of-freedom (6-DoF) poses of objects from images. In this paper, we propose a method combining a state-of-the-art AI-based pose estimator for objects in camera images with data from an inertial measurement unit (IMU) for 6-DoF multi-object relative state estimation of a mobile robot. The AI-based pose estimator detects multiple objects of interest in camera images along with their relative poses. These measurements are fused with IMU data in a state-of-the-art sensor fusion framework. We illustrate the feasibility of our proposed method with real world experiments for different trajectories and number of arbitrarily placed objects. We show that the results can be reliably reproduced due to the self-calibrating capabilities of our approach.

IROS Conference 2023 Conference Paper

FUSE-D: Framework for UAV System-Parameter Estimation with Disturbance Detection

Christoph Böhm 0004
Stephan Weiss 0002

Modern unmanned aerial vehicles (UAVs) with sophisticated mechanics ask for extended online system identification to aid model-based controls in task execution. In addition, UAVs in adverse environmental conditions require a more detailed environmental disturbance understanding. The necessary combination of online system identification, sensor suite self-calibration, and external disturbance analysis to tackle these issues holistically is currently an open issue. Our proposed FUSE-D approach combines these elements based on a system model at the rotor-speed level and a single global pose sensor (e. g. , a tracking system like Optitrack). Besides sensor intrinsics and extrinsics, the framework allows estimating the UAV's rotor geometry, mass, moments of inertia, and the rotors' aerodynamic properties, as well as an external force and where it acts on the UAV. The general formulation allows us to extend the approach to an N-rotor (multi-rotor) UAV and classify the type of external disturbance. We perform a detailed non-linear observability analysis for the 43 + 7N states and do a statistically relevant embedded hardware-in-the-loop performance analysis in the realistic simulation environment Gazebo with RotorS.

IROS Conference 2023 Conference Paper

Graph-Based Visual-Kinematic Fusion and Monte Carlo Initialization for Fast-Deployable Cable-Driven Robots

Rooholla Khorrambakht
Hamed Damirchi
M. R. Dindarloo
A. Saki
S. A. Khalilpour
Hamid D. Taghirad
Stephan Weiss 0002

Ease of calibration and high-accuracy task-space state-estimation purely based on onboard sensors is a key requirement for enabling easily deployable cable robots in real-world applications. In this work, we incorporate the onboard camera and kinematic sensors to drive a statistical fusion framework that presents a unified localization and calibration system which requires no initial values for the kinematic parameters. This is achieved by formulating a Monte-Carlo algorithm that initializes a factor-graph representation of the calibration and localization problem. With this, we are able to jointly identify both the kinematic parameters and the visual odometry scale alongside their corresponding uncertainties. We demonstrate the practical applicability of the framework using our state-estimation dataset recorded with the ARAS-CAM suspended cable driven parallel robot, and published as part of this manuscript.

ICRA Conference 2023 Conference Paper

Multi-State Tightly-Coupled EKF-Based Radar-Inertial Odometry With Persistent Landmarks

Jan Michalczyk
Roland Jung
Christian Brommer
Stephan Weiss 0002

In this paper, we present a Radar-Inertial Odometry (RIO) approach that utilizes performance improving modules, enhanced for the sparse and noisy radar signals, from the vision community in order to estimate the full 6DoF pose and 3D velocity of a robot in an unprepared environment. Our method leverages a multi-state approach in which we make use of several past robot poses and trails of measurements from a lightweight and inexpensive Frequency Modulated Continuous Wave (FMCW) radar sensor. Furthermore, in our estimation framework we include a method for promoting measurement trails to persistent landmarks which correspond to salient features in the environment. In an Extended Kalman Filter (EKF) framework, we fuse the range measurements to the persistent landmarks, trails, and the velocity measurements of the detected 3D points together with the Inertial Measurement Unit (IMU) readings. Our method is particularly relevant for (but not limited to) Unmanned Aerial Vehicles (UAV), enabling them to localize while performing missions in Global Navigation Satellite System (GNSS)-denied environments and, thanks to the properties of the radar sensor, in environments generally challenging for robot perception due to external factors such as smoke or extreme illumination. We show in real flight experiments the effectiveness of our estimator and compare it to the state-of-the-art.

IROS Conference 2023 Conference Paper

UVIO: An UWB-Aided Visual-Inertial Odometry Framework with Bias-Compensated Anchors Initialization

Giulio Delama
Farhad Shamsfakhr
Stephan Weiss 0002
Daniele Fontanelli
Alessandro Fornasier

This paper introduces UVIO, a multi-sensor framework that leverages Ultra Wide Band (UWB) technology and Visual-Inertial Odometry (VIO) to provide robust and low-drift localization. In order to include range measurements in state estimation, the position of the UWB anchors must be known. This study proposes a multi-step initialization procedure to map multiple unknown anchors by an Unmanned Aerial Vehicle (UAV), in a fully autonomous fashion. To address the limitations of initializing UWB anchors via a random trajectory, this paper uses the Geometric Dilution of Precision (GDOP) as a measure of optimality in anchor position estimation, to compute a set of optimal waypoints and synthesize a trajectory that minimizes the mapping uncertainty. After the initialization is complete, the range measurements from multiple anchors, including measurement biases, are tightly integrated into the VIO system. While in range of the initialized anchors, the VIO drift in position and heading is eliminated. The effectiveness of UVIO and our initialization procedure has been validated through a series of simulations and real-world experiments.

IROS Conference 2022 Conference Paper

Autonomous Control of Redundant Hydraulic Manipulator Using Reinforcement Learning with Action Feedback

Rohit Dhakate
Christian Brommer
Christoph Böhm 0004
Harald Gietler
Stephan Weiss 0002
Jan Steinbrener

This article presents an entirely data-driven approach for autonomous control of redundant manipulators with hydraulic actuation. The approach only requires minimal system information, which is inherited from a simulation model. The non-linear hydraulic actuation dynamics are modeled using actuator networks from the data gathered during the manual operation of the manipulator to effectively emulate the real system in a simulation environment. A neural network control policy for autonomous control, based on end-effector (EE) position tracking is then learned using Reinforcement Learning (RL) with Ornstein-Uhlenbeck process noise (OUNoise) for efficient exploration. The RL agent also receives feedback based on supervised learning of the forward kinematics which facilitates selecting the best suitable action from exploration. The control policy directly provides the joint variables as outputs based on provided target EE position while taking into account the system dynamics. The joint variables are then mapped to the hydraulic valve commands, which are then fed to the system without further modifications. The proposed approach is implemented on a scaled hydraulic forwarder crane with three revolute and one prismatic joint to track the desired position of the EE in 3-Dimensional (3D) space. With the emulated dynamics and extensive learning in simulation, the results demonstrate the feasibility of deploying the learned controller directly on the real system.

IROS Conference 2022 Conference Paper

Centralized-Equivalent Pairwise Estimation with Asynchronous Communication Constraints for two Robots

Eren Allak
Axel Barrau
Roland Jung
Jan Steinbrener
Stephan Weiss 0002

Collaboratively estimating the state of two robots under communication constraints is challenging regarding computational complexity and statistical optimality. Previous work only achieves practical solutions by either disregarding parts of the measurements or imposing a communication overhead, being non-optimal or not entirely distributed, respectively. In this work, we present a centralized-equivalent but dis-tributed approach for pairwise state estimation where two agents only communicate when they meet. Our approach utilizes elements from wave scattering theory to efficiently and consistently summarize (pre-compute) past estimator information (i. e. , state evolution and uncertainty) between encounters of two agents. This summarized information is then used in a joint correction step taking into account all past information of each agent in a statistically correct way. This novel approach enables us to distribute the pre-computations of both state evolution and uncertainties on the agents and reconstruct the centralized-equivalent system estimate with very few computations once the agents meet again while still applying all measurements from both agents on both estimates upon encounter. We compare our approach on a real-world dataset against a state of the art collaborative state estimation approach.

ICRA Conference 2022 Conference Paper

COP: Control & Observability-aware Planning

Christoph Böhm 0004
Pascal Brault
Quentin Delamare
Paolo Robuffo Giordano
Stephan Weiss 0002

In this research, we aim to answer the question: How to combine Closed-Loop State and Input Sensitivity-based with Observability-aware trajectory planning? These possibly op-posite optimization objectives can be used to improve trajectory control tracking and, at the same time, estimation performance. Our proposed novel Control & Observability-aware Planning (COP) framework is the first that uses these possibly opposing objectives in a Single-Objective Optimization Problem (SOOP) based on the Augmented Weighted Tchebycheff method to perform the balancing of them and generation of Bézier curve-based trajectories. Statistically relevant simulations for a 3D quadrotor unmanned aerial vehicle (UAV) case study produce results that support our claims and show the negative correlation between both objectives. We were able to reduce the positional mean integral error norm as well as the estimation uncertainty with the same trajectory to comparable levels of the trajectories optimized with individual objectives.

ICRA Conference 2022 Conference Paper

Equivariant Filter Design for Inertial Navigation Systems with Input Measurement Biases

Alessandro Fornasier
Yonhon Ng
Robert E. Mahony
Stephan Weiss 0002

Inertial Navigation Systems (INS) are a key technology for autonomous vehicles applications. Recent advances in estimation and filter design for the INS problem have exploited geometry and symmetry to overcome limitations of the classical Extended Kalman Filter (EKF) approach that formed the mainstay of INS systems since the mid-twentieth century. The industry standard INS filter, the Multiplicative Extended Kalman Filter (MEKF), uses a geometric construction for attitude estimation coupled with classical Euclidean construction for position, velocity and bias estimation. The recent Invariant Extended Kalman Filter (IEKF) provides a geometric framework for the full navigation states, integrating attitude, position and velocity, but still uses the classical Euclidean construction to model the bias states. In this paper, we use the recently proposed Equivariant Filter (EqF) framework to derive a novel observer for biased inertial-based navigation in a fully geometric framework. The introduction of virtual velocity inputs with associated virtual bias leads to a full equivariant symmetry on the augmented system. The resulting filter performance is evaluated with both simulated and real-world data, and demonstrates increased robustness to a wide range of erroneous initial conditions, and improved accuracy when compared with the industry standard Multiplicative EKF (MEKF) approach.

ICRA Conference 2022 Conference Paper

Improved State Propagation through AI-based Pre-processing and Down-sampling of High-Speed Inertial Data

Jan Steinbrener
Christian Brommer
Thomas Jantos
Alessandro Fornasier
Stephan Weiss 0002

We present a novel approach to improve 6 degree-of-freedom state propagation for unmanned aerial vehicles in a classical filter through pre-processing of high-speed inertial data with AI algorithms. We evaluate both an LSTM-based approach as well as a Transformer encoder architecture. Both algorithms take as input short sequences of fixed length N of high-rate inertial data provided by an inertial measurement unit (IMU) and are trained to predict in turn one pre-processed IMU sample that minimizes the state propagation error of a classical filter across M sequences. This setup allows us to provide sufficient temporal history to the networks for good performance while maintaining a high propagation rate of pre-processed IMU samples important for later deployment on real-world systems. In addition, our network architectures are formulated to directly accept input data at variable rates thus minimizing necessary data preprocessing. The results indicate that the LSTM based architecture outperforms the Transformer encoder architecture and significantly improves the propagation error even for long IMU propagation times.

IROS Conference 2022 Conference Paper

Kinematics-Inertial Fusion for Localization of a 4-Cable Underactuated Suspended Robot Considering Cable Sag

Eren Allak
Rooholla Khorrambakht
Christian Brommer
Stephan Weiss 0002

Suspended Cable-Driven Parallel Robots (SCDPR) have intriguing capabilities on large scales but still have open challenges in precisely estimating the end-effector pose. The cables exhibit a downward curved shape, also known as cable sag which needs to be accounted for in the pose estimation. The catenary equations can accurately describe this phenomenon but are only accurate in equilibrium conditions. Thus, pose estimation for large-scale SCDPR in dynamic motion is an open challenge. This work proposes a real-time pose estimation algorithm for dynamic trajectories of SCDPRs, which is accurate over large areas. We present a novel approach that considers cable sag to reduce the estimation error for large scales while also employing an Inertial Measurement Unit (IMU) to improve estimation accuracy for dynamic motion. Our approach reduces the RMSE to less than a third compared to standard methods not considering cable sag. Similarly, the inclusion of the IMU reduces the RMSE in dynamic situations by 40% compared to non-IMU aided approaches considering cable sag. Further-more, we evaluate our Extended Kalman Filter (EKF) based algorithm on a real system with ground truth pose information.

IROS Conference 2022 Conference Paper

Scalable and Modular Ultra-Wideband Aided Inertial Navigation

Roland Jung
Stephan Weiss 0002

Navigating accurately in potentially GPS-denied environments is a perquisite of autonomous systems. Relative localization based on ultra-wideband (UWB) is - especially indoors - a promising technology. In this paper, we present a probabilistic filter based Modular Multi-Sensor Fusion (MMSF) approach with the capability of using efficiently all information in a fully meshed UWB ranging network. This allows an accurate mobile agent state estimation and the calibration of the ranging network's spatial constellation. We advocate a new paradigm that includes elements from Collaborative State Estimation (CSE) and allows us considering all stationary UWB anchors and the mobile agent as a decentralized set of estimtors/filters. With this, our method can include all meshed (inter-)sensor observations tightly coupled in a modular estimator. We show that the application of our CSE-inspired method in such a context breaks the computational barrier. Otherwise, it would, for the sakeof complexity-reduction, prohibit the use of all available information or would lead to significant estimator inconsistencies due to coarse approximations. We compare the proposed approach against different MMSF strategies in terms of execution time, accuracy, and filter credibility on both synthetic data and on a dataset from real Unmanned Aerial Vehicles (UAVs).

IROS Conference 2022 Conference Paper

Tightly-Coupled EKF-Based Radar-Inertial Odometry

Jan Michalczyk
Roland Jung
Stephan Weiss 0002

Multicopter Unmanned Aerial Vehicles (UAV) are small and agile robots with the potential to become prominent in performing autonomous tasks in various Global Navigation Satellite System (GNSS)-denied environments. These environments can potentially be rendered even more challenging due to external factors impairing the robot's perception, such as low or too bright light, permeation with aerosols or smoke. A precondition of autonomous operation, though, is the ability of a robot to accurately localize itself in the surrounding environment. Millimeter-wave Frequency Modulated Continuous Wave (FMCW) radar sensors are resilient to the aforementioned factors while being lightweight, inexpensive and highly accurate. In this paper, we present a Radar-Inertial Odometry (RIO) method for estimating the full 6DoF pose and 3D velocity of a UAV. In an Extended Kalman Filter (EKF) framework, we fuse range measurements and velocity measurements of 3D points detected by an FMCW radar sensor together with Inertial Measurement Unit (IMU) readings. In real experiments we show that our approach enables accurate state estimation of a UAV and that it exhibits improvements over similar existing state-of-the-art method.

ICRA Conference 2021 Conference Paper

Bias Compensated UWB Anchor Initialization using Information-Theoretic Supported Triangulation Points

Julian Blueml
Alessandro Fornasier
Stephan Weiss 0002

For Ultra-Wide-Band (UWB) based navigation, an accurate initialization of the anchors in a reference coordinate system is crucial for precise subsequent UWB-inertial based pose estimation. This paper presents a strategy based on information theory to initialize such UWB anchors using raw distance measurements from tag to anchor(s) and aerial vehicle poses. We include a linear distance-dependent bias term and an offset in our estimation process in order to achieve unprecedented accuracy in the 3D position estimates of the anchors (error reduction by a factor of about 3. 5 compared to current approaches) without the need of prior knowledge. After an initial coarse position triangulation of the anchors using random vehicle positions, a bounding volume is created in the vicinity of the roughly estimated anchor position. In this volume, we calculate points which provide the maximal triangulation related information based on the Fisher Information Theory. Using these information theoretic optimal points, a fine triangulation is done including bias term estimation. We evaluate our approach in simulations with realistic sensor noise as well as with real world experiments. We also fly an aerial vehicle with UWB-inertial based closed loop control demonstrating that precise anchor initialization does improve navigation precision. Our initialization approach is compared to state-of-the-art as well as to an initialization without the simultaneous bias estimation.

ICRA Conference 2021 Conference Paper

Combined System Identification and State Estimation for a Quadrotor UAV

Christoph Böhm 0004
Christian Brommer
Alexander Hardt-Stremayr
Stephan Weiss 0002

Precise system identification is an important aspect of adequate control design and parameter definition to allow for accurate and reliable navigation. While this is well known in robotics, the community working with small rotorcraft Unmanned Aerial Vehicles (UAVs) has yet to discover the benefits. In contrast to existing work, which often performs offline or deterministic (i. e. closed-form) system identification, we present a probabilistic approach to the online estimation of system identification parameters and self-calibration states. Instead of decoupling system identification and state estimation for vehicle control, we merge the entire process into a holistic probabilistic framework to allow self-awareness and self-healing. Our observability analysis shows that most of the system identification parameters are observable and converge quickly to the optimal value using a combination of inertial cues, dynamic modeling, and an additional exteroceptive sensor. We support our theoretical findings with extensive tests simulating realistic data in Gazebo.

ICRA Conference 2021 Conference Paper

Consistent State Estimation on Manifolds for Autonomous Metal Structure Inspection

Bryan Starbuck
Alessandro Fornasier
Stephan Weiss 0002
Cédric Pradalier

This work presents the Manifold Invariant Extended Kalman Filter, a novel approach for better consistency and accuracy in state estimation on manifolds. The robustness of this filter allows for techniques with high noise potential like ultra-wideband localization to be used for a wider variety of applications like autonomous metal structure inspection. The filter is derived and its performance is evaluated by testing it on two different manifolds: a cylindrical one and a bivariate b-spline representation of a real vessel surface, showing its flexibility to being used on different types of surfaces. Its comparison with a standard EKF that uses virtual, noise-free measurements as manifold constraints proves that it outperforms standard approaches in consistency and accuracy. Further, an experiment using a real magnetic crawler robot on a curved metal surface with ultra-wideband localization shows that the proposed approach is viable in the real world application of autonomous metal structure inspection.

ICRA Conference 2021 Conference Paper

Mid-Air Range-Visual-Inertial Estimator Initialization for Micro Air Vehicles

Martin Scheiber
Jeff Delaune
Stephan Weiss 0002
Roland Brockers

Monocular Visual-Inertial Odometry (VIO) has become ubiquitous for navigation of autonomous Micro Air Vehicles (MAVs). Yet, state-of-the-art VIO is still very failure-prone, which can have dramatic consequences. To prevent this, VIO must be able to re-initialize in mid-air, either during a free fall or on a constant velocity trajectory after attitude control has been re-established. However, for both of these trajectories, the visual scale cannot be observed with VIO batch initializers because of the absence of acceleration change. We propose to use a small and lightweight laser-range finder (LRF) and a scene facet model to initialize vision-based navigation at the right scale under any motion condition and over any scene structure. This new range constraint is integrated into a visual-inertial bundle-adjustment initializer. We evaluate our approach in simulation, including robustness to various parameters, and demonstrate on real data how this approach can address midair state estimation failure in real-time.

ICRA Conference 2021 Conference Paper

Scalable Recursive Distributed Collaborative State Estimation for Aided Inertial Navigation

Roland Jung
Stephan Weiss 0002

This paper presents a novel approach to recover outdated cross-covariance between correlated agents at the moment they perform joint observations. This allows to render Collaborative State Estimation (CSE) fully distributed, with communication only required for the moment of joint observation and most importantly, it significantly reduces the maintenance effort in case of high frequent propagation sensors. These properties make the approach suitable to a wide range of multi-robot applications. In our evaluation on a Quaternion-based Error-State Extended Kalman Filter (Q-ESEKF) using an Inertial Measurement Unit (IMU) as propagation sensor at a rate of 200Hz, we showed a significant speedup against our previous approach for maintaining a couple of interdependence. We compared the approach in total against four different approaches on both, a simulation and on a real-world dataset for Micro Aerial Vehicles (MAVs). Video: https://youtu.be/xkljfwbhMP0

ICRA Conference 2021 Conference Paper

Time and Energy Optimized Trajectory Generation for Multi-Agent Constellation Changes

Paul Ladinig
Bernhard Rinner
Stephan Weiss 0002

Planning the simultaneous movement of multiple agents represents a challenging coordination problem, and ideally safety and efficiency are jointly addressed. This paper introduces a planning algorithm for fast and energy-efficient trajectories with reduced collision potential from a start to an end constellation. This new approach combines trajectory approximation based on model predictive control, collision avoidance with potential fields, and flight energy optimization with minimum snap trajectories. Our approach results in unprecedented transition times and success rates with less energy consumption, as shown in simulation and real experiments with 16 drones.

ICRA Conference 2021 Conference Paper

VINSEval: Evaluation Framework for Unified Testing of Consistency and Robustness of Visual-Inertial Navigation System Algorithms

Alessandro Fornasier
Martin Scheiber
Alexander Hardt-Stremayr
Roland Jung
Stephan Weiss 0002

The research community presented significant advances in many different Visual-Inertial Navigation System (VINS) algorithms to localize mobile robots or hand-held devices in a 3D environment. While authors of the algorithms of-ten do compare to, at that time, existing competing approaches, their comparison methods, rigor, depth, and repeatability at later points in time have a large spread. Further, with existing simulators and photo-realistic frameworks, the user is not able to easily test the sensitivity of the algorithm under examination with respect to specific environmental conditions and sensor specifications. Rather, tests often include unwillingly many polluting effects falsifying the analysis and interpretations. In addition, edge cases and corresponding failure modes often remain undiscovered due to the limited breadth of the test sequences. Our unified evaluation framework allows, in a fully automated fashion, a reproducible analysis of different VINS methods with respect to specific environmental and sensor parameters. The analyses per parameter are done over a multitude of test sets to obtain both statistically valid results and an average over other, potentially polluting effects with respect to the one parameter under test to mitigate biased interpretations. The automated performance results per method over all tested parameters are then summarized in unified radar charts for a fair comparison across authors and institutions.

IROS Conference 2020 Conference Paper

Consistent Covariance Pre-Integration for Invariant Filters with Delayed Measurements

Eren Allak
Alessandro Fornasier
Stephan Weiss 0002

Sensor fusion systems merging (multiple) delayed sensor signals through a statistical approach are challenging setups, particularly for resource constrained platforms. For statistical consistency, one would be required to keep an appropriate history, apply the correcting signal at the given time stamp in the past, and re-apply all information received until the present time. This re-calculation becomes impractical (the bottleneck being the re-propagation of the covariance matrices for estimator consistency) for platforms with multiple sensors/states and low compute power. This work presents a novel approach for consistent covariance pre-integration allowing delayed sensor signals to be incorporated in a statistically consistent fashion with very low complexity. We leverage recent insights in Invariant Extended Kalman Filters (IEKF) and their log-linear, state independent error propagation together with insights from the scattering theory to mimic the re-calculation process as a medium through which we can propagate waves (covariance information in this case) in single operation steps. We support our findings in simulation and with real data.

ICRA Conference 2020 Conference Paper

Decentralized Collaborative State Estimation for Aided Inertial Navigation

Roland Jung
Christian Brommer
Stephan Weiss 0002

In this paper, we present a Quaternion-based Error-State Extended Kalman Filter (Q-ESEKF) based on IMU propagation with an extension for Collaborative State Estimation (CSE) and a communication complexity of O(1) (in terms of required communication links). Our approach combines a versatile filter formulation with the concept of CSE, allowing independent state estimation on each of the agents and at the same time leveraging and statistically maintaining interdependencies between agents, after joint measurements and communication (i. e. relative position measurements) occur. We discuss the development of the overall framework and the probabilistic (re-)initialization of the agent's states upon initial or recurring joint observations. Our approach is evaluated in a simulation framework on two prominent benchmark datasets in 3D.

ICRA Conference 2020 Conference Paper

Monocular Visual-Inertial Odometry in Low-Textured Environments with Smooth Gradients: A Fully Dense Direct Filtering Approach

Alexander Hardt-Stremayr
Stephan Weiss 0002

State of the art visual-inertial odometry approaches suffer from the requirement of high gradients and sufficient visual texture. Even direct photometric approaches select a subset of the image with high-gradient areas and ignore smooth gradients or generally low-textured areas. In this work, we show that taking all image information (i. e. every single pixel) enables visual-inertial odometry even on areas with very low texture and smooth gradients, inherently interpolating and estimating the scene with no texture based on its informative surrounding. This information propagation is only possible as we estimate all states and their uncertainties (robot pose, extrinsic sensor calibration, and scene depth) jointly in a fully dense filter framework. Our complexity reduction approach enables real-time execution despite the large size of the state vector. Compared to our previous basic feasibility study on this topic, this work includes higher order covariance propagation and improved state handling for a significant performance gain, thorough comparisons to state-of-the-art algorithms, larger mapping components with uncertainty, self-calibration capability, and real-data tests.

IROS Conference 2019 Conference Paper

Covariance Pre-Integration for Delayed Measurements in Multi-Sensor Fusion

Eren Allak
Roland Jung
Stephan Weiss 0002

Delay compensation in filter based sensor fusion frameworks for multiple sensors with varying delays and different rates quickly results in large computational overhead should the delayed measurements be incorporated in a statistically meaningful way. Even more so if high rate propagation sensors (e. g. IMU) are used. This work presents an approach to implement such frameworks with significant complexity reduction compared to standard implementations. We set particular focus on the state covariance propagation as this chain of re-computations (i. e. $F P F^{T}+Q$ per propagation step) upon a delayed update is the dominant bottleneck. We draw our inspiration from the scattering theory and propose a method which projects the idea of wave propagation to an efficient concatenation of covariance propagation steps between filter updates. Through this approach, we reach a speed-up of more than a factor of 10 for the covariance propagation and render the computational complexity independent of the number of propagation steps between filter updates. We evaluated our method in simulation and with real data.

ICRA Conference 2019 Conference Paper

Towards Fully Dense Direct Filter-Based Monocular Visual-Inertial Odometry

Alexander Hardt-Stremayr
Stephan Weiss 0002

We propose a fully dense direct filter-based visual-inertial odometry method estimating both pixel depth for all pixels and robot state simultaneously, having all uncertainties in the same state vector. Due to the fully dense method, our approach works even in low-textured areas with very low, smooth gradients (i. e. scenes where feature based or semi-dense approaches fail). Our algorithm performs in real-time on a CPU with a time complexity linearly dependent on the amount of pixels in the provided image. To achieve this, we propose complexity reduction methods for fast matrix inversion, exploiting specific structures of the covariance matrix. We provide both simulated and real-world results in low-textured areas with a smooth gradient.

IROS Conference 2019 Conference Paper

Visual-Inertial On-Board Throw-and-Go Initialization for Micro Air Vehicles

Martin Scheiber
Jeff Delaune
Roland Brockers
Stephan Weiss 0002

We propose an approach to the throw-and-go (TnG) problem for micro air vehicles (MAVs) using visual and inertial sensors. The key challenge is the fast on-board initialization of the visual odometry (VO) system, which usually requires user input to recover the visual scale. Our approach is based on the identification of the gravity vector from the acceleration data computed with images of the ground during in free fall. This enables scaling of the poses reconstructed with visual information. The proposed framework use inertial data to control the MAV attitude so the ground is visible after the throw. Using image to image homography a metric scale is estimated with which the MAV’s height is propagated. Unlike existing literature, this approach requires no additional sensor nor user input or pre-throw assumptions and can recover from any initial attitude. We show results on both simulation and real data.

IROS Conference 2018 Conference Paper

Key-Frame Strategy During Fast Image-Scale Changes and Zero Motion in VIO Without Persistent Features

Eren Allak
Alexander Hardt-Stremayr
Stephan Weiss 0002

Many of today's Visual-Inertial Odometry (VIO)frameworks work well under regular motion but have issues and need special treatment under special motion. Here, special does not imply bad or corrupted data but stands for increased difficulty to treat clean data. Common special motion for VIO are large feature displacement due to fast motion close to a scene and zero motion phases not providing sufficient baseline. In this paper we present a feature and frame selection approach which seamlessly handles all motion scenarios without the need of (error prone)motion case identification and subsequent case-specific heuristics. We further show that this approach allows to eliminate features in the state vector (persistent features)altogether while still being able to inherently handle zero motion phases. This reduces computational complexity while maintaining the ability to hover in place. We integrate our frame selection approach into our own VIO algorithm and compare its performance against three state-of-the-art algorithms with real data on a real platform. While our approach shows slightly higher global drift it is the only algorithm that can reliably estimate the pose over a large motion spectrum from fast scale change down to zero motion.

ICRA Conference 2016 Conference Paper

Self-calibrating multi-sensor fusion with probabilistic measurement validation for seamless sensor switching on a UAV

Karol Hausman
Stephan Weiss 0002
Roland Brockers
Larry H. Matthies
Gaurav S. Sukhatme

Fusing data from multiple sensors on-board a mobile platform can significantly augment its state estimation abilities and enable autonomous traversals of different domains by adapting to changing signal availabilities. However, due to the need for accurate calibration and initialization of the sensor ensemble as well as coping with erroneous measurements that are acquired at different rates with various delays, multi-sensor fusion still remains a challenge. In this paper, we introduce a novel multi-sensor fusion approach for agile aerial vehicles that allows for measurement validation and seamless switching between sensors based on statistical signal quality analysis. Moreover, it is capable of self-initialization of its extrinsic sensor states. These initialized states are maintained in the framework such that the system can continuously self-calibrate. We implement this framework on-board a small aerial vehicle and demonstrate the effectiveness of the above capabilities on real data. As an example, we fuse GPS data, ultra-wideband (UWB) range measurements, visual pose estimates, and IMU data. Our experiments demonstrate that our system is able to seamlessly filter and switch between different sensors modalities during run time.

IROS Conference 2015 Conference Paper

Detection and characterization of moving objects with aerial vehicles using inertial-optical flow

Daniel Meier
Roland Brockers
Larry H. Matthies
Roland Siegwart
Stephan Weiss 0002

In this paper, we present a novel approach in combining visual and inertial measurements in non-static environments for first order characterization of the metric motion of non-static objects in the scene. Our approach leverages online estimated ego motion states and uses a novel inertial-optical flow (IOF) measurement analysis to identify moving objects and to characterize them in their angular and linear velocities. The novelty of our algorithm lies in the identification and segmentation of consistent optical flow outliers in the so-called kinematic space. These consistent outliers in combination with the IOF information for ego-motion estimation yield a first order estimation of the moving object in full 3D and in metric units. The approach is highly efficient as it only requires matched features in two consecutive images. We evaluate and demonstrate our algorithm in simulations and in real world tests.

ICRA Conference 2014 Conference Paper

Stereo vision-based obstacle avoidance for micro air vehicles using disparity space

Larry H. Matthies
Roland Brockers
Yoshiaki Kuwata
Stephan Weiss 0002

We address obstacle avoidance for outdoor flight of micro air vehicles. The highly textured nature of outdoor scenes enables camera-based perception, which will scale to very small size, weight, and power with very wide, two-axis field of regard. In this paper, we use forward-looking stereo cameras for obstacle detection and a downward-looking camera as an input to state estimation. For obstacle representation, we use image space with the stereo disparity map itself. We show that a C-space-like obstacle expansion can be done with this representation and that collision checking can be done by projecting candidate 3-D trajectories into image space and performing a z-buffer-like operation with the disparity map. This approach is very efficient in memory and computing time. We do motion planning and trajectory generation with an adaptation of a closed-loop RRT planner to quadrotor dynamics and full 3D search. We validate the performance of the system with Monte Carlo simulations in virtual worlds and flight tests of a real quadrotor through a grove of trees. The approach is designed to support scalability to high speed flight and has numerous possible generalizations to use other polar or hybrid polar/Cartesian representations and to fuse data from additional sensors, such as peripheral optical flow or radar.

IROS Conference 2013 Conference Paper

4DoF drift free navigation using inertial cues and optical flow

Stephan Weiss 0002
Roland Brockers
Larry H. Matthies

In this paper, we describe a novel approach in fusing optical flow with inertial cues (3D acceleration and 3D angular velocities) in order to navigate a Micro Aerial Vehicle (MAV) drift free in 4DoF and metric velocity. Our approach only requires two consecutive images with a minimum of three feature matches. It does not require any (point) map nor any type of feature history. Thus it is an inherently failsafe approach that is immune to map and feature-track failures. With these minimal requirements we show in real experiments that the system is able to navigate drift free in all angles including yaw, in one metric position axis, and in 3D metric velocity. Furthermore, it is a power-on-and-go system able to online self-calibrate the inertial biases, the visual scale and the full 6DoF extrinsic transformation parameters between camera and IMU.

IROS Conference 2013 Conference Paper

A robust and modular multi-sensor fusion approach applied to MAV navigation

Simon Lynen
Markus W. Achtelik
Stephan Weiss 0002
Margarita Chli
Roland Siegwart

It has been long known that fusing information from multiple sensors for robot navigation results in increased robustness and accuracy. However, accurate calibration of the sensor ensemble prior to deployment in the field as well as coping with sensor outages, different measurement rates and delays, render multi-sensor fusion a challenge. As a result, most often, systems do not exploit all the sensor information available in exchange for simplicity. For example, on a mission requiring transition of the robot from indoors to outdoors, it is the norm to ignore the Global Positioning System (GPS) signals which become freely available once outdoors and instead, rely only on sensor feeds (e. g. , vision and laser) continuously available throughout the mission. Naturally, this comes at the expense of robustness and accuracy in real deployment. This paper presents a generic framework, dubbed MultiSensor-Fusion Extended Kalman Filter (MSF-EKF), able to process delayed, relative and absolute measurements from a theoretically unlimited number of different sensors and sensor types, while allowing self-calibration of the sensor-suite online. The modularity of MSF-EKF allows seamless handling of additional/lost sensor signals during operation while employing a state buffering scheme augmented with Iterated EKF (IEKF) updates to allow for efficient re-linearization of the prediction to get near optimal linearization points for both absolute and relative state updates. We demonstrate our approach in outdoor navigation experiments using a Micro Aerial Vehicle (MAV) equipped with a GPS receiver as well as visual, inertial, and pressure sensors.

ICRA Conference 2013 Conference Paper

Path planning for motion dependent state estimation on micro aerial vehicles

Markus W. Achtelik
Stephan Weiss 0002
Margarita Chli
Roland Siegwart

With navigation algorithms reaching a certain maturity in the field of mobile robots, the community now focuses on more advanced tasks like path planning towards increased autonomy. While the goal is to efficiently compute a path to a target destination, the uncertainty in the robot's perception cannot be ignored if a realistic path is to be computed. With most state of the art navigation systems providing the uncertainty in motion estimation, here we propose to exploit this information. This leads to a system that can plan safe avoidance of obstacles, and more importantly, it can actively aid navigation by choosing a path that minimizes the uncertainty in the monitored states. Our proposed approach is applicable to systems requiring certain excitations in order to render all their states observable, such as a MAV with visual-inertial based localization. In this work, we propose an approach which takes into account this necessary motion during path planning: by employing Rapidly exploring Random Belief Trees (RRBT), the proposed approach chooses a path to a goal which allows for best estimation of the robot's states, while inherently avoiding motion in unobservable modes. We discuss our findings within the scenario of vision-based aerial navigation as one of the most challenging navigation problem, requiring sufficient excitation to reach full observability.

ICRA Conference 2012 Conference Paper

Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments

Stephan Weiss 0002
Markus W. Achtelik
Simon Lynen
Margarita Chli
Roland Siegwart

The combination of visual and inertial sensors has proved to be very popular in robot navigation and, in particular, Micro Aerial Vehicle (MAV) navigation due the flexibility in weight, power consumption and low cost it offers. At the same time, coping with the big latency between inertial and visual measurements and processing images in real-time impose great research challenges. Most modern MAV navigation systems avoid to explicitly tackle this by employing a ground station for off-board processing. In this paper, we propose a navigation algorithm for MAVs equipped with a single camera and an Inertial Measurement Unit (IMU) which is able to run onboard and in real-time. The main focus here is on the proposed speed-estimation module which converts the camera into a metric body-speed sensor using IMU data within an EKF framework. We show how this module can be used for full self-calibration of the sensor suite in real-time. The module is then used both during initialization and as a fall-back solution at tracking failures of a keyframe-based VSLAM module. The latter is based on an existing high-performance algorithm, extended such that it achieves scalable 6DoF pose estimation at constant complexity. Fast onboard speed control is ensured by sole reliance on the optical flow of at least two features in two consecutive camera frames and the corresponding IMU readings. Our nonlinear observability analysis and our real experiments demonstrate that this approach can be used to control a MAV in speed, while we also show results of operation at 40Hz on an onboard Atom computer 1. 6 GHz.

IROS Conference 2012 Conference Paper

SFly: Swarm of micro flying robots

Markus W. Achtelik
Michael Achtelik
Yorick Brunet
Margarita Chli
Savvas A. Chatzichristofis
Jean-Dominique Decotignie
Klaus-Michael Doth
Friedrich Fraundorfer

The SFly project is an EU-funded project, with the goal to create a swarm of autonomous vision controlled micro aerial vehicles. The mission in mind is that a swarm of MAV's autonomously maps out an unknown environment, computes optimal surveillance positions and places the MAV's there and then locates radio beacons in this environment. The scope of the work includes contributions on multiple different levels ranging from theoretical foundations to hardware design and embedded programming. One of the contributions is the development of a new MAV, a hexacopter, equipped with enough processing power for onboard computer vision. A major contribution is the development of monocular visual SLAM that runs in real-time onboard of the MAV. The visual SLAM results are fused with IMU measurements and are used to stabilize and control the MAV. This enables autonomous flight of the MAV, without the need of a data link to a ground station. Within this scope novel analytical solutions for fusing IMU and vision measurements have been derived. In addition to the realtime local SLAM, an offline dense mapping process has been developed. For this the MAV's are equipped with a payload of a stereo camera system. The dense environment map is used to compute optimal surveillance positions for a swarm of MAV's. For this an optimiziation technique based on cognitive adaptive optimization has been developed. Finally, the MAV's have been equipped with radio transceivers and a method has been developed to locate radio beacons in the observed environment.

ICRA Conference 2012 Conference Paper

Versatile distributed pose estimation and sensor self-calibration for an autonomous MAV

Stephan Weiss 0002
Markus W. Achtelik
Margarita Chli
Roland Siegwart

In this paper, we present a versatile framework to enable autonomous flights of a Micro Aerial Vehicle (MAV) which has only slow, noisy, delayed and possibly arbitrarily scaled measurements available. Using such measurements directly for position control would be practically impossible as MAVs exhibit great agility in motion. In addition, these measurements often come from a selection of different onboard sensors, hence accurate calibration is crucial to the robustness of the estimation processes. Here, we address these problems using an EKF formulation which fuses these measurements with inertial sensors. We do not only estimate pose and velocity of the MAV, but also estimate sensor biases, scale of the position measurement and self (inter-sensor) calibration in real-time. Furthermore, we show that it is possible to obtain a yaw estimate from position measurements only. We demonstrate that the proposed framework is capable of running entirely onboard a MAV performing state prediction at the rate of 1 kHz. Our results illustrate that this approach is able to handle measurement delays (up to 500ms), noise (std. deviation up to 20 cm) and slow update rates (as low as 1 Hz) while dynamic maneuvers are still possible. We present a detailed quantitative performance evaluation of the real system under the influence of different disturbance parameters and different sensor setups to highlight the versatility of our approach.

IROS Conference 2012 Conference Paper

Visual-inertial SLAM for a small helicopter in large outdoor environments

Markus W. Achtelik
Simon Lynen
Stephan Weiss 0002
Laurent Kneip
Margarita Chli
Roland Siegwart

In this video, we present our latest results towards fully autonomous flights with a small helicopter. Using a monocular camera as the only exteroceptive sensor, we fuse inertial measurements to achieve a self-calibrating power-on-and-go system, able to perform autonomous flights in previously unknown, large, outdoor spaces. Our framework achieves Simultaneous Localization And Mapping (SLAM) with previously unseen robustness in onboard aerial navigation for small platforms with natural restrictions on weight and computational power. We demonstrate successful operation in flights with altitude between 0. 2–70 m, trajectories with 350 m length, as well as dynamic maneuvers with track speed of 2 m/s. All flights shown are performed autonomously using vision in the loop, with only high-level waypoints given as directions.

IROS Conference 2011 Conference Paper

3D surveillance coverage using maps extracted by a monocular SLAM algorithm

Lefteris Doitsidis
Alessandro Renzaglia
Stephan Weiss 0002
Elias B. Kosmatopoulos
Davide Scaramuzza 0001
Roland Siegwart

This paper deals with the problem of deploying a team of flying robots to perform surveillance coverage missions over a terrain of arbitrary morphology. In such missions, a key factor for the successful completion is the knowledge of the terrain's morphology. In this paper, we introduce a two-step centralized procedure to align optimally a swarm of flying vehicles for the aforementioned task. Initially, a single robot constructs a map of the area of interest using a novel monocular-vision-based approach. A state-of-the-art visual-SLAM algorithm tracks the pose of the camera while, simultaneously, building an incremental map of the surrounding environment. The map generated is processed and serves as an input in an optimization procedure using the cognitive adaptive methodology initially introduced in [1], [2]. The output of this procedure is the optimal arrangement of the robot team, which maximizes the monitored area. The efficiency of our approach is demonstrated using real data collected from aerial robots in different outdoor areas.

ICRA Conference 2011 Conference Paper

Closed-form solution for absolute scale velocity determination combining inertial measurements and a single feature correspondence

Laurent Kneip
Agostino Martinelli
Stephan Weiss 0002
Davide Scaramuzza 0001
Roland Siegwart

This paper presents a closed-form solution for metric velocity estimation of a single camera using inertial measurements. It combines accelerometer and attitude measurements with feature observations in order to compute both the distance to the feature and the speed of the camera inside the camera frame. Notably, we show that this is possible by just using three consecutive camera positions and a single feature correspondence. Our approach represents a compact linear and multirate solution for estimating complementary information to regular essential matrix computation, namely the scale of the problem. The algorithm is thoroughly validated on simulated and real data and conditions for good quality of the results are identified.

IROS Conference 2011 Conference Paper

Collaborative stereo

Markus W. Achtelik
Stephan Weiss 0002
Margarita Chli
Frank Dellaert
Roland Siegwart

In this paper, we propose a method to recover the relative pose of two robots in absolute scale and in real-time using one monocular camera on each robot. We achieve this by fusing measurements from the onboard inertial sensors on each platform with information obtained from feature correspondences between the two cameras using an Extended Kalman Filter (EKF). This forms a flexible stereo rig, providing the ability to treat the two robots as one single dynamic sensor, which can adapt to the environment and thus improve environmental mapping, obstacle avoidance and navigation. We demonstrate the power of this approach on both simulation and real datasets, employing two micro aerial vehicles (MAVs) to illustrate successful operation over general 3D motion.

IROS Conference 2011 Conference Paper

Deterministic initialization of metric state estimation filters for loosely-coupled monocular vision-inertial systems

Laurent Kneip
Stephan Weiss 0002
Roland Siegwart

In this work, we present a novel, deterministic closed-form solution for computing the scale factor and the gravity direction of a moving, loosely-coupled, and monocular vision-inertial system. The methodology is based on analysing delta-velocities. On one hand, they are obtained from a differentiation of the up-to-scale camera pose computation by a visual odometry or visual SLAM algorithm. On the other hand, they can also be retrieved from the gravity-affected short-term integration of acceleration signals. We derive a method for separating the gravity contribution and recovering the metric scale factor of the vision algorithm. The method thus also recovers the offset in roll and pitch angles of the vision reference frame with respect to the direction of the gravity vector. It uses only a single inertial integration period, and no absolute orientation information is required. For optimal sensor-fusion and metric scale-estimation filters in the loosely-coupled case, it has been shown that the convergence of the fusion of an up-to-scale pose information with inertial measurements largely depends on the availability of a good initial value for the scale factor. We show how this problem can be tackled by applying the method presented in this paper. Finally, we present results in simulation and on real data, demonstrating the suitability of the method in real scenarios.

ICRA Conference 2011 Conference Paper

Onboard IMU and monocular vision based control for MAVs in unknown in- and outdoor environments

Markus W. Achtelik
Michael Achtelik
Stephan Weiss 0002
Roland Siegwart

In this paper, we present our latest achievements towards the goal of autonomous flights of an MAV in unknown environments, only having a monocular camera as exteroceptive sensor. As MAVs are highly agile, it is not sufficient to directly use the visual input for position control at the framerates that can be achieved with small onboard computers. Our contributions in this work are twofold. First, we present a solution to overcome the issue of having a low frequent onboard visual pose update versus the high agility of an MAV. This is solved by filtering visual information with inputs from inertial sensors. Second, as our system is based on monocular vision, we present a solution to estimate the metric visual scale aid of an air pressure sensor. All computation is running onboard and is tightly integrated on the MAV to avoid jitter and latencies. This framework enables stable flights indoors and outdoors even under windy conditions.

ICRA Conference 2011 Conference Paper

Real-time metric state estimation for modular vision-inertial systems

Stephan Weiss 0002
Roland Siegwart

Single camera solutions such as monocular visual odometry or monoSLAM approaches - found a wide echo in the community. All the monocular approaches, however, suffer from the lack of metric scale. In this paper, we present a solution to tackle this issue by adding an inertial sensor equipped with a three-axis accelerometer and gyroscope. In contrast to previous approaches, our solution is independent of the underlying vision algorithm which estimates the camera poses. As a direct consequence, the algorithm presented here operates at a constant computational complexity in real time. We treat the visual framework as a black box and thus the approach is modular and widely applicable to existing monocular solutions. It can be used with any pose estimation algorithm such as visual odometry, visual SLAM, monocular or stereo setups or even GPS solutions with gravity and compass attitude estimation. In this paper, we show the thorough development of the metric state estimation based on an Extended Kalman Filter. Furthermore, even though we treat the visual framework as a black box, we show how to detect failures and estimate drifts in it. We implement our solution on a monocular vision pose estimation framework and show the results both in simulation and on real data.

IROS Conference 2011 Conference Paper

Robust embedded egomotion estimation

Rainer Voigt
Janosch Nikolic
Christoph Hürzeler
Stephan Weiss 0002
Laurent Kneip
Roland Siegwart

This work presents a method for estimating the egomotion of an aerial vehicle in challenging industrial environments. It combines binocular visual and inertial cues in a tightly-coupled fashion and operates in real time on an embedded platform. An extended Kalman filter fuses measurements and makes motion estimation rely more on inertial data if visual feature constellation is degenerate. Errors in roll and pitch are bounded implicitly by the gravity vector. Inertial sensors are used for efficient outlier detection and enable operation in poorly and repetitively textured environments. We demonstrate robustness and accuracy in an industrial scenario as well as in general indoor environments. The former is accompanied by a detailed performance evaluation supported with ground truth measurements from an external tracking system.

ICRA Conference 2010 Conference Paper

MAV navigation through indoor corridors using optical flow

Simon Zingg
Davide Scaramuzza 0001
Stephan Weiss 0002
Roland Siegwart

Safe navigation through corridors plays a major role in the autonomous use of Micro Aerial Vehicles (MAVs) in indoor environments. In this paper, we present an approach for wall collision avoidance using a depth map based on optical flow from on board camera images. An omnidirectional fisheye camera is used as a primary sensor, while IMU data is needed for compensating rotational effects of the optical flow. The here presented approach is designed for safely maneuvering a helicopter through an indoor corridor. Results based on real images taken in a corridor with textured walls are shown at the end of this paper.

ICRA Conference 2010 Conference Paper

Vision based MAV navigation in unknown and unstructured environments

Michael Bloesch
Stephan Weiss 0002
Davide Scaramuzza 0001
Roland Siegwart

Within the research on Micro Aerial Vehicles (MAVs), the field on flight control and autonomous mission execution is one of the most active. A crucial point is the localization of the vehicle, which is especially difficult in unknown, GPS-denied environments. This paper presents a novel vision based approach, where the vehicle is localized using a downward looking monocular camera. A state-of-the-art visual SLAM algorithm tracks the pose of the camera, while, simultaneously, building an incremental map of the surrounding region. Based on this pose estimation a LQG/LTR based controller stabilizes the vehicle at a desired setpoint, making simple maneuvers possible like take-off, hovering, setpoint following or landing. Experimental data show that this approach efficiently controls a helicopter while navigating through an unknown and unstructured environment. To the best of our knowledge, this is the first work describing a micro aerial vehicle able to navigate through an unexplored environment (independently of any external aid like GPS or artificial beacons), which uses a single camera as only exteroceptive sensor.