Arrow Research search

Author name cluster

Simon Lynen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

18 papers
2 author rows

Possible papers

18

NeurIPS Conference 2025 Conference Paper

Scaling Image Geo-Localization to Continent Level

  • Philipp Lindenberger
  • Paul-Edouard Sarlin
  • Jan Hosang
  • Marc Pollefeys
  • Simon Lynen
  • Eduard Trulls

Determining the precise geographic location of an image at a global scale remains an unsolved challenge. Standard image retrieval techniques are inefficient due to the sheer volume of images (>100M) and fail when coverage is insufficient. Scalable solutions, however, involve a trade-off: global classification typically yields coarse results (10+ kilometers), while cross-view retrieval between ground and aerial imagery suffers from a domain gap and has been primarily studied on smaller regions. This paper introduces a hybrid approach that achieves fine-grained geo-localization across a large geographic expanse the size of a continent. We leverage a proxy classification task during training to learn rich feature representations that implicitly encode precise location information. We combine these learned prototypes with embeddings of aerial imagery to increase robustness to the sparsity of ground-level data. This enables direct, fine-grained retrieval over areas spanning multiple countries. Our extensive evaluation demonstrates that our approach can localize within 200m more than 68\% of queries of a dataset covering a large part of Europe. The code is publicly available at scaling-geoloc. github. io.

PDF Details

NeurIPS Conference 2023 Conference Paper

SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding

  • Paul-Edouard Sarlin
  • Eduard Trulls
  • Marc Pollefeys
  • Jan Hosang
  • Simon Lynen

Semantic 2D maps are commonly used by humans and machines for navigation purposes, whether it's walking or driving. However, these maps have limitations: they lack detail, often contain inaccuracies, and are difficult to create and maintain, especially in an automated fashion. Can we use raw imagery to automatically create better maps that can be easily interpreted by both humans and machines? We introduce SNAP, a deep network that learns rich 2D neural maps from ground-level and overhead images. We train our model to align neural maps estimated from different inputs, supervised only with camera poses over tens of millions of StreetView images. SNAP can resolve the location of challenging image queries beyond the reach of traditional methods, outperforming the state of the art in localization by a large margin. Moreover, our neural maps encode not only geometry and appearance but also high-level semantics, discovered without explicit supervision. This enables effective pre-training for data-efficient semantic scene understanding, with the potential to unlock cost-efficient creation of more detailed maps.

PDF Details

IROS Conference 2018 Conference Paper

LandmarkBoost: Efficient visualContext Classifiers for Robust Localization

  • Marcin Dymczyk
  • Igor Gilitschenski
  • Juan I. Nieto 0001
  • Simon Lynen
  • Bernhard Zeisl
  • Roland Siegwart

The growing popularity of autonomous systems creates a need for reliable and efficient metric pose retrieval algorithms. Currently used approaches tend to rely on nearest neighbor search of binary descriptors to perform the 2D-3D matching and guarantee realtime capabilities on mobile platforms. These methods struggle, however, with the growing size of the map, changes in viewpoint or appearance, and visual aliasing present in the environment. The rigidly defined descriptor patterns only capture a limited neighborhood of the keypoint and completely ignore the overall visual context. We propose LandmarkBoost - an approach that, in contrast to the conventional 2D-3D matching methods, casts the search problem as a landmark classification task. We use a boosted classifier to classify landmark observations and directly obtain correspondences as classifier scores. We also introduce a formulation of visual context that is flexible, efficient to compute, and can capture relationships in the entire image plane. The original binary descriptors are augmented with contextual information and informative features are selected by the boosting framework. Through detailed experiments, we evaluate the retrieval quality and performance of Landmark-Boost, demonstrating that it outperforms common state-of-the-art descriptor matching methods.

Details

ICRA Conference 2017 Conference Paper

Efficient descriptor learning for large scale localization

  • Antonio Loquercio
  • Marcin Dymczyk
  • Bernhard Zeisl
  • Simon Lynen
  • Igor Gilitschenski
  • Roland Siegwart

Many robotics and Augmented Reality (AR) systems that use sparse keypoint-based visual maps operate in large and highly repetitive environments, where pose tracking and localization are challenging tasks. Additionally, these systems usually face further challenges, such as limited computational power, or insufficient memory for storing large maps of the entire environment. Thus, developing compact map representations and improving retrieval is of considerable interest for enabling large-scale visual place recognition and loop-closure. In this paper, we propose a novel approach to compress descriptors while increasing their discriminability and match-ability, based on recent advances in neural networks. At the same time, we target resource-constrained robotics applications in our design choices. The main contributions of this work are twofold. First, we propose a linear projection from descriptor space to a lower-dimensional Euclidean space, based on a novel supervised learning strategy employing a triplet loss. Second, we show the importance of including contextual appearance information to the visual feature in order to improve matching under strong viewpoint, illumination and scene changes. Through detailed experiments on three challenging datasets, we demonstrate significant gains in performance over state-of-the-art methods.

Details

ICRA Conference 2016 Conference Paper

Point cloud descriptors for place recognition using sparse visual information

  • Titus Cieslewski
  • Elena Stumm
  • Abel Gawel
  • Mike Bosse
  • Simon Lynen
  • Roland Siegwart

Place recognition is a core component in simultaneous localization and mapping (SLAM), limiting positional drift over space and time to unlock precise robot navigation. Determining which previously visited places belong together continues to be a highly active area of research as robotic applications demand increasingly higher accuracies. A large number of place recognition algorithms have been proposed, capable of consuming a variety of sensor data including laser, sonar and depth readings. The best performing solutions, however, have utilized visual information by either matching entire images or parts thereof. Most commonly, vision based approaches are inspired by information retrieval and utilize 3D-geometry information about the observed scene as a post-verification step. In this paper we propose to use the 3D-scene information from sparse-visual feature maps directly at the core of the place recognition pipeline. We propose a novel structural descriptor which aggregates sparse triangulated landmarks from SLAM into a compact signature. The resulting 3D-features provide a discriminative fingerprint to recognize places over seasonal and viewpoint changes which are particularly challenging for approaches based on sparse visual descriptors. We evaluate our system on publicly available datasets and show how its complementary nature can provide an improvement over visual place recognition.

Details

ICRA Conference 2016 Conference Paper

Reshaping our model of the world over time

  • Marius Fehr
  • Marcin Dymczyk
  • Simon Lynen
  • Roland Siegwart

An accurate estimate of the 3D-structure in the environment is key to robotic applications such as autonomous inspection, obstacle avoidance and manipulation. Recent years have seen substantial algorithmic advances towards creating highly accurate models of small objects as well as large scale architectural structures. Most commonly a rich set of images covering a static scene are used to jointly estimate the pose of the cameras and the observed 3D-structure. For many practical application however the assumption of static scenes and sufficient coverage by images does not hold. In fact for industrial inspection the change in the scene is of most interest and the limited resources on mobile platforms don't allow for extensive data captures. In this paper we investigate the potential of combining multiple independent captures of a place to selectively reconstruct a scene over time. We propose an incremental reconstruction algorithm which identifies and fuses novel data into a joint model of the scene. Being able to identify changing parts of the scene is particularly interesting for mobile applications where bandwidth, storage and processing power are limited. Through detailed experiments, we show the potential of our approach to use multiple mobile devices to reconstruct and update a model of the static part of the environment over time.

Details

IROS Conference 2016 Conference Paper

Robustness to connectivity loss for collaborative mapping

  • Anwar Quraishi
  • Titus Cieslewski
  • Simon Lynen
  • Roland Siegwart

Having a team of robots to perform a task such as mapping is faster and more reliable than doing the same with a single robot, which can be crucial in scenarios such as search and rescue. We are developing a fully distributed framework for collaborative mapping with large robot swarms that is robust to abrupt departure of robots due to malfunctions or network problems. While several approaches to multi-robot mapping have been proposed, most of them either build a collection of local sub-maps, or rely on a central authority to merge maps built by individual robots. Our framework is unique in that it requires no central authority, yet allows robots to simultaneously contribute to a single global map, which is stored in a decentralized fashion. This greatly improves the scalability of our system with respect to number of robots. However, our approach requires systematic coordination among robots in order to make modifications to the map. Unannounced departure of the robots makes coordination challenging, and can potentially make the map inconsistent or result in loss of data. We borrow ideas from the domain of distributed computing to address those challenges. Further, we demonstrate the robustness of the proposed system by subjecting it to various conditions in which participating robots fail.

Details

IROS Conference 2015 Conference Paper

Keep it brief: Scalable creation of compressed localization maps

  • Marcin Dymczyk
  • Simon Lynen
  • Michael Bosse
  • Roland Siegwart

Robust, scalable localization unlocks path-planning, obstacle avoidance as well as manipulation and thus is a core competency for many robotic applications. However, as we leave the lab and move out in the world, models of the environment no longer span distances of meters but kilometers in length. Now, gigabytes instead of megabytes of memory are required to hold the model of the environment required for localization. Discarding data and keeping the map representation compact is thus essential for any meaningful application. This paper presents and evaluates a map compression algorithm that approaches this data-reduction as an constrained optimization problem. At the core of the algorithm is the concept of a Summary Map, a reduced map representation that includes only the landmarks that are deemed most useful for place recognition. To assign landmarks to the map we have to satisfy the conflicting goals of map coverage and localizability as well as our tight memory budget. While using an optimization approach for compression is not novel, in this paper we propose adaptations to drastically reduce the computational requirements. Our approach improves scalability from trajectories of a few tens of meters manageable by the state of the art to virtually unlimited dataset sizes in our system. We evaluate the performance of various compression levels as well as several methods for selecting the best localization landmarks from outdoor datasets.

Details

ICRA Conference 2015 Conference Paper

Map API - scalable decentralized map building for robots

  • Titus Cieslewski
  • Simon Lynen
  • Marcin Dymczyk
  • Stéphane Magnenat
  • Roland Siegwart

Large scale, long-term, distributed mapping is a core challenge to modern field robotics. Using the sensory output of multiple robots and fusing it in an efficient way enables the creation of globally accurate and consistent metric maps. To combine data from multiple agents into a global map, most existing approaches use a central entity that collects and manages the information from all agents. Often, the raw sensor data of one robot needs to be made available to processing algorithms on other agents due to the lack of computational resources on that robot. Unfortunately, network latency and low bandwidth in the field limit the generality of such an approach and make multi-robot map building a tedious task. In this paper, we present a distributed and decentralized back-end for concurrent and consistent robotic mapping. We propose a set of novel approaches that reduce the bandwidth usage and increase the effectiveness of inter-robot communication for distributed mapping. Instead of locking access to the map during operations, we define a version control system which allows concurrent and consistent access to the map data. Updates to the map are then shared asynchronously with agents which previously registered notifications. A technique for data lookup is provided by state-of-the-art algorithms from distributed computing. We validate our approach on real-world datasets and demonstrate the effectiveness of the proposed algorithms.

Details

IROS Conference 2015 Conference Paper

Real-time visual-inertial localization for aerial and ground robots

  • Helen Oleynikova
  • Michael Burri
  • Simon Lynen
  • Roland Siegwart

Localization is essential for robots to operate autonomously, especially for extended periods of time, when estimator drift tends to destroy alignment to any global map. Though there has been extensive work in vision-based localization in recent years, including several systems that show real-time performance, none have been demonstrated running entirely on-board in closed loop on robotic platforms. We propose a fast, real-time localization system that keeps the existing local visual-inertial odometry frame consistent for controllers and collision avoidance, while correcting drift and alignment to a global coordinate frame. We demonstrate our localization system entirely on-board an aerial and ground robot, showing a collaboration experiment where both robots are able to localize against the same map accurately enough to allow the multicopter to land on top of the ground robot. We also perform extensive evaluations for the proposed closed-loop system on ground-truth datasets from MAV flight in an industrial setting.

Details

ICRA Conference 2015 Conference Paper

The gist of maps - summarizing experience for lifelong localization

  • Marcin Dymczyk
  • Simon Lynen
  • Titus Cieslewski
  • Michael Bosse
  • Roland Siegwart
  • Paul Timothy Furgale

Robust, scalable place recognition is a core competency for many robotic applications. However, when revisiting places over and over, many state-of-the-art approaches exhibit reduced performance in terms of computation and memory complexity and in terms of accuracy. For successful deployment of robots over long time scales, we must develop algorithms that get better with repeated visits to the same environment, while still working within a fixed computational budget. This paper presents and evaluates an algorithm that alternates between online place recognition and offline map maintenance with the goal of producing the best performance with a fixed map size. At the core of the algorithm is the concept of a Summary Map, a reduced map representation that includes only the landmarks that are deemed most useful for place recognition. To assign landmarks to the map, we use a scoring function that ranks the utility of each landmark and a sampling policy that selects the landmarks for each place. The Summary Map can then be used by any descriptor-based inference method for constant-complexity online place recognition. We evaluate a number of scoring functions and sampling policies and show that it is possible to build and maintain maps of a constant size and that place-recognition performance improves over multiple visits.

Details

ICRA Conference 2014 Conference Paper

People detection and tracking from aerial thermal views

  • Jan Portmann
  • Simon Lynen
  • Margarita Chli
  • Roland Siegwart

Detection and tracking of people in visible-light images has been subject to extensive research in the past decades with applications ranging from surveillance to search-and-rescue. Following the growing availability of thermal cameras and the distinctive thermal signature of humans, research effort has been focusing on developing people detection and tracking methodologies applicable to this sensing modality. However, a plethora of challenges arise on the transition from visible-light to thermal images, especially with the recent trend of employing thermal cameras onboard aerial platforms (e. g. in search-and-rescue research) capturing oblique views of the scenery. This paper presents a new, publicly available dataset of annotated thermal image sequences, posing a multitude of challenges for people detection and tracking. Moreover, we propose a new particle filter based framework for tracking people in aerial thermal images. Finally, we evaluate the performance of this pipeline on our dataset, incorporating a selection of relevant, state-of-the-art methods and present a comprehensive discussion of the merits spawning from our study.

Details

IROS Conference 2013 Conference Paper

A robust and modular multi-sensor fusion approach applied to MAV navigation

  • Simon Lynen
  • Markus W. Achtelik
  • Stephan Weiss 0002
  • Margarita Chli
  • Roland Siegwart

It has been long known that fusing information from multiple sensors for robot navigation results in increased robustness and accuracy. However, accurate calibration of the sensor ensemble prior to deployment in the field as well as coping with sensor outages, different measurement rates and delays, render multi-sensor fusion a challenge. As a result, most often, systems do not exploit all the sensor information available in exchange for simplicity. For example, on a mission requiring transition of the robot from indoors to outdoors, it is the norm to ignore the Global Positioning System (GPS) signals which become freely available once outdoors and instead, rely only on sensor feeds (e. g. , vision and laser) continuously available throughout the mission. Naturally, this comes at the expense of robustness and accuracy in real deployment. This paper presents a generic framework, dubbed MultiSensor-Fusion Extended Kalman Filter (MSF-EKF), able to process delayed, relative and absolute measurements from a theoretically unlimited number of different sensors and sensor types, while allowing self-calibration of the sensor-suite online. The modularity of MSF-EKF allows seamless handling of additional/lost sensor signals during operation while employing a state buffering scheme augmented with Iterated EKF (IEKF) updates to allow for efficient re-linearization of the prediction to get near optimal linearization points for both absolute and relative state updates. We demonstrate our approach in outdoor navigation experiments using a Micro Aerial Vehicle (MAV) equipped with a GPS receiver as well as visual, inertial, and pressure sensors.

Details

IROS Conference 2013 Conference Paper

Collaborative monocular SLAM with multiple Micro Aerial Vehicles

  • Christian Forster
  • Simon Lynen
  • Laurent Kneip
  • Davide Scaramuzza 0001

This paper presents a framework for collaborative localization and mapping with multiple Micro Aerial Vehicles (MAVs) in unknown environments. Each MAV estimates its motion individually using an onboard, monocular visual odometry algorithm. The system of MAVs acts as a distributed preprocessor that streams only features of selected keyframes and relative-pose estimates to a centralized ground station. The ground station creates an individual map for each MAV and merges them together whenever it detects overlaps. This allows the MAVs to express their position in a common, global coordinate frame. The key to real-time performance is the design of data-structures and processes that allow multiple threads to concurrently read and modify the same map. The presented framework is tested in both indoor and outdoor environments with up to three MAVs. To the best of our knowledge, this is the first work on real-time collaborative monocular SLAM, which has also been applied to MAVs.

Details

IROS Conference 2013 Conference Paper

Inversion based direct position control and trajectory following for micro aerial vehicles

  • Markus W. Achtelik
  • Simon Lynen
  • Margarita Chli
  • Roland Siegwart

In this work, we present a powerful, albeit simple position control approach for Micro Aerial Vehicles (MAVs) targeting specifically multicopter systems. Exploiting the differential flatness of four of the six outputs of multicopters, namely position and yaw, we show that the remaining outputs of pitch and roll need not be controlled states, but rather just need to be known. Instead of the common approach of having multiple cascaded control loops (position — velocity — acceleration/attitude — angular rates), the proposed method employs an outer control loop based on dynamic inversion, which directly commands angular rates and thrust. The inner control loop then reduces to a simple proportional controller on the angular rates. As a result, not only does this combination allow for higher bandwidth compared to common control approaches, but also eliminates many mathematical operations (only one trigonometric function is called), speeding up the necessary processing especially on embedded systems. This approach assumes a reliable state estimation framework, which we are able to provide with through previous work. As a result, with this work, we provide the missing elements necessary for a complete approach on autonomous navigation of MAVs.

Details

ICRA Conference 2012 Conference Paper

Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments

  • Stephan Weiss 0002
  • Markus W. Achtelik
  • Simon Lynen
  • Margarita Chli
  • Roland Siegwart

The combination of visual and inertial sensors has proved to be very popular in robot navigation and, in particular, Micro Aerial Vehicle (MAV) navigation due the flexibility in weight, power consumption and low cost it offers. At the same time, coping with the big latency between inertial and visual measurements and processing images in real-time impose great research challenges. Most modern MAV navigation systems avoid to explicitly tackle this by employing a ground station for off-board processing. In this paper, we propose a navigation algorithm for MAVs equipped with a single camera and an Inertial Measurement Unit (IMU) which is able to run onboard and in real-time. The main focus here is on the proposed speed-estimation module which converts the camera into a metric body-speed sensor using IMU data within an EKF framework. We show how this module can be used for full self-calibration of the sensor suite in real-time. The module is then used both during initialization and as a fall-back solution at tracking failures of a keyframe-based VSLAM module. The latter is based on an existing high-performance algorithm, extended such that it achieves scalable 6DoF pose estimation at constant complexity. Fast onboard speed control is ensured by sole reliance on the optical flow of at least two features in two consecutive camera frames and the corresponding IMU readings. Our nonlinear observability analysis and our real experiments demonstrate that this approach can be used to control a MAV in speed, while we also show results of operation at 40Hz on an onboard Atom computer 1. 6 GHz.

Details

IROS Conference 2012 Conference Paper

SFly: Swarm of micro flying robots

  • Markus W. Achtelik
  • Michael Achtelik
  • Yorick Brunet
  • Margarita Chli
  • Savvas A. Chatzichristofis
  • Jean-Dominique Decotignie
  • Klaus-Michael Doth
  • Friedrich Fraundorfer

The SFly project is an EU-funded project, with the goal to create a swarm of autonomous vision controlled micro aerial vehicles. The mission in mind is that a swarm of MAV's autonomously maps out an unknown environment, computes optimal surveillance positions and places the MAV's there and then locates radio beacons in this environment. The scope of the work includes contributions on multiple different levels ranging from theoretical foundations to hardware design and embedded programming. One of the contributions is the development of a new MAV, a hexacopter, equipped with enough processing power for onboard computer vision. A major contribution is the development of monocular visual SLAM that runs in real-time onboard of the MAV. The visual SLAM results are fused with IMU measurements and are used to stabilize and control the MAV. This enables autonomous flight of the MAV, without the need of a data link to a ground station. Within this scope novel analytical solutions for fusing IMU and vision measurements have been derived. In addition to the realtime local SLAM, an offline dense mapping process has been developed. For this the MAV's are equipped with a payload of a stereo camera system. The dense environment map is used to compute optimal surveillance positions for a swarm of MAV's. For this an optimiziation technique based on cognitive adaptive optimization has been developed. Finally, the MAV's have been equipped with radio transceivers and a method has been developed to locate radio beacons in the observed environment.

Details

IROS Conference 2012 Conference Paper

Visual-inertial SLAM for a small helicopter in large outdoor environments

  • Markus W. Achtelik
  • Simon Lynen
  • Stephan Weiss 0002
  • Laurent Kneip
  • Margarita Chli
  • Roland Siegwart

In this video, we present our latest results towards fully autonomous flights with a small helicopter. Using a monocular camera as the only exteroceptive sensor, we fuse inertial measurements to achieve a self-calibrating power-on-and-go system, able to perform autonomous flights in previously unknown, large, outdoor spaces. Our framework achieves Simultaneous Localization And Mapping (SLAM) with previously unseen robustness in onboard aerial navigation for small platforms with natural restrictions on weight and computational power. We demonstrate successful operation in flights with altitude between 0. 2–70 m, trajectories with 350 m length, as well as dynamic maneuvers with track speed of 2 m/s. All flights shown are performed autonomously using vision in the loop, with only high-level waypoints given as directions.

Details
v2026.09.13