Author name cluster

Markus Spies

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

ICRA Conference 2025 Conference Paper

Segment Any Repeated Object

Yushi Liu
Christian Graf
Markus Spies
Margret Keuper

Understanding a scene in terms of objects and their properties is fundamental for various vision-based robotic applications, including item picking. To effectively clear a bin, a robot must comprehend objects as graspable entities, often without prior access to models of the target object. This study focuses on open world object segmentation with the additional requirement of assigning identical class labels for repeated instances of the same object. This capability enables item picking tasks with homogeneous bins, filtering out packaging material, and sorting tasks. We propose a novel pipeline for detecting repeated instances of identical objects, building on recent advancements in vision foundation models and exploring approaches for estimating object similarities based on feature embeddings or keypoint correspondence matching. Through a comprehensive experimental evaluation, we establish a new state-of-the-art on ARMBench repeated objects segmentation, a particularly challenging open problem in bin-picking robotics. Additionally, we demonstrate the real-world application of our method integrated into a robot picking cell to showcase its relevance to industrial use cases.

Details

ICRA Conference 2024 Conference Paper

Cycle-Correspondence Loss: Learning Dense View-Invariant Visual Features from Unlabeled and Unordered RGB Images

David B. Adrian
Andras G. Kupcsik
Markus Spies
Heiko Neumann

Robot manipulation relying on learned object-centric descriptors became popular in recent years. Visual descriptors can easily describe manipulation task objectives, they can be learned efficiently using self-supervision, and they can encode actuated and even non-rigid objects. However, learning robust, view-invariant keypoints in a self-supervised approach requires a meticulous data collection approach involving precise calibration and expert supervision. In this paper we introduce Cycle-Correspondence Loss (CCL) for view-invariant dense descriptor learning, which adopts the concept of cycle-consistency, enabling a simple data collection pipeline and training on unpaired RGB camera views. The key idea is to autonomously detect valid pixel correspondences by attempting to use a prediction over a new image to predict the original pixel in the original image, while scaling error terms based on the estimated confidence. Our evaluation shows that we outperform other self-supervised RGB-only methods, and approach performance of supervised methods, both with respect to keypoint tracking as well as for a robot grasping downstream task.

Details

ICRA Conference 2024 Conference Paper

Efficient End-to-End Detection of 6-DoF Grasps for Robotic Bin Picking

Yushi Liu
Alexander Qualmann
Zehao Yu 0002
Miroslav Gabriel
Philipp Schillinger
Markus Spies
Ngo Anh Vien
Andreas Geiger 0001

Bin picking is an important building block for many robotic systems, in logistics, production or in household use-cases. In recent years, machine learning methods for the prediction of 6-DoF grasps on diverse and unknown objects have shown promising progress. However, existing approaches only consider a single ground truth grasp orientation at a grasp location during training and therefore can only predict limited grasp orientations which leads to a reduced number of feasible grasps in bin picking with restricted reachability. In this paper, we propose a novel approach for learning dense and diverse 6-DoF grasps for parallel-jaw grippers in robotic bin picking. We introduce a parameterized grasp distribution model based on Power-Spherical distributions that enables a training based on all possible ground truth samples. Thereby, we also consider the grasp uncertainty enhancing the model’s robustness to noisy inputs. As a result, given a single top-down view depth image, our model can generate diverse grasps with multiple collision-free grasp orientations. Experimental evaluations in simulation and on a real robotic bin picking setup demonstrate the model’s ability to generalize across various object categories achieving an object clearing rate of around 90% in simulation and real-world experiments. We also outperform state of the art approaches. Moreover, the proposed approach exhibits its usability in real robot experiments without any refinement steps, even when only trained on a synthetic dataset, due to the probabilistic grasp distribution modeling.

Details

ICRA Conference 2022 Conference Paper

Efficient and Robust Training of Dense Object Nets for Multi-Object Robot Manipulation

David B. Adrian
Andras G. Kupcsik
Markus Spies
Heiko Neumann

We propose a framework for robust and efficient training of Dense Object Nets (DON) [1] with a focus on industrial multi-object robot manipulation scenarios. DON is a popular approach to obtain dense, view-invariant object descriptors, which can be used for a multitude of downstream tasks in robot manipulation, such as, pose estimation, state representation for control, etc. However, the original work [1] focused training on singulated objects, with limited results on instance-specific, multi-object applications. Additionally, a complex data collection pipeline, including 3D reconstruction and mask annotation of each object, is required for training. In this paper, we further improve the efficacy of DON with a simplified data collection and training regime, that consistently yields higher precision and enables robust tracking of keypoints with less data requirements. In particular, we focus on training with multi-object data instead of singulated objects, combined with a well-chosen augmentation scheme. We additionally propose an alternative loss formulation to the original pixel wise formulation that offers better results and is less sensitive to hyperparameters. Finally, we demonstrate the robustness and accuracy of our proposed framework on a real-world robotic grasping task.

Details

AAAI Conference 2021 Conference Paper

Supervised Training of Dense Object Nets using Optimal Descriptors for Industrial Robotic Applications

Andras Gabor Kupcsik
Markus Spies
Alexander Klein
Marco Todescato
Nicolai Waniek
Philipp Schillinger
Mathias Bürger

Dense Object Nets (DONs) by Florence, Manuelli and Tedrake (2018) introduced dense object descriptors as a novel visual object representation for the robotics community. It is suitable for many applications including object grasping, policy learning, etc. DONs map an RGB image depicting an object into a descriptor space image, which implicitly encodes key features of an object invariant to the relative camera pose. Impressively, the self-supervised training of DONs can be applied to arbitrary objects and can be evaluated and deployed within hours. However, the training approach relies on accurate depth images and faces challenges with small, reflective objects, typical for industrial settings, when using consumer grade depth cameras. In this paper we show that given a 3D model of an object, we can generate its descriptor space image, which allows for supervised training of DONs. We rely on Laplacian Eigenmaps (LE) to embed the 3D model of an object into an optimally generated space. While our approach uses more domain knowledge, it can be efficiently applied even for smaller and reflective objects, as it does not rely on depth information. We compare the training methods on generating 6D grasps for industrial objects and show that our novel supervised training approach improves the pick-andplace performance in industry-relevant tasks.

PDF Details

IROS Conference 2020 Conference Paper

Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks

Leonel Rozo
Meng Guo 0002
Andras G. Kupcsik
Marco Todescato
Philipp Schillinger
Markus Giftthaler
Matthias Ochs
Markus Spies

Enabling robots to quickly learn manipulation skills is an important, yet challenging problem. Such manipulation skills should be flexible, e. g. , be able adapt to the current workspace configuration. Furthermore, to accomplish complex manipulation tasks, robots should be able to sequence several skills and adapt them to changing situations. In this work, we propose a rapid robot skill-sequencing algorithm, where the skills are encoded by object-centric hidden semi-Markov models. The learned skill models can encode multimodal (temporal and spatial) trajectory distributions. This approach significantly reduces manual modeling efforts, while ensuring a high degree of flexibility and re-usability of learned skills. Given a task goal and a set of generic skills, our framework computes smooth transitions between skill instances. To compute the corresponding optimal end-effector trajectory in task space we rely on Riemannian optimal controller. We demonstrate this approach on a 7 DoF robot arm for industrial assembly tasks.

Details

AAAI Conference 2019 Conference Paper

Bounded Suboptimal Search with Learned Heuristics for Multi-Agent Systems

Markus Spies
Marco Todescato
Hannes Becker
Patrick Kesper
Nicolai Waniek
Meng Guo

A wide range of discrete planning problems can be solved optimally using graph search algorithms. However, optimal search quickly becomes infeasible with increased complexity of a problem. In such a case, heuristics that guide the planning process towards the goal state can increase performance considerably. Unfortunately, heuristics are often unavailable or need manual and time-consuming engineering. Building upon recent results on applying deep learning to learn generalized reactive policies, we propose to learn heuristics by imitation learning. After learning heuristics based on optimal examples, they are used to guide a classical search algorithm to solve unseen tasks. However, directly applying learned heuristics in search algorithms such as A∗ breaks optimality guarantees, since learned heuristics are not necessarily admissible. Therefore, we (i) propose a novel method that utilizes learned heuristics to guide Focal Search A∗, a variant of A∗ with guarantees on bounded suboptimality; (ii) compare the complexity and performance of jointly learning individual policies for multiple robots with an approach that learns one policy for all robots; (iii) thoroughly examine how learned policies generalize to previously unseen environments and demonstrate considerably improved performance in a simulated complex dynamic coverage problem.

PDF Details

ICRA Conference 2019 Conference Paper

Informed Information Theoretic Model Predictive Control

Raphael Kusumoto
Luigi Palmieri
Markus Spies
Akos Csiszar
Kai O. Arras

The problem of minimizing cost in nonlinear control systems with uncertainties or disturbances remains a major challenge. Model predictive control (MPC), and in particular sampling-based MPC has recently shown great success in complex domains such as aggressive driving with highly nonlinear dynamics. Sampling-based methods rely on a prior distribution to generate samples in the first place. Obviously, the choice of this distribution highly influences efficiency of the controller. Existing approaches such as sampling around the control trajectory of the previous time step perform suboptimally, especially in multi-modal or highly dynamic settings. In this work, we therefore propose to learn models that generate samples in low-cost areas of the state-space, conditioned on the environment and on contextual information of the task to solve. By using generative models as an informed sampling distribution, our approach exploits guidance from the learned models and at the same time maintains robustness properties of the MPC methods. We use Conditional Variational Autoencoders (CVAE) to learn distributions that imitate samples from a training dataset containing optimized controls. An extensive evaluation in the autonomous navigation domain suggests that replacing previous sampling schemes with our learned models considerably improves performance in terms of path quality and planning efficiency.

Details

ICRA Conference 2019 Conference Paper

Trust Regions for Safe Sampling-Based Model Predictive Control

Martin Koch 0008
Markus Spies
Mathias Bürger

Guaranteeing safe constraint satisfaction in nonlinear control systems with uncertainty remains a major challenge for control. The most successful control method handling constraints under uncertainty has without doubt been model predictive control (MPC). In particular, recent sampling-based MPC methods have shown success in controlling stochastic systems with complex, nonlinear dynamics. The sampling-based schemes are appealing since they do not need strong assumptions on the underlying model, except that it can be forward simulated. At the same time, the lack of major assumptions on the models make the statement of safety or robustness guarantees difficult. However, the samples drawn during the control process inherently contain probabilistic information about these properties. In this paper, we formally describe the problem that results by adding chance constraints to a sampling-based MPC scheme. Furthermore, based on a variant of the Chernoff bound, we derive trust regions, in which the sampling based estimation of the safety constraint satisfies a specified quality. Finally, we present a case study in the navigation domain to demonstrate the applicability of the proposed approach.

Details