Author name cluster

Eckehard G. Steinbach

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

29 papers

1 author row

IROS Conference 2025 Conference Paper

Model-Mediated Teleoperation with 3D Dynamic Environment Tracking (MMT-DET): A Comparative Study of Task Performance with Time-Domain Passivity Control

Diego Fernandez Prado
Xiao Chen
Jean Elsner
Hamid Sadeghian
Nader Rajaei
Abdeldjallil Naceri
Sami Haddadin
Eckehard G. Steinbach

Teleoperation with haptic feedback allows users to interact with remote environments while retaining a sense of touch. However, the stability and transparency of these systems are compromised under communication network delay. This paper presents an augmented Model-Mediated Teleoperation with 3D object and dynamic environment tracking (MMT-DET) by a vision-based algorithm, enabling users to receive haptic feedback in structured dynamic environments while maintaining robustness against network delays. A user study comparing the proposed method with teleoperation using the Time Domain Passivity Approach (TDPA) was conducted. The results demonstrate that our MMT-DET exhibits robustness to varying delays in task performance and outperforms TDPA at higher delay levels.

ICLR Conference 2024 Conference Paper

DeepSPF: Spherical SO(3)-Equivariant Patches for Scan-to-CAD Estimation

Driton Salihu
Adam Misik
Yuankai Wu
Constantin Patsch
Fabián Seguel
Eckehard G. Steinbach

Recently, SO(3)-equivariant methods have been explored for 3D reconstruction via Scan-to-CAD. Despite significant advancements attributed to the unique characteristics of 3D data, existing SO(3)-equivariant approaches often fall short in seamlessly integrating local and global contextual information in a widely generalizable manner. Our contributions in this paper are threefold. First, we introduce Spherical Patch Fields, a representation technique designed for patch-wise, SO(3)-equivariant 3D point clouds, anchored theoretically on the principles of Spherical Gaussians. Second, we present the Patch Gaussian Layer, designed for the adaptive extraction of local and global contextual information from resizable point cloud patches. Culminating our contributions, we present Learnable Spherical Patch Fields (DeepSPF) – a versatile and easily integrable backbone suitable for instance-based point networks. Through rigorous evaluations, we demonstrate significant enhancements in Scan-to-CAD performance for point cloud registration, retrieval, and completion: a significant reduction in the rotation error of existing registration methods, an improvement of up to 17\% in the Top-1 error for retrieval tasks, and a notable reduction of up to 30\% in the Chamfer Distance for completion models, all attributable to the incorporation of DeepSPF.

IROS Conference 2024 Conference Paper

Enhanced Robotic Assistance for Human Activities through Human-Object Interaction Segment Prediction

Yuankai Wu
Rayene Messaoud
Arne-Christoph Hildebrandt
Marco Baldini
Driton Salihu
Constantin Patsch
Eckehard G. Steinbach

Robotic assistance is a current research topic with high application value and multiple challenges. Assistive robots are used in various scenarios, such as production lines, operating tables, and elderly care. While providing effective assistance, most of the assistance tasks that current robots can perform are limited to predefined tasks. This limitation arises from the insufficiency of the current robot perception system to forecast future human activities. To address this issue, we propose a novel 2-stage robotic assistant for human activities through future human-object interaction (HOI) segment prediction. Unlike previous work focusing on predefined or short-term tasks, our robotic assistant can make predictions for future assistance according to human habits. In the first stage, we propose a visual-based human-object interaction segment prediction method to predict human activities, which enables the robotic system to infer human intention. Moreover, we define the robotic executable tasks as an interactive tuple to keep the robotic assistance normatively consistent with human activity. Meanwhile, a graph convolutional network with geometric features that can predict human-object interaction segments is proposed to provide target manipulation and target object for the assistive robot. In the second stage, we present a mobile task completion process including visual navigation, object localization and grasping. The perception stage is evaluated on the MPHOI dataset and custom-collected SPHOI dataset. Finally, we evaluate our comprehensive framework through real-time experimentation.

ICRA Conference 2024 Conference Paper

HEGN: Hierarchical Equivariant Graph Neural Network for 9DoF Point Cloud Registration

Adam Misik
Driton Salihu
Xin Su
Heike Brock
Eckehard G. Steinbach

Given its wide application in robotics, point cloud registration is a widely researched topic. Conventional methods aim to find a rotation and translation that align two point clouds in 6 degrees of freedom (DoF). However, certain tasks in robotics, such as category-level pose estimation, involve non-uniformly scaled point clouds, requiring a 9DoF transform for accurate alignment. We propose HEGN, a novel equivariant graph neural network for 9DoF point cloud registration. HEGN utilizes equivariance to rotation, translation, and scaling to estimate the transformation without relying on point correspondences. Based on graph representations for both point clouds, we extract equivariant node features aggregated in their local, cross-, and global context. In addition, we introduce a novel node pooling mechanism that leverages the cross-context importance of nodes to pool the graph representation. By repeating the feature extraction and node pooling, we obtain a graph hierarchy. Finally, we determine rotation and translation by aligning equivariant features aggregated over the graph hierarchy. To estimate scaling, we leverage scale information in the vector norm of the equivariant features. We evaluate the effectiveness of HEGN through experiments with the synthetic ModelNet40 dataset and the real-world ScanObjectNN dataset. The results show the superior performance of HEGN in 9DoF point cloud registration and its competitive performance in conventional 6DoF point cloud registration.

ICRA Conference 2024 Conference Paper

HPF-SLAM: An Efficient Visual SLAM System Leveraging Hybrid Point Features

Xin Su
Sebastian Eger
Adam Misik
Dong Yang
Rastin Pries
Eckehard G. Steinbach

Visual SLAM is an essential tool in diverse applications such as robot perception and extended reality, where feature-based methods are prevalent due to their accuracy and robustness. However, existing methods employ either hand-crafted or solely learnable point features and are thus limited by the feature attributes. In this paper, we propose incorporating hybrid point features efficiently into a single system. By integrating hand-crafted and learnable features, we seek to capitalize on their complementary attributes in both key-point identification and descriptor expressiveness. To this purpose, we design a pre-processing module, which includes extraction, inter-class processing, and post-processing of hybrid point features. We present an efficient matching approach to exclusively perform the data association within the same class of features. Moreover, we design a Hybrid Bag-of-Words (H-BoW) model to deal with hybrid point features in matching and loop-closure-detection. By integrating the proposed framework into a modern feature-based system, we introduce HPF-SLAM. We evaluate the system on EuRoC-MAV and TUM-RGBD benchmarks. The experimental results show that our method consistently surpasses the baseline at comparable speed.

IROS Conference 2024 Conference Paper

Rethinking 3D Geometric Object Features for Enhancing Skeleton-based Action Recognition

Yuankai Wu
Chi Wang
Driton Salihu
Constantin Patsch
Marsil Zakour
Eckehard G. Steinbach

Human action recognition is crucial for intelligent robots, especially in the realm of human-robot collaboration research. Recent advancements in human pose estimation algorithms have shifted the focus of action recognition towards skeleton-based models, which exhibit robustness to changes in background and illumination. However, many state-of-the-art action recognition models rely on 2D skeleton data, neglecting object features. This limitation becomes obvious in complex scenarios where human interactions with objects are crucial, potentially compromising the reliability of assistive robots in understanding human behavior in their environment. To address this issue, we propose a method that effectively integrates 3D geometric object features into skeleton data using graph convolutional neural networks (GCNs). In addition to analyzing the effectiveness of information from different dimensions such as object center position, category, translation, and rotation, we explore various adjacency matrix designs for graph networks. Our model performance is evaluated on two challenging datasets: IKEA ASM and Bimanual Actions. The results demonstrate a significant improvement in action recognition by integrating object features into skeleton-based models. Specifically, on the IKEA-ASM dataset, our approach achieves a frame-wise Top-1 score improvement of 10. 8% and an average F1@k improvement of 13. 3%, while on the Bimanual Actions dataset, it achieves a frame-wise Top-1 score improvement of 11. 4% and an average F1@k improvement of 5. 3%, with negligible increases in model complexity.

IROS Conference 2024 Conference Paper

Sim-to-Real Domain Shift in Online Action Detection

Constantin Patsch
Wael Torjmene
Marsil Zakour
Yuankai Wu
Driton Salihu
Eckehard G. Steinbach

Human reasoning comprises the ability to understand and reason about the current action solely based on past information. To provide effective assistance in an eldercare or household environment an assistive robot or intelligent assistive system has to assess human actions correctly. Based on this presumption, the task of online action detection determines the current action solely based on the past without access to future information. During inference, the performance of the model is largely impacted by the attributes of the underlying training dataset. However, as high costs and ethical concerns are associated with the real-world data collection process, synthetically created data provides a way to mitigate these problems while providing additional data for the training process of the underlying action detection model to improve performanceDue to the inherent domain shift between the synthetic and real data, we introduce a new egocentric dataset called Human Kitchen Interactions (HKI) to investigate the sim-to-real gap. Our dataset contains in total 100 synthetic and real videos in which 21 different actions are executed in a kitchen environment. The synthetic data is acquired in an egocentric virtual reality (VR) setup while capturing the virtual environment in a game engine. We evaluate state-of-the-art online action detection models on our dataset and provide insights into sim-to-real domain shift. Upon acceptance, we will release our dataset and the corresponding features at https://c-patsch.github.io/HKI/.

IROS Conference 2023 Conference Paper

Dynamic Multi-Query Motion Planning with Differential Constraints and Moving Goals

Michael Gentner
Fabian Zillenbiller
André Kraft
Eckehard G. Steinbach

Planning robot motions in complex environments is a fundamental research challenge and central to the autonomy, efficiency, and ultimately adoption of robots. While often the environment is assumed to be static, real-world settings, such as assembly lines, contain complex shaped, moving obstacles and changing target states. Therein robots must perform safe and efficient motions to achieve their tasks. In repetitive environments and multi-goal settings, reusable roadmaps can substantially reduce the overall query time. Most dynamic roadmap-based planners operate in state-time-space, which is computationally demanding. Interval-based methods store availabilities as node attributes and thereby circumvent the dimensionality increase. However, current approaches do not consider higher-order constraints, which can ultimately lead to collisions during execution. Furthermore, current approaches must replan when the goal changes. To this end, we propose a novel roadmap-based planner for systems with third-order differential constraints operating in dynamic environments with moving goals. We construct a roadmap with availabilities as node attributes. During the query phase, we use a Double-Integrator Minimum Time (DIMT) solver to recursively build feasible trajectories and accurately estimate arrival times. An exit node set in combination with a moving goal heuristic is used to efficiently find the fastest path through the roadmap to the moving goal. We evaluate our method with a simulated UAV operating in dynamic 2D environments and show that it also transfers to a 6-DoF manipulator. We show higher success rates than other state-of-the-art methods both in collision avoidance and reaching a moving goal.

IROS Conference 2023 Conference Paper

Haptic Dataset Augmentation with Subjective QoE Labels using Conditional Generative Adversarial Network

Zican Wang
Xiao Xu 0001
Dong Yang
Zhenyu Wang 0010
Sarah Shtaierman
Eckehard G. Steinbach

This paper proposes a novel Generative Adversarial Network (GAN)-based strategy to augment subjective haptic Quality of Experience (QoE) datasets for bilateral teleoperation with haptic feedback without conducting time-consuming subjective experiments. In our previous work, we proposed a multi-assessment fusion approach to predict subjective haptic quality using a collection of objective metrics. This method requires a sufficiently large haptic dataset with QoE labels. The proposed generative approach automatically expands the existing haptic quality dataset by combining a modified conditional GAN (CGAN) and Style GAN (StyleGAN) architecture. The most important feature of our method is that it learns from the labeled training data and focuses on synthesizing signals with artifacts according to new input labels containing the QoE score, time delay, control method, and data reduction information. Extensive experiments are conducted to validate the suitability of the expanded dataset. The results show that our approach is able to generate new data, which match the label and signal distribution of the original data with categorical rank and linear correlation of over 0. 85.

ICRA Conference 2023 Conference Paper

SRI-Graph: A Novel Scene-Robot Interaction Graph for Robust Scene Understanding

Dong Yang
Xiao Xu 0001
Mengchen Xiong
Edwin Babaians
Eckehard G. Steinbach

We propose a novel scene-robot interaction graph (SRI-Graph) that exploits the known position of a mobile manipulator for robust and accurate scene understanding. Compared to the state-of-the-art scene graph approaches, the proposed SRI-Graph captures not only the relationships between the objects, but also the relationships between the robot manipulator and objects with which it interacts. To improve the detection accuracy of spatial relationships, we leverage the 3D position of the mobile manipulator in addition to RGB images. The manipulator's ego information is crucial for a successful scene understanding when the relationships are visually uncertain. The proposed model is validated for a real-world 3D robot-assisted feeding task. We release a new dataset named 3DRF-Pos for training and validation. We also develop a tool, named LabelImg-Rel, as an extension of the open-sourced image annotation tool LabelImg for a convenient annotation in robot-environment interaction scenarios *. Our experimental results using the Movo platform show that SRI-Graph outperforms the state-of-the-art approach and improves detection accuracy by up to 9. 83%.

IROS Conference 2022 Conference Paper

Block-based Novel Haptic Data Reduction for Time-delayed Teleoperation

Ming Gui
Xiao Xu 0001
Eckehard G. Steinbach

This work proposes a novel haptic data reduction scheme for time-delayed teleoperation by coding information as blocks. State-of-the-art (SOTA) haptic data reduction approaches are mainly sampled-based schemes. They encode haptic signals sample by sample in order to minimize the introduced coding delay. In contrast, our proposed block-based coding approach transmits a sample block as a single unit (haptic packet). Although it introduces additional algorithmic delays that are proportional to the block length, block coding has benefits since the packet rate is easy to control, the coding approach can be lossless, and the intra-block information can be employed to improve the force feedback quality. We further develop an energy adjustment approach that uses the information in a block to mitigate force oscillations caused by the Time Domain Passivity Approach. Simulation experiments and subjective tests demonstrate that our method reduces network load and significantly increases force feedback quality compared with the SOTA sample-based coding schemes, particularly for mid- to high-latency networks and low packet rates.

IROS Conference 2022 Conference Paper

PourNet: Robust Robotic Pouring Through Curriculum and Curiosity-based Reinforcement Learning

Edwin Babaians
Tapan Sharma
Mojtaba Karimi
Sahand Sharifzadeh
Eckehard G. Steinbach

Pouring liquids accurately into containers is one of the most challenging tasks for robots as they are unaware of the complex fluid dynamics and the behavior of liquids when pouring. Therefore, it is not possible to formulate a generic pouring policy for real-time applications. In this paper, we propose PourNet, as a generalized solution to pouring different liquids into containers. PourNet is a hybrid planner that uses deep reinforcement learning, for end-effector planning, and Nonlinear Model Predictive Control, for joint planning. In this work, we introduce a novel simulation environment using Unity3D and NVIDIA-Flex to train our agents. By effective choice of the state space, action space and the reward functions, we allow for a direct sim-to-real transfer of the learned skills without additional training. In the simulation, PourNet outperforms state-of-the-art by an average of 4. 9g deviation for water-like, and 9. 2g deviation for honey-like liquids. In the real-world scenario using Kinova Movo Platform, PourNet achieves an average pouring deviation of 2. 3g for dish soap when using a novel pouring container. The average pouring deviation measured for water was 5. 5g. All comprehensive experiments and the simulation environment is available at: http://cxdcxd.github.io/RRS/.

IROS Conference 2022 Conference Paper

Skill-CPD: Real-time Skill Refinement for Shared Autonomy in Manipulator Teleoperation

Edwin Babaians
Dong Yang
Mojtaba Karimi
Xiao Xu 0001
Serkut Ayvasik
Eckehard G. Steinbach

Advanced wireless communication networks provide lower latency and a higher transmission rate. Although this is an enabler for many new teleoperation applications, the risk of network instability or packet drop is still unavoidable. Real-time manipulator teleoperation requires data transmission with no discontinuity. Shared autonomy (SA) is a standard method to mitigate this issue. In this way, if the data from the remote side is unavailable, the controller can continue based on the previously observed models. However, due to the spatial gap between human and robot trajectories, indisputable fluctuations occur, which cause issues in teleoperation applications. This motivates us to propose a new skill refinement strategy to modify the previously trained skill and mitigate the sudden unwanted motions within the control takeover phase. To this end, our approach comprises applying the Hidden Semi-Markov Model (HSMM) and Linear Quadratic Tracker (LQT) in combination to learn and predict the user's intentions and then exploiting Coherent Point Drift (CPD) to refine the executable trajectory. We test our method both in simulation and in the real world for 2D English letter drawing and 3D robot-assisted feeding scenarios. Our experimental results using the Kinova® Movo platform show that the proposed refinement approach generates a stable trajectory and mitigates the control switching inconsistency. All comprehensive experiments and source code is available at: http://cxdcxd.github.io/SkillCPD.

IROS Conference 2021 Conference Paper

NMPC-MP: Real-time Nonlinear Model Predictive Control for Safe Motion Planning in Manipulator Teleoperation

Siqi Hu
Edwin Babaians
Mojtaba Karimi
Eckehard G. Steinbach

Motion control and planning for the manipulator are critical components in manipulator teleoperation. Online (real-time) motion control is challenging for active obstacle avoidance and often results in fluctuating and unsafe motion. Offline motion planning, on the other hand, generates precise and secure trajectories for complex manipulation. In this paper, a real-time nonlinear model predictive control based motion planner (NMPC-MP) is designed for teleoperated manipulation. In contrast to traditional NMPC-based approaches, our model considers a complex environment with dynamic obstacles. Our multi-threaded NMPC-MP allows for real-time planning, including dynamic objects. We evaluate our approach both in a simulated environment and with real-world experiments using the Kinova ® Movo platform. The comparison to state-of-the-art approaches (e. g. , RRT-Connect, CHOMP, and STOMP) shows a significant improvement in real-time motion planning using NMPC-MP. In real-world tests, the proposed planner was applied on a human-shaped dual manipulator setup. Our results show that the NMPC-MP runs in real-time and generates smooth and reliable trajectories. The experiments validate that the planner is able to precisely track active goals from the teleoperator while avoiding self-collision and obstacles.

IROS Conference 2021 Conference Paper

QoE-driven Delay-adaptive Control Scheme Switching for Time-delayed Bilateral Teleoperation with Haptic Data Reduction

Xiao Xu 0001
Siyuan Zhang
Qian Liu 0001
Eckehard G. Steinbach

Teleoperation systems with haptic feedback allow a human user to remotely interact with a dangerous or inac-cessible environment, perform various tasks, and perceive the haptic feedback. To ensure system stability while maintaining the best possible quality of experience (QoE), different teleoperation control schemes and haptic communication strategies need to be selected to adapt to varying network conditions and teleoperation tasks. In this paper, we propose a QoE-driven control scheme switching approach, which adaptively selects the control scheme that provides the best possible QoE for varying communication delay. A transition period is designed to moderate the artifacts during the switching phase. Haptic data reduction approaches are developed for the switching strategy to match the characteristics of each control scheme. Our experiments verify the feasibility of the proposed scheme. Subjective tests confirm that the proposed adaptive switching scheme is able to achieve a superior user QoE in contrast to a fixed control scheme in the presence of varying communication delay up to 200 ms.

ICRA Conference 2020 Conference Paper

6DFC: Efficiently Planning Soft Non-Planar Area Contact Grasps using 6D Friction Cones

Jingyi Xu
Michael Danielczuk
Eckehard G. Steinbach
Ken Goldberg

Analytic grasp planning algorithms typically approximate compliant contacts with soft point contact models to compute grasp quality, but these models are overly conservative and do not capture the full range of grasps available. While area contact models can reduce the number of false negatives predicted by point contact models, they have been restricted to a 3D analysis of the wrench applied at the contact and so are still overly conservative. We extend traditional 3D friction cones and present an efficient algorithm for calculating the 6D friction cone (6DFC) for a non-planar area contact between a compliant gripper and a rigid object. We introduce a novel sampling algorithm to find the 6D friction limit surface for a non-planar area contact and a linearization method for these ellipsoids that reduces the computation of 6DFC constraints to a quadratic program. We show that constraining the wrench applied at the contact in this way increases recall, a metric inversely related to the number of false negative predictions, by 17% and precision, a metric inversely related to the number of false positive predictions, by 2% over soft point contact models on results from 1500 physical grasps on 12 3D printed nonplanar objects with an ABB YuMi robot. The 6DFC algorithm also achieves 6% higher recall with similar precision and 85x faster runtime than a previously proposed area contact model.

ICRA Conference 2020 Conference Paper

Minimal Work: A Grasp Quality Metric for Deformable Hollow Objects

Jingyi Xu
Michael Danielczuk
Jeffrey Ichnowski
Jeffrey Mahler
Eckehard G. Steinbach
Ken Goldberg

Robot grasping of deformable hollow objects such as plastic bottles and cups is challenging, as the grasp should resist disturbances while minimally deforming the object so as not to damage it or dislodge liquids. We propose minimal work as a novel grasp quality metric that combines wrench resistance and object deformation. We introduce an efficient algorithm to compute the work required to resist an external wrench for a manipulation task by solving a linear program. The algorithm first computes the minimum required grasp force and an estimation of the gripper jaw displacements based on the object's empirical stiffness at different locations. The work done by the jaws is the product of the grasp force and the displacements. Grasps requiring minimal work are considered to be of high quality. We collect 460 physical grasps with a UR5 robot and a Robotiq gripper. We consider a grasp to be successful if it completes the task without damaging the object or dislodging the content. Physical experiments suggest that the minimal work quality metric reaches 74. 2% balanced accuracy, a metric that is the raw accuracy normalized by the number of successful and failed real-world grasps, and is up to 24. 2% higher than classical wrench-based quality metrics.

ICRA Conference 2018 Conference Paper

A Delay Compensation Approach for Pan-Tilt-Unit-based Stereoscopic 360 Degree Telepresence Systems Using Head Motion Prediction

Tamay Aykut
Chenxi Zou
Jingyi Xu
Dominik Van Opdenbosch
Eckehard G. Steinbach

The acceptance of teleoperation applications like tele-driving, tele-surgery, tele-maintenance, etc. , is challenged by the quality-reducing effect of end-to-end latency. Particularly, when users wear Head-Mounted Displays to enhance the immersive experience, the lag between head motion and display response leads to unbearable motion sickness, indisposition, and, in the worst case, abortion of the teleoperation session. In this paper, we propose a delay compensation approach with head motion prediction that can be applied to pan-tilt-unit-based stereoscopic telepresence systems. We provide the user with the impression of a 3D 360° video that represents the remote scene without noticing the present delay, even when rotating the head. To this end, we propose a novel prediction paradigm for head motion estimation to substantially mitigate the negative impact of the latency on the quality of experience. We re-implemented state-of-the-art head movement predictors and compare them to our proposed approach by means of qualitative measures. In our experiments, we used two real and independent head motion datasets for validation and tested communication delays between 100-1000ms. Our results show that mean compensation rates of more than 99% are able with our approach.

IROS Conference 2018 Conference Paper

Learning-Based Modular Task-Oriented Grasp Stability Assessment

Jingyi Xu
Amit Bhardwaj
Ge Sun
Tamay Aykut
Nicolas Alt
Mojtaba Karimi
Eckehard G. Steinbach

Assessing grasp stability is essential to prevent the failure of robotic manipulation tasks due to sensory data and object uncertainties. Learning-based approaches are widely deployed to infer the success of a grasp. Typically, the underlying model used to estimate the grasp stability is trained for a specific task, such as lifting, hand-over, or pouring. Since every task has individual stability demands, it is important to adapt the trained model to new manipulation actions. If the same trained model is directly applied to a new task, unnecessary grasp adaptations might be triggered, or in the worst case, the manipulation might fail. To address this issue, we divide the manipulation task used for training into seven sub-tasks, defined as modular tasks. We deploy a learning-based approach and assess the stability for each modular task separately. We further propose analytical features to reduce the dimensionality and the redundancy of the tactile sensor readings. A main task can thereby be represented as a sequence of relevant modular tasks. The stability prediction of the main task is computed based on the inferred success labels of the modular tasks. Our experimental evaluation shows that the proposed feature set lowers the prediction error up to 5. 69% compared to other sets used in state-of-the-art methods. Robotic experiments demonstrate that our modular task-oriented stability assessment avoids unnecessary grasp force adaptations and regrasps for various manipulation tasks.

ICRA Conference 2018 Conference Paper

Selection and Compression of Local Binary Features for Remote Visual SLAM

Dominik Van Opdenbosch
Martin Oelsch
Adrian Garcea
Tamay Aykut
Eckehard G. Steinbach

In the field of autonomous robotics, Simultaneous Localization and Mapping (SLAM) is still a challenging problem. With cheap visual sensors attracting more and more attention, various solutions to the SLAM problem using visual cues have been proposed. However, current visual SLAM systems are still computationally demanding, especially on embedded devices. In addition, collaborative SLAM approaches emerge using visual information acquired from multiple robots simultaneously to build a joint map. In order to address both challenges, we present an approach for remote visual SLAM where local binary features are extracted at the robot, compressed and sent over a network to a centralized powerful processing node running the visual SLAM algorithm. To this end, we propose a new feature coding scheme including a feature selection stage which ensures that only relevant information is transmitted. We demonstrate the effectiveness of our approach on well-known datasets. With the proposed approach, it is possible to build an accurate map while limiting the data rate to 75 kbits/frame.

ICRA Conference 2017 Conference Paper

Grasping posture estimation for a two-finger parallel gripper with soft material jaws using a curved contact area friction model

Jingyi Xu
Nicolas Alt
Zhongyao Zhang
Eckehard G. Steinbach

We present a friction model for the curved contact area between a deformable object and soft parallel gripper jaws for grasping posture estimation. We show that the assumption of a planar contact area leads to an overestimation of the frictional force and torque, which might cause the object to slip. We simulate the contact with the Finite Element Method, then compute the friction wrenches, which are fitted with two limit surface models: an ellipsoid and a convex 4th-order polynomial. Despite a slightly higher fitting error, the ellipsoid limit surface is chosen to compute the grasp quality because of its simplicity. We compare the limit surfaces of our friction model with the planar contact model and show the improved accuracy obtainable with our model. We then apply the presented model for grasping posture estimation by simulating the contact for all grasp candidates. We show a grasp quality map (quality of all grasp candidates) and the best possible grasp location for several deformable objects.

ICRA Conference 2012 Conference Paper

Beyond classical teleoperation: Assistance, cooperation, data reduction, and spatial audio

Thomas Schauss
Carolina Passenberg
Nikolay Stefanov
Daniela Feth
Iason Vittorias
Angelika Peer
Sandra Hirche
Martin Buss

In this video we present a teleoperation system which is capable of solving complex tasks in human-sized wide area environments. The system consists of two mobile teleoperators controlled by two operators, and offers haptic, visual, and auditory feedback. The task examined here, consists of repairing a robot by removing a computer and replacing a defective hard-drive. To cope with the complexity of such a task, we go beyond classical teleoperation by integrating several advanced software algorithms into the system.

ICRA Conference 2012 Conference Paper

Real-time compression of point cloud streams

Julius Kammerl
Nico Blodow
Radu Bogdan Rusu
Suat Gedikli
Michael Beetz
Eckehard G. Steinbach

We present a novel lossy compression approach for point cloud streams which exploits spatial and temporal redundancy within the point data. Our proposed compression framework can handle general point cloud streams of arbitrary and varying size, point order and point density. Furthermore, it allows for controlling coding complexity and coding precision. To compress the point clouds, we perform a spatial decomposition based on octree data structures. Additionally, we present a technique for comparing the octree data structures of consecutive point clouds. By encoding their structural differences, we can successively extend the point clouds at the decoder. In this way, we are able to detect and remove temporal redundancy from the point cloud data stream. Our experimental results show a strong compression performance of a ratio of 14 at 1 mm coordinate precision and up to 40 at a coordinate precision of 9 mm.

ICRA Conference 2011 Conference Paper

Surprise-driven acquisition of visual object representations for cognitive mobile robots

Werner Maier 0001
Eckehard G. Steinbach

Robots in a household environment have to execute a variety of tasks including carrying objects. In order to grasp the correct objects for a desired action it is indispensable that the robot is able to recognize the objects in its environment. From time to time, the robot will encounter new unknown objects which it has never seen before. In order to recognize them in a later task the robot has to acquire an internal representation of them. In this paper, we present an approach for the autonomous acquisition of visual object representations in a cluttered environment. Guided by surprise, the robot detects novel objects in a familiar environment, selects local image features which represent their appearance and stores them in a database. Experimental results show that our method for the detection of surprising events reliably directs the robot's attention to the novel objects and that the recognition behavior based on our acquired object representations outperforms a state-of-the-art approach.

ICRA Conference 2010 Conference Paper

High-fidelity telepresence and teleaction

Robert Bauernschmitt
Martin Buss
Barbara Deml
Klaus Diepold
Berthold Färber
Georg Färber
Ulrich Hagn
Gerhard Hirzinger

The collaborative research center SFB453 (www. sfb453.de) aims to realize high-fidelity telepresence and teleaction systems.

ICRA Conference 2010 Conference Paper

Illumination-invariant image-based novelty detection in a cognitive mobile robot's environment

Werner Maier 0001
Fengqing Bao
Elmar Mair
Eckehard G. Steinbach
Darius Burschka

Image-based scene representations enable a mobile robot to make a realistic prediction of its environment. Hence, it is able to rapidly detect changes in its surroundings by comparing a virtual image generated from previously acquired reference images and its current observation. This facilitates attentional control to novel events. However, illumination effects can impair attentional control if the robot does not take them into account. To address this issue, we present in this paper an approach for the acquisition of illumination-invariant scene representations. Using multiple spatial image sequences which are captured under varying illumination conditions the robot computes an illumination-invariant image-based environment model. With this representation and statistical models about the illumination behavior, the robot is able to robustly detect texture changes in its environment under different lighting. Experimental results show high-quality images which are free of illumination effects as well as more robust novelty detection compared to state-of-the-art methods.

ICRA Conference 2009 Conference Paper

Visual homing and surprise detection for cognitive mobile robots using image-based environment representations

Werner Maier 0001
Elmar Mair
Darius Burschka
Eckehard G. Steinbach

One important feature of a cognitive system is to perceive and understand its environment and to adapt its actions to changes and unforeseen situations. In this paper, we propose a scheme for visual surprise detection in cognitive mobile robots. With the robot's observation and a set of reference images previously acquired near its current viewpoint, a pixel-wise surprise trigger is computed using Bayesian probabilistic inference techniques. With appropriate mathematical approximations this algorithm can be implemented on modern graphics hardware which nearly allows for real-time surprise detection. In order to refer to prior observations, a mobile robot has to be able to re-localize itself with respect to its environment. Thus, we also present two online image-based homing algorithms which both facilitate the computation of location-independent surprise triggers. Experiments show acceptable results in terms of robust and fast detection of unexpected changes in the environment.

IROS Conference 2008 Conference Paper

Multi-modal multi-user telepresence and teleaction system

Martin Buss
Angelika Peer
Thomas Schauss
Nikolay Stefanov
Ulrich Unterhinninghofen
Stephan Behrendt
Georg Färber
Jan Leupold

The video shows a rich multi-modal multi-user telepresence system, which was developed within the SFB453 funded by the German Research Foundation (www. sfb453.de). As a complex application scenario, the remote repairing of a broken pipe is presented, see Fig. 1.

ICRA Conference 2004 Conference Paper

Disposal of Explosive Ordnances by Use of a Bimanual Haptic Telepresence System

Alexander Kron
Günther Schmidt 0001
Bernd Petzold
Michael Friedrich Zäh
Peter Hinterseer
Eckehard G. Steinbach

The paper outlines a novel approach for performing explosive ordnance disposal by use of a bimanual haptic telepresence system. This system enables an operator to perceive multimodal feedback from a remote environment for proper task execution. The developed experimental setup, comprising a bimanual human system interface and the corresponding bimanual teleoperator for use of both hands, is presented in detail. The teleoperation control architecture is discussed as well as a local model-based impedance control algorithm for manipulator control. Human-system performance is improved by means of stereo visualization of the tele-environment together with overlayed Augmented Reality assistance, and an algorithm for avoidance of dangerous manipulator configurations supported by augmented force feedback. Furthermore, a recently developed UDP communication library is presented for system interconnection taking into account compression of haptic data. Thus efficient and low delay of data transfer is ensured. The usability and effectiveness of the developed bimanual telepresence system are demonstrated by focusing a relevant task scenario, such as demining operations in a remote environment.