Patrick van der Smagt

TMLR Journal 2025 Journal Article

Inherently Robust Control through Maximum-Entropy Learning-Based Rollout

• Felix Bok
• Atanas Mirchev
• Baris Kayalibay
• Ole Jonas Wenzel
• Patrick van der Smagt
• Justin Bayer

Reinforcement Learning has recently proven extremely successful in the context of robot control. One of the major reasons is massively parallel simulation in conjunction with controlling for the so-called ``sim to real'' gap: training on a distribution of environments, which is assumed to contain the real one, is sufficient for finding neural policies that successfully transfer from computer simulations to real robots. Often, this is accompanied by a layer of system identification during deployment to close the gap further. Still, the efficacy of these approaches hinges on reasonable simulation capabilities with an adequately rich task distribution containing the real environment. This work aims to provide a complementary solution in cases where the aforementioned criteria may prove challenging to satisfy. We combine two approaches, $\textit{maximum-entropy reinforcement learning}$ (MaxEntRL) and $\textit{rollout}$, into an inherently robust control method called $\textbf{Maximum-Entropy Learning-Based Rollout (MELRO)}$. Both promise increased robustness and adaptability on their own. While MaxEntRL has been shown to be an adversarially-robust approach in disguise, rollout greatly improves over parametric models through an implicit Newton step on a model of the environment. We find that our approach works excellently in the vast majority of cases on both the Real World Reinforcement Learning (RWRL) benchmark and on our own environment perturbations of the popular DeepMind Control (DMC) suite, which move beyond simple parametric noise. We also show its success in ``sim to real'' transfer with the Franka Panda robot arm.

ICRA Conference 2025 Conference Paper

LIMT: Language-Informed Multi-Task Visual World Models

• Elie Aljalbout
• Nikolaos Sotirakis
• Patrick van der Smagt
• Maximilian Karl
• Nutan Chen

Most recent successes in robot reinforcement learning involve learning a specialized single-task agent. However, robots capable of performing multiple tasks can be much more valuable in real-world applications. Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objectives. Previous work on this topic is dominated by model-free approaches. The latter can be very sample inefficient even when learning specialized single-task agents. In this work, we focus on model-based multi-task reinforcement learning. We propose a method for learning multi-task visual world models, leveraging pre-trained language models to extract semantically meaningful task representations. These representations are used by the world model and policy to reason about task similarity in dynamics and behavior. Our results highlight the benefits of using language-driven task representations for world models and a clear advantage of model-based multi-task learning over the more common model-free paradigm.

TMLR Journal 2025 Journal Article

Overcoming Knowledge Barriers: Online Imitation Learning from Visual Observation with Pretrained World Models

• Xingyuan Zhang
• Philip Becker-Ehmck
• Patrick van der Smagt
• Maximilian Karl

Pretraining and finetuning models has become increasingly popular in decision-making. But there are still serious impediments in Imitation Learning from Observation (ILfO) with pretrained models. This study identifies two primary obstacles: the Embodiment Knowledge Barrier (EKB) and the Demonstration Knowledge Barrier (DKB). The EKB emerges due to the pretrained models' limitations in handling novel observations, which leads to inaccurate action inference. Conversely, the DKB stems from the reliance on limited demonstration datasets, restricting the model's adaptability across diverse scenarios. We propose separate solutions to overcome each barrier and apply them to Action Inference by Maximising Evidence (AIME), a state-of-the-art algorithm. This new algorithm, AIME-NoB, integrates online interactions and a data-driven regulariser to mitigate the EKB. Additionally, it uses a surrogate reward function to broaden the policy's supported states, addressing the DKB. Our experiments on vision-based control tasks from the DeepMind Control Suite and MetaWorld benchmarks show that AIME-NoB significantly improves sample efficiency and converged performance, presenting a robust framework for overcoming the challenges in ILfO with pretrained models. Code available at https://github.com/IcarusWizard/AIME-NoB.

ICRA Conference 2024 Conference Paper

Accurate Kinematic Modeling using Autoencoders on Differentiable Joints

• Nikolas J. Wilhelm
• Sami Haddadin
• Rainer Burgkart
• Patrick van der Smagt
• Maximilian Karl

In robotics and biomechanics, accurately determining joint parameters and computing the corresponding forward and inverse kinematics are critical yet often challenging tasks, especially when dealing with highly individualized and partly unknown systems. This paper unveils a cutting-edge kinematic optimizer, underpinned by an autoencoder-based architecture, to address these challenges. Utilizing a neural network, our approach simulates inverse kinematics, converting measurement data into joint-specific parameters during encoding, enabling a stable optimization process. These parameters are subsequently processed through a predefined, differentiable forward kinematics model, resulting in a decoded representation of the original data. Beyond offering a comprehensive solution to kinematics challenges, our method also unveils previously unidentified joint parameters. Real experimental data from knee and hand joints validate the optimizer’s efficacy. Additionally, our optimizer is multifunctional: it streamlines the modeling and automation of kinematics and enables a nuanced evaluation of diverse modeling techniques. By assessing the differences in reconstruction losses, we illuminate the merits of each approach. Collectively, this preliminary study signifies advancements in kinematic optimization, with potential applications spanning both biomechanics and robotics.

NeurIPS Conference 2024 Conference Paper

Constrained Latent Action Policies for Model-Based Offline Reinforcement Learning

• Marvin Alles
• Philip Becker-Ehmck
• Patrick van der Smagt
• Maximilian Karl

In offline reinforcement learning, a policy is learned using a static dataset in the absence of costly feedback from the environment. In contrast to the online setting, only using static datasets poses additional challenges, such as policies generating out-of-distribution samples. Model-based offline reinforcement learning methods try to overcome these by learning a model of the underlying dynamics of the environment and using it to guide policy search. It is beneficial but, with limited datasets, errors in the model and the issue of value overestimation among out-of-distribution states can worsen performance. Current model-based methods apply some notion of conservatism to the Bellman update, often implemented using uncertainty estimation derived from model ensembles. In this paper, we propose Constrained Latent Action Policies (C-LAP) which learns a generative model of the joint distribution of observations and actions. We cast policy learning as a constrained objective to always stay within the support of the latent action distribution, and use the generative capabilities of the model to impose an implicit constraint on the generated actions. Thereby eliminating the need to use additional uncertainty penalties on the Bellman update and significantly decreasing the number of gradient steps required to learn a policy. We empirically evaluate C-LAP on the D4RL and V-D4RL benchmark, and show that C-LAP is competitive to state-of-the-art methods, especially outperforming on datasets with visual observations.

ICRA Conference 2024 Conference Paper

Design and Implementation of a Robotic Testbench for Analyzing Pincer Grip Execution in Human Specimen Hands

• Nikolas J. Wilhelm
• Claudio Glowalla
• Sami Haddadin
• Julian Schote
• Hannes Höppner
• Patrick van der Smagt
• Maximilian Karl
• Rainer Burgkart

This study presents an innovative test rig engineered to explore the kinematic and viscoelastic characteristics of human specimen hands. The rig features eight force-controlled motors linked to muscle tendons, enabling precise stimulation of hand specimens. Hand movements are monitored through an optical tracking system, while a force-torque sensor quantifies the resultant fingertip loads. Employing this setup, we successfully demonstrated a pincer grip using a cadaver hand and measured both muscle forces and grip strength. Our results reveal a nonlinear relationship between tendon forces and grip strength, which can be modeled by an exponential fit. This investigation serves as a nexus between biomechanical and robotics-focused research, providing critical insights for the advancement of robotic hand actuation and therapeutic interventions.

NeurIPS Conference 2023 Conference Paper

Action Inference by Maximising Evidence: Zero-Shot Imitation from Observation with World Models

• Xingyuan Zhang
• Philip Becker-Ehmck
• Patrick van der Smagt
• Maximilian Karl

Unlike most reinforcement learning agents which require an unrealistic amount of environment interactions to learn a new behaviour, humans excel at learning quickly by merely observing and imitating others. This ability highly depends on the fact that humans have a model of their own embodiment that allows them to infer the most likely actions that led to the observed behaviour. In this paper, we propose Action Inference by Maximising Evidence (AIME) to replicate this behaviour using world models. AIME consists of two distinct phases. In the first phase, the agent learns a world model from its past experience to understand its own body by maximising the ELBO. While in the second phase, the agent is given some observation-only demonstrations of an expert performing a novel task and tries to imitate the expert's behaviour. AIME achieves this by defining a policy as an inference model and maximising the evidence of the demonstration under the policy and world model. Our method is "zero-shot" in the sense that it does not require further training for the world model or online interactions with the environment after given the demonstration. We empirically validate the zero-shot imitation performance of our method on the Walker and Cheetah embodiment of the DeepMind Control Suite and find it outperforms the state-of-the-art baselines. Code is available at: https: //github. com/argmax-ai/aime.

NeurIPS Conference 2021 Conference Paper

Latent Matters: Learning Deep State-Space Models

• Alexej Klushyn
• Richard Kurle
• Maximilian Soelch
• Botond Cseke
• Patrick van der Smagt

Deep state-space models (DSSMs) enable temporal predictions by learning the underlying dynamics of observed sequence data. They are often trained by maximising the evidence lower bound. However, as we show, this does not ensure the model actually learns the underlying dynamics. We therefore propose a constrained optimisation framework as a general approach for training DSSMs. Building upon this, we introduce the extended Kalman VAE (EKVAE), which combines amortised variational inference with classic Bayesian filtering/smoothing to model dynamics more accurately than RNN-based DSSMs. Our results show that the constrained optimisation framework significantly improves system identification and prediction accuracy on the example of established state-of-the-art DSSMs. The EKVAE outperforms previous models w. r. t. prediction accuracy, achieves remarkable results in identifying dynamical systems, and can furthermore successfully learn state-space representations where static and dynamic features are disentangled.

ICLR Conference 2021 Conference Paper

Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models

• Justin Bayer
• Maximilian Soelch 0001
• Atanas Mirchev
• Baris Kayalibay
• Patrick van der Smagt

Amortised inference enables scalable learning of sequential latent-variable models (LVMs) with the evidence lower bound (ELBO). In this setting, variational posteriors are often only partially conditioned. While the true posteriors depend, e.g., on the entire sequence of observations, approximate posteriors are only informed by past observations. This mimics the Bayesian filter---a mixture of smoothing posteriors. Yet, we show that the ELBO objective forces partially-conditioned amortised posteriors to approximate products of smoothing posteriors instead. Consequently, the learned generative model is compromised. We demonstrate these theoretical findings in three scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using fully-conditioned approximate posteriors, performance improves in terms of generative modelling and multi-step prediction.

ICLR Conference 2021 Conference Paper

Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

• Atanas Mirchev
• Baris Kayalibay
• Patrick van der Smagt
• Justin Bayer

We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep state-space model. Our approach leverages both learning and domain knowledge from multiple-view geometry and rigid-body dynamics. This results in an expressive predictive model of the world, often missing in current state-of-the-art visual SLAM solutions. The combination of variational inference, neural networks and a differentiable raycaster ensures that our model is amenable to end-to-end gradient-based optimisation. We evaluate our approach on realistic unmanned aerial vehicle flight data, nearing the performance of state-of-the-art visual-inertial odometry systems. We demonstrate the applicability of the model to generative prediction and planning.

ICLR Conference 2020 Conference Paper

Continual Learning with Bayesian Neural Networks for Non-Stationary Data

• Richard Kurle
• Botond Cseke
• Alexej Klushyn
• Patrick van der Smagt
• Stephan Günnemann

This work addresses continual learning for non-stationary data, using Bayesian neural networks and memory-based online variational Bayes. We represent the posterior approximation of the network weights by a diagonal Gaussian distribution and a complementary memory of raw data. This raw data corresponds to likelihood terms that cannot be well approximated by the Gaussian. We introduce a novel method for sequentially updating both components of the posterior approximation. Furthermore, we propose Bayesian forgetting and a Gaussian diffusion process for adapting to non-stationary data. The experimental results show that our update method improves on existing approaches for streaming data. Additionally, the adaptation methods lead to better predictive performance for non-stationary data.

ICML Conference 2020 Conference Paper

Learning Flat Latent Manifolds with VAEs

• Nutan Chen
• Alexej Klushyn
• Francesco Ferroni
• Justin Bayer
• Patrick van der Smagt

Measuring the similarity between data points often requires domain knowledge, which can in parts be compensated by relying on unsupervised methods such as latent-variable models, where similarity/distance is estimated in a more compact latent space. Prevalent is the use of the Euclidean metric, which has the drawback of ignoring information about similarity of data stored in the decoder, as captured by the framework of Riemannian geometry. We propose an extension to the framework of variational auto-encoders allows learning flat latent manifolds, where the Euclidean metric is a proxy for the similarity between data points. This is achieved by defining the latent space as a Riemannian manifold and by regularising the metric tensor to be a scaled identity matrix. Additionally, we replace the compact prior typically used in variational auto-encoders with a recently presented, more expressive hierarchical one—and formulate the learning problem as a constrained optimisation problem. We evaluate our method on a range of data-sets, including a video-tracking benchmark, where the performance of our unsupervised approach nears that of state-of-the-art supervised approaches, while retaining the computational efficiency of straight-line-based approaches.

NeurIPS Conference 2019 Conference Paper

Learning Hierarchical Priors in VAEs

• Alexej Klushyn
• Nutan Chen
• Richard Kurle
• Botond Cseke
• Patrick van der Smagt

We propose to learn a hierarchical prior in the context of variational autoencoders to avoid the over-regularisation resulting from a standard normal prior distribution. To incentivise an informative latent representation of the data, we formulate the learning problem as a constrained optimisation problem by extending the Taming VAEs framework to two-level hierarchical models. We introduce a graph-based interpolation method, which shows that the topology of the learned latent representation corresponds to the topology of the data manifold---and present several examples, where desired properties of latent representation such as smoothness and simple explanatory factors are learned by the prior.

AAAI Conference 2019 Conference Paper

Multi-Source Neural Variational Inference

• Richard Kurle
• Stephan Günnemann
• Patrick van der Smagt

Learning from multiple sources of information is an important problem in machine-learning research. The key challenges are learning representations and formulating inference methods that take into account the complementarity and redundancy of various information sources. In this paper we formulate a variational autoencoder based multi-source learning framework in which each encoder is conditioned on a different information source. This allows us to relate the sources via the shared latent variables by computing divergence measures between individual source’s posterior approximations. We explore a variety of options to learn these encoders and to integrate the beliefs they compute into a consistent posterior approximation. We visualise learned beliefs on a toy dataset and evaluate our methods for learning shared representations and structured output prediction, showing trade-offs of learning separate encoders for each information source. Furthermore, we demonstrate how conflict detection and redundancy can increase robustness of inference in a multi-source setting.

ICML Conference 2019 Conference Paper

Switching Linear Dynamics for Variational Bayes Filtering

• Philip Becker-Ehmck
• Jan Peters 0001
• Patrick van der Smagt

System identification of complex and nonlinear systems is a central problem for model predictive control and model-based reinforcement learning. Despite their complexity, such systems can often be approximated well by a set of linear dynamical systems if broken into appropriate subsequences. This mechanism not only helps us find good approximations of dynamics, but also gives us deeper insight into the underlying system. Leveraging Bayesian inference, Variational Autoencoders and Concrete relaxations, we show how to learn a richer and more meaningful state space, e. g. encoding joint constraints and collisions with walls in a maze, from partial and high-dimensional observations. This representation translates into a gain of accuracy of learned dynamics showcased on various simulated tasks.

IROS Conference 2018 Conference Paper

Active Learning based on Data Uncertainty and Model Sensitivity

• Nutan Chen
• Alexej Klushyn
• Alexandros Paraschos
• Djalel Benbouzid
• Patrick van der Smagt

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisions with the environment. Active learning can quantify the uncertainty of performing the task and, in general, locate regions of missing information. We introduce a novel algorithm for active learning and demonstrate its utility for generating smooth trajectories. Our approach is based on deep generative models and metric learning in latent spaces. It relies on the Jacobian of the likelihood to detect non-smooth transitions in the latent space, i. e. , transitions that lead to abrupt changes in the movement of the robot. When non-smooth transitions are detected, our algorithm asks for an additional demonstration from that specific region. The newly acquired knowledge modifies the data manifold and allows for learning a latent representation for generating smooth movements. We demonstrate the efficacy of our approach on generalising elementary skills, transitioning across different skills, and implicitly avoiding collisions with the environment. For our experiments, we use a simulated pendulum where we observe its motion from images and a 7-DoF anthropomorphic arm.

ICRA Conference 2017 Conference Paper

Hitting the sweet spot: Automatic optimization of energy transfer during tool-held hits

• Jörn Vogel
• Naohiro Takemura
• Hannes Höppner
• Patrick van der Smagt
• Gowrishankar Ganesh

Tool-held hitting tasks, like hammering a nail or striking a ball with a bat, require humans, and robots, to purposely collide and transfer momentum from their limbs to the environment. Due to the vibrational dynamics, every tool has a location where a hit is most efficient results in minimal tool vibrations, and consequently maximum energy transfer to the environment. In sports, this location is often referred to as the “sweet spot” of a bat, or racquet. Our recent neuroscience study suggests that humans optimize hits by using the jerk and torque felt at their hand. Motivated by this result, in this work we first analyze the vibrational dynamics of an end-effector-held bat to understand the signature projected by a sweet spot on the jerk and torque sensed at the end-effector. We then use this analysis to develop a controller for a robotic “baseball hitter”. The controller enables the robot-hitter to iteratively adjust its swing trajectory to ensure that the contact with the ball occurs at the sweet spot of the bat. We tested the controller on the DLR LWR III manipulator with three different bats. Like a human, our robot hitter is able to optimize the energy transfer, specifically maximize the ball velocity, during hits, by using its end effector position and torque sensors, and without any prior knowledge of the shape, size or material of the held bat.

IROS Conference 2017 Conference Paper

Two-stream RNN/CNN for action recognition in 3D videos

• Rui Zhao
• Haider Ali
• Patrick van der Smagt

The recognition of actions from video sequences has many applications in health monitoring, assisted living, surveillance, and smart homes. Despite advances in sensing, in particular related to 3D video, the methodologies to process the data are still subject to research. We demonstrate superior results by a system which combines recurrent neural networks with convolutional neural networks in a voting approach. The gated-recurrent-unit-based neural networks are particularly well-suited to distinguish actions based on long-term information from optical tracking data; the 3D-CNNs focus more on detailed, recent information from video data. The resulting features are merged in an SVM which then classifies the movement. In this architecture, our method improves recognition rates of state-of-the-art methods by 14% on standard data sets.

IROS Conference 2016 Conference Paper

Stable reinforcement learning with autoencoders for tactile and visual data

• Herke van Hoof
• Nutan Chen
• Maximilian Karl
• Patrick van der Smagt
• Jan Peters 0001

For many tasks, tactile or visual feedback is helpful or even crucial. However, designing controllers that take such high-dimensional feedback into account is non-trivial. Therefore, robots should be able to learn tactile skills through trial and error by using reinforcement learning algorithms. The input domain for such tasks, however, might include strongly correlated or non-relevant dimensions, making it hard to specify a suitable metric on such domains. Auto-encoders specialize in finding compact representations, where defining such a metric is likely to be easier. Therefore, we propose a reinforcement learning algorithm that can learn non-linear policies in continuous state spaces, which leverages representations learned using auto-encoders. We first evaluate this method on a simulated toy-task with visual input. Then, we validate our approach on a real-robot tactile stabilization task.

IROS Conference 2015 Conference Paper

Measuring fingertip forces from camera images for random finger poses

• Nutan Chen
• Sebastian Urban
• Justin Bayer
• Patrick van der Smagt

Robust fingertip force detection from fingernail image is a critical strategy that can be applied in many areas. However, prior research fixed many variables that influence the finger color change. This paper analyzes the effect of the finger joint on the force detection in order to deal with the constrained finger position setting. A force estimator method is designed: a model to predict the fingertip force from finger joints measured from 2D cameras and 3 rectangular markers in cooperation with the fingernail images are trained. Then the error caused by the color changes of the joint bending can be avoided. This strategy is a significant step forward from a finger force estimator that requires tedious finger joint setting. The approach is evaluated experimentally. The result shows that it increases the accuracy over 10% for the force in conditions of the finger joint free movement. The estimator is used to demonstrate lifting and replacing objects with various weights.

IROS Conference 2015 Conference Paper

Two-dimensional orthoglide mechanism for revealing areflexive human arm mechanical properties

• Hannes Höppner
• Markus Grebenstein
• Patrick van der Smagt

The most accurate and dependable approach to the in-vivo identification of human limb stiffness is by position perturbation. Moving the limb over a small distance and measuring the effective force gives, when states are steady, direct information about said stiffness. However, existing manipulandi are comparatively slow and/or not very stiff, such that a lumped stiffness is measured. This lumped stiffness includes the limb response during or after reflexes influenced by both, the passive musculotendon and active neuronal component. As this approach usually leads to inconsistencies between the data and the stiffness model, we argue in favour of fast, pre-reflex impedance measurements-i. e. , completing the perturbation movement and collecting the data before effects of spinal reflexes or even from the motor cortex can influence the measurements. To obtain such fast planar movements, we constructed a dedicated orthoglide robot while focusing on a lightweight and stiff design. Our subject study of a force task with this device lead to very clean data with always positive definite Cartesian stiffness matrices. By representing them as ellipses, we found them to be substantially bigger in comparison to standard literature which we address to a larger number of recruited motor units. While ellipses orientation and the length of their main axis increased, the shape decreased with the exerted force. The device will be used to derive design criteria for variable-stiffness robots, and to investigate the relation between muscular activity and areflexive joint stiffness for teleoperational approaches.

ICRA Conference 2014 Conference Paper

A new biarticular joint mechanism to extend stiffness ranges

• Hannes Höppner
• Wolfgang Wiedmeyer
• Patrick van der Smagt

We introduce a six-actuator robotic joint mechanism with biarticular coupling inspired by the human limb which neither requires pneumatic artificial muscles nor tendon coupling. The actuator can independently change monoarticular and biarticular stiffness as well as both joint positions. We model and analyse the actuator with respect to stiffness variability in comparison with an actuator without biarticular coupling. We demonstrate that the biarticular coupling considerably extends the range of stiffness with an 70-fold improvement in versatility, in particular with respect to the end-point Cartesian stiffness shape and orientation. We suggest using Cartesian stiffness isotropy as an optimisation criterion for future under-actuated versions.

ICRA Conference 2014 Conference Paper

Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian processes

• Nutan Chen
• Sebastian Urban
• Christian Osendorfer
• Justin Bayer
• Patrick van der Smagt

Estimating human fingertip forces is required to understand force distribution in grasping and manipulation. Human grasping behavior can then be used to develop force-and impedance-based grasping and manipulation strategies for robotic hands. However, estimating human grip force naturally is only possible with instrumented objects or unnatural gloves, thus greatly limiting the type of objects used. In this paper we describe an approach which uses images of the human fingertip to reconstruct grip force and torque at the finger. Our approach does not use finger-mounted equipment, but instead a steady camera observing the fingers of the hand from a distance. This allows for finger force estimation without any physical interference with the hand or object itself, and is therefore universally applicable. We construct a 3-dimensional finger model from 2D images. Convolutional Neural Networks (CNN) are used to predict the 2D image to a 3D model transformation matrix. Two methods of CNN are designed for separate and combined outputs of orientation and position. After learning, our system shows an alignment accuracy over 98% on unknown data. In the final step, a Gaussian process estimates finger force and torque from the aligned images based on color changes and deformations of the nail and its surrounding skin. Experimental results shows that the accuracy achieves about 95% in the force estimation and 90% in the torque.

IROS Conference 2014 Conference Paper

Model-free robot anomaly detection

• Rachel Hornung
• Holger Urbanek
• Julian Klodmann
• Christian Osendorfer
• Patrick van der Smagt

Safety is one of the key issues in the use of robots, especially when human-robot interaction is targeted. Although unforeseen environment situations, such as collisions or unexpected user interaction, can be handled with specially tailored control algorithms, hard- or software failures typically lead to situations where too large torques are controlled, which cause an emergency state: hitting an end stop, exceeding a torque, and so on-which often halts the robot when it is too late. No sufficiently fast and reliable methods exist which can early detect faults in the abundance of sensor and controller data. This is especially difficult since, in most cases, no anomaly data are available. In this paper we introduce a new robot anomaly detection system (RADS) which can cope with abundant data in which no or very little anomaly information is present.

IROS Conference 2013 Conference Paper

Computing grip force and torque from finger nail images using Gaussian processes

• Sebastian Urban
• Justin Bayer
• Christian Osendorfer
• Göran Westling
• Benoni B. Edin
• Patrick van der Smagt

We demonstrate a simple approach with which finger force can be measured from nail coloration. By automatically extracting features from nail images of a finger-mounted CCD camera, we can directly relate these images to the force measured by a force-torque sensor. The method automatically corrects orientation and illumination differences. Using Gaussian processes, we can relate prepro-cessed images of the finger nail to measured force and torque of the finger, allowing us to predict the finger force at a level of 95%–98% accuracy at force ranges up to 10N, and torques around 90% accuracy, based on training data gathered in 90s.

IROS Conference 2013 Conference Paper

Continuous robot control using surface electromyography of atrophic muscles

• Jörn Vogel
• Justin Bayer
• Patrick van der Smagt

The development of new, light robotic systems has opened up a wealth of human-robot interaction applications. In particular, the use of robot manipulators as personal assistant for the disabled is realistic and affordable, but still requires research as to the brain-computer interface. Based on our previous work with tetraplegic individuals, we investigate the use of low-cost yet stable surface Electromyography (sEMG) interfaces for individuals with Spinal Muscular Atrophy (SMA), a disease leading to the death of neuronal cells in the anterior horn of the spinal cord; with sEMG, we can record remaining active muscle fibers. We show the ability of two individuals with SMA to actively control a robot in 3. 5D continuously decoded through sEMG after a few minutes of training, allowing them to regain some independence in daily life. Although movement is not nearly as fast as natural, unimpaired movement, reach and grasp success rates are near 100% after 50s of movement.

IROS Conference 2012 Conference Paper

Optimal torque and stiffness control in compliantly actuated robots

• David J. Braun
• Florian Petit
• Felix Huber
• Sami Haddadin
• Patrick van der Smagt
• Alin Albu-Schäffer
• Sethu Vijayakumar

Anthropomorphic robots that aim to approach human performance agility and efficiency are typically highly redundant not only in their kinematics but also in actuation. Variable-impedance actuators, used to drive many of these devices, are capable of modulating torque and passive impedance (stiffness and/or damping) simultaneously and independently. Here, we propose a framework for simultaneous optimisation of torque and impedance (stiffness) profiles in order to optimise task performance, tuned to the complex hardware and incorporating real-world constraints. Simulation and hardware experiments validate the viability of this approach to complex, state dependent constraints and demonstrate task performance benefits of optimal temporal impedance modulation.

IROS Conference 2011 Conference Paper

EMG-based teleoperation and manipulation with the DLR LWR-III

• Jörn Vogel
• Claudio Castellini
• Patrick van der Smagt

In this paper we describe and practically demonstrate a robotic arm/hand system that is controlled in realtime in 6D Cartesian space through measured human muscular activity. The soft-robotics control architecture of the robotic system ensures safe physical human robot interaction as well as stable behaviour while operating in an unstructured environment. Muscular control is realised via surface electromyography, a non-invasive and simple way to gather human muscular activity from the skin. A standard supervised machine learning system is used to create a map from muscle activity to hand position, orientation and grasping force which then can be evaluated in real time—the existence of such a map is guaranteed by gravity compensation and low-speed movement. No kinematic or dynamic model of the human arm is necessary, which makes the system quickly adaptable to anyone. Numerical validation shows that the system achieves good movement precision. Live evaluation and demonstration of the system during a robotic trade fair is reported and confirms the validity of the approach, which has potential applications in muscle-disorder rehabilitation or in teleoperation where a close-range, safe master/slave interaction is required, and/or when optical/magnetic position tracking cannot be enforced.

ICRA Conference 2011 Conference Paper

The Grasp Perturbator: Calibrating human grasp stiffness during a graded force task

• Hannes Höppner
• Dominic Lakatos
• Holger Urbanek
• Claudio Castellini
• Patrick van der Smagt

In this paper we present a novel and simple handheld device for measuring in vivo human grasp impedance. The measurement method is based on a static identification method and intrinsic impedance is identified inbetween 25 ms. Using this device it is possbile to develop continuous grasp impedance measurement methods as it is an active research topic in physiology as well as in robotics, especially since nowadays (bio-inspired) robotics can be impedance-controlled. Potential applications of human impedance estimation range from impedance-controlled telesurgery to limb prosthetics and rehabilitation robotics. We validate the device through a physiological experiment in which the device is used to show a linear relationship between finger stiffness and grip force.

IROS Conference 2010 Conference Paper

The DLR touch sensor I: A flexible tactile sensor for robotic hands based on a crossed-wire approach

• Michael Strohmayr
• Hannes P. Saal
• Abhijit Potdar
• Patrick van der Smagt

One of the main challenges in service robotics is to equip dexterous robotic hands with sensitive tactile sensors in order to cope with the inherent problems posed by unknown and unstructured environments. As the increasing mechatronic integration of complex robotic hands leaves little additional space for proprioceptive sensors, exteroceptive tactile sensors become more and more important. We present a novel tactile sensor design, based on piezo-resistive soft material and a crossed-wire approach. We present the development of a first prototype and its evaluation in various classification tasks, showing promising results.

ICRA Conference 2008 Conference Paper

Surface EMG for force control of mechanical hands

• Claudio Castellini
• Patrick van der Smagt
• Giulio Sandini
• Gerhard Hirzinger

The dexterity of active hand prosthetics is limited not only due to the limited availability of dexterous prosthetic hands, but mainly due to limitations in interfaces. How is an amputee supposed to command the prosthesis what to do (i. e. , how to grasp an object) and with what force (i. e. , holding a hammer or grasping an egg)? So far, in literature, the most interesting results have been achieved by applying machine learning to forearm surface electromyography (EMG) to classify finger movements; but this approach lacks, in general, the possibility of quantitatively determining the force applied during the grasping act. In this paper we address the issue by applying machine learning to the problem of regression from the EMG signal to the force a human subject is applying to a force sensor. A detailed comparative analysis among three different machine learning approaches (Neural Networks, Support Vector Machines and Locally Weighted Projection Regression) reveals that the type of grasp can be reconstructed with an average accuracy of 90%, and the applied force can be predicted with an average error of 10%, corresponding to about 5N over a range of 50N. None of the tested approaches clearly outperforms the others, which seems to indicate that machine learning as a whole is a viable approach.

ICRA Conference 2006 Conference Paper

Learning EMG Control of a Robotic Hand: Towards Active Prostheses

• Sebastian Bitzer
• Patrick van der Smagt

We introduce a method based on support vector machines which can detect opening and closing actions of the human thumb, index finger, and other fingers recorded via surface EMG only. The method is shown to be robust across sessions and can be used independently of the position of the arm. With these stability criteria, the method is ideally suited for the control of active prosthesis with a high number of active degrees of freedom. The method is successfully demonstrated on a robotic four-finger hand, and can be used to grasp objects

IROS Conference 2004 Conference Paper

Learning from demonstration: repetitive movements for autonomous service robotics

• Holger Urbanek
• Alin Albu-Schäffer
• Patrick van der Smagt

This paper presents a method for learning and generating rhythmic movement patterns based on a simple central oscillator. It can be used to generate cyclic movements for a robot system which has to solve complex tasks. The system is laid out in such a way that multiple motion dimensions, or degrees of freedom of the robot, are represented independent of each other; therefore, an extension to higher-dimensional problems is easily possible. Guiding the robot by holding its end-effector, the user teaches simple movement primitives forming the basis for a more complex task. Each movement primitive is represented in the system using an oscillator combined with a learned nonlinear mapping. These primitives are then optimally combined to a complete solution to the posed problem. Said optimality is obtained using simulated annealing with the A* global search algorithm. Our approach is demonstrated on the problem of wiping a table, but can be used for many typical problems in service and household robotics.

ICRA Conference 1998 Conference Paper

Learning Techniques in a Dataglove Based Telemanipulation System for the DLR Hand

• Max Fischer
• Patrick van der Smagt
• Gerhard Hirzinger

We present a setup to control a four-finger anthropomorphic robot hand using a dataglove. To be able to accurately use the dataglove we implemented a nonlinear learning calibration using a novel neural network technique. Experiments show that a resulting positioning error not exceeding 1. 8 mm, but typically 0. 5 mm, per finger can be obtained; this accuracy is sufficiently precise for grasping tasks. Based on the dataglove calibration we present a solution for the mapping of human and artificial hand workspaces that enables an operator to intuitively and easily telemanipulate objects with the artificial hand.

IROS Conference 1992 Conference Paper

A Self-learning Controller For Monocular Grasping

• Patrick van der Smagt
• Ben J. A. Kröse
• Frans C. A. Groen