Author name cluster

Justin Bayer

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

TMLR Journal 2025 Journal Article

Inherently Robust Control through Maximum-Entropy Learning-Based Rollout

Felix Bok
Atanas Mirchev
Baris Kayalibay
Ole Jonas Wenzel
Patrick van der Smagt
Justin Bayer

Reinforcement Learning has recently proven extremely successful in the context of robot control. One of the major reasons is massively parallel simulation in conjunction with controlling for the so-called ``sim to real'' gap: training on a distribution of environments, which is assumed to contain the real one, is sufficient for finding neural policies that successfully transfer from computer simulations to real robots. Often, this is accompanied by a layer of system identification during deployment to close the gap further. Still, the efficacy of these approaches hinges on reasonable simulation capabilities with an adequately rich task distribution containing the real environment. This work aims to provide a complementary solution in cases where the aforementioned criteria may prove challenging to satisfy. We combine two approaches, $\textit{maximum-entropy reinforcement learning}$ (MaxEntRL) and $\textit{rollout}$, into an inherently robust control method called $\textbf{Maximum-Entropy Learning-Based Rollout (MELRO)}$. Both promise increased robustness and adaptability on their own. While MaxEntRL has been shown to be an adversarially-robust approach in disguise, rollout greatly improves over parametric models through an implicit Newton step on a model of the environment. We find that our approach works excellently in the vast majority of cases on both the Real World Reinforcement Learning (RWRL) benchmark and on our own environment perturbations of the popular DeepMind Control (DMC) suite, which move beyond simple parametric noise. We also show its success in ``sim to real'' transfer with the Franka Panda robot arm.

PDF Details

ICLR Conference 2021 Conference Paper

Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models

Justin Bayer
Maximilian Soelch 0001
Atanas Mirchev
Baris Kayalibay
Patrick van der Smagt

Amortised inference enables scalable learning of sequential latent-variable models (LVMs) with the evidence lower bound (ELBO). In this setting, variational posteriors are often only partially conditioned. While the true posteriors depend, e.g., on the entire sequence of observations, approximate posteriors are only informed by past observations. This mimics the Bayesian filter---a mixture of smoothing posteriors. Yet, we show that the ELBO objective forces partially-conditioned amortised posteriors to approximate products of smoothing posteriors instead. Consequently, the learned generative model is compromised. We demonstrate these theoretical findings in three scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using fully-conditioned approximate posteriors, performance improves in terms of generative modelling and multi-step prediction.

Details

ICLR Conference 2021 Conference Paper

Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

Atanas Mirchev
Baris Kayalibay
Patrick van der Smagt
Justin Bayer

We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep state-space model. Our approach leverages both learning and domain knowledge from multiple-view geometry and rigid-body dynamics. This results in an expressive predictive model of the world, often missing in current state-of-the-art visual SLAM solutions. The combination of variational inference, neural networks and a differentiable raycaster ensures that our model is amenable to end-to-end gradient-based optimisation. We evaluate our approach on realistic unmanned aerial vehicle flight data, nearing the performance of state-of-the-art visual-inertial odometry systems. We demonstrate the applicability of the model to generative prediction and planning.

Details

ICML Conference 2020 Conference Paper

Learning Flat Latent Manifolds with VAEs

Nutan Chen
Alexej Klushyn
Francesco Ferroni
Justin Bayer
Patrick van der Smagt

Measuring the similarity between data points often requires domain knowledge, which can in parts be compensated by relying on unsupervised methods such as latent-variable models, where similarity/distance is estimated in a more compact latent space. Prevalent is the use of the Euclidean metric, which has the drawback of ignoring information about similarity of data stored in the decoder, as captured by the framework of Riemannian geometry. We propose an extension to the framework of variational auto-encoders allows learning flat latent manifolds, where the Euclidean metric is a proxy for the similarity between data points. This is achieved by defining the latent space as a Riemannian manifold and by regularising the metric tensor to be a scaled identity matrix. Additionally, we replace the compact prior typically used in variational auto-encoders with a recently presented, more expressive hierarchical one—and formulate the learning problem as a constrained optimisation problem. We evaluate our method on a range of data-sets, including a video-tracking benchmark, where the performance of our unsupervised approach nears that of state-of-the-art supervised approaches, while retaining the computational efficiency of straight-line-based approaches.

Details

IROS Conference 2015 Conference Paper

Measuring fingertip forces from camera images for random finger poses

Nutan Chen
Sebastian Urban
Justin Bayer
Patrick van der Smagt

Robust fingertip force detection from fingernail image is a critical strategy that can be applied in many areas. However, prior research fixed many variables that influence the finger color change. This paper analyzes the effect of the finger joint on the force detection in order to deal with the constrained finger position setting. A force estimator method is designed: a model to predict the fingertip force from finger joints measured from 2D cameras and 3 rectangular markers in cooperation with the fingernail images are trained. Then the error caused by the color changes of the joint bending can be avoided. This strategy is a significant step forward from a finger force estimator that requires tedious finger joint setting. The approach is evaluated experimentally. The result shows that it increases the accuracy over 10% for the force in conditions of the finger joint free movement. The estimator is used to demonstrate lifting and replacing objects with various weights.

Details

ICRA Conference 2014 Conference Paper

Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian processes

Nutan Chen
Sebastian Urban
Christian Osendorfer
Justin Bayer
Patrick van der Smagt

Estimating human fingertip forces is required to understand force distribution in grasping and manipulation. Human grasping behavior can then be used to develop force-and impedance-based grasping and manipulation strategies for robotic hands. However, estimating human grip force naturally is only possible with instrumented objects or unnatural gloves, thus greatly limiting the type of objects used. In this paper we describe an approach which uses images of the human fingertip to reconstruct grip force and torque at the finger. Our approach does not use finger-mounted equipment, but instead a steady camera observing the fingers of the hand from a distance. This allows for finger force estimation without any physical interference with the hand or object itself, and is therefore universally applicable. We construct a 3-dimensional finger model from 2D images. Convolutional Neural Networks (CNN) are used to predict the 2D image to a 3D model transformation matrix. Two methods of CNN are designed for separate and combined outputs of orientation and position. After learning, our system shows an alignment accuracy over 98% on unknown data. In the final step, a Gaussian process estimates finger force and torque from the aligned images based on color changes and deformations of the nail and its surrounding skin. Experimental results shows that the accuracy achieves about 95% in the force estimation and 90% in the torque.

Details

IROS Conference 2013 Conference Paper

Computing grip force and torque from finger nail images using Gaussian processes

Sebastian Urban
Justin Bayer
Christian Osendorfer
Göran Westling
Benoni B. Edin
Patrick van der Smagt

We demonstrate a simple approach with which finger force can be measured from nail coloration. By automatically extracting features from nail images of a finger-mounted CCD camera, we can directly relate these images to the force measured by a force-torque sensor. The method automatically corrects orientation and illumination differences. Using Gaussian processes, we can relate prepro-cessed images of the finger nail to measured force and torque of the finger, allowing us to predict the finger force at a level of 95%–98% accuracy at force ranges up to 10N, and torques around 90% accuracy, based on training data gathered in 90s.

Details

IROS Conference 2013 Conference Paper

Continuous robot control using surface electromyography of atrophic muscles

Jörn Vogel
Justin Bayer
Patrick van der Smagt

The development of new, light robotic systems has opened up a wealth of human-robot interaction applications. In particular, the use of robot manipulators as personal assistant for the disabled is realistic and affordable, but still requires research as to the brain-computer interface. Based on our previous work with tetraplegic individuals, we investigate the use of low-cost yet stable surface Electromyography (sEMG) interfaces for individuals with Spinal Muscular Atrophy (SMA), a disease leading to the death of neuronal cells in the anterior horn of the spinal cord; with sEMG, we can record remaining active muscle fibers. We show the ability of two individuals with SMA to actively control a robot in 3. 5D continuously decoded through sEMG after a few minutes of training, allowing them to regain some independence in daily life. Although movement is not nearly as fast as natural, unimpaired movement, reach and grasp success rates are near 100% after 50s of movement.

Details

JMLR Journal 2010 Journal Article

PyBrain

Tom Schaul
Justin Bayer
Daan Wierstra
Yi Sun
Martin Felder
Frank Sehnke
Thomas Rückstieß
Jürgen Schmidhuber

PyBrain is a versatile machine learning library for Python. Its goal is to provide flexible, easy-to-use yet still powerful algorithms for machine learning tasks, including a variety of predefined environments and benchmarks to test and compare algorithms. Implemented algorithms include Long Short-Term Memory (LSTM), policy gradient methods, (multidimensional) recurrent neural networks and deep belief networks. [abs] [ pdf ][ bib ] &copy JMLR 2010. ( edit, beta )

PDF Details