Author name cluster

Katherine J. Kuchenbecker

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

17 papers

1 author row

IROS Conference 2025 Conference Paper

Diffusion-Based Approximate MPC: Fast and Consistent Imitation of Multi-Modal Action Distributions

Pau Marquez Julbe
Julian Nubert
Henrik Hose
Sebastian Trimpe
Katherine J. Kuchenbecker

Approximating model predictive control (MPC) using imitation learning (IL) allows for fast control without solving expensive optimization problems online. However, methods that use neural networks in a simple L2-regression setup fail to approximate multi-modal (set-valued) solution distributions caused by local optima found by the numerical solver or nonconvex constraints, such as obstacles, significantly limiting the applicability of approximate MPC in practice. We solve this issue by using diffusion models to accurately represent the complete solution distribution (i. e. , all modes) up to kilohertz sampling rates. This work shows that diffusion-based AMPC significantly outperforms L2-regression-based approximate MPC for multi-modal action distributions. In contrast to most earlier work on IL, we also focus on running the diffusion-based controller at a higher rate and in joint space instead of end-effector space. Additionally, we propose the use of gradient guidance during the denoising process to consistently pick the same mode in closed loop to prevent switching between solutions. We propose using the cost and constraint satisfaction of the original MPC problem during parallel sampling of solutions from the diffusion model to pick a better mode online. We evaluate our method on the fast and accurate control of a 7-DoF robot manipulator both in simulation and on hardware deployed at 250 Hz, achieving a speedup of more than 70 times compared to solving the MPC problem online and also outperforming the numerical optimization (used for training) in success ratio.

ICRA Conference 2025 Conference Paper

Visuo-Tactile Object Pose Estimation for a Multi-Finger Robot Hand With Low-Resolution in-Hand Tactile Sensing

Lukas Mack
Felix Grüninger
Benjamin A. Richardson
Regine Lendway
Katherine J. Kuchenbecker
Jörg Stückler

Accurate 3D pose estimation of grasped objects is an important prerequisite for robots to perform assembly or in-hand manipulation tasks, but object occlusion by the robot's own hand greatly increases the difficulty of this perceptual task. Here, we propose that combining visual information and proprioception with binary, low-resolution tactile contact measurements from across the interior surface of an articulated robotic hand can mitigate this issue. The visuo-tactile object-pose-estimation problem is formulated probabilistically in a factor graph. The pose of the object is optimized to align with the three kinds of measurements using a robust cost function to reduce the influence of visual or tactile outlier readings. The advantages of the proposed approach are first demonstrated in simulation: a custom 15-DoF robot hand with one binary tactile sensor per link grasps 17 YCB objects while observed by an RGB-D camera. This low-resolution inhand tactile sensing significantly improves object-pose estimates under high occlusion and also high visual noise. We also show these benefits through grasping tests with a preliminary real version of our tactile hand, obtaining reasonable visuo-tactile estimates of object pose at approximately 13. 3 Hz on average.

ICRA Conference 2021 Conference Paper

Robot Interaction Studio: A Platform for Unsupervised HRI

Mayumi Mohan
Cara M. Nunez
Katherine J. Kuchenbecker

Robots hold great potential for supporting exercise and physical therapy, but such systems are often cumbersome to set up and require expert supervision. We aim to solve these concerns by combining Captury Live, a real-time markerless motion-capture system, with a Rethink Robotics Baxter Research Robot to create the Robot Interaction Studio. We evaluated this platform for unsupervised human-robot interaction (HRI) through a 75-minute-long user study with seven adults who were given minimal instructions and no feedback about their actions. The robot used sounds, facial expressions, facial colors, head motions, and arm motions to sequentially present three categories of cues in randomized order while constantly rotating its face screen to look at the user. Analysis of the captured user motions shows that the cue type significantly affected the distance subjects traveled and the amount of time they spent within the robot’s reachable workspace, in alignment with the design of the cues. Heat map visualizations of the recorded user hand positions confirm that users tended to mimic the robot’s arm poses. Despite some initial frustration, taking part in this study did not significantly change user opinions of the robot. We reflect on the advantages of the proposed approach to unsupervised HRI as well as the limitations and possible future extensions of our system.

IROS Conference 2021 Conference Paper

Sensorimotor-inspired Tactile Feedback and Control Improve Consistency of Prosthesis Manipulation in the Absence of Direct Vision

Neha Thomas
Farimah Fazlollahi
Jeremy D. Brown
Katherine J. Kuchenbecker

The lack of haptically aware upper-limb prostheses forces amputees to rely largely on visual cues to complete activities of daily living. In contrast, non-amputees inherently rely on conscious haptic perception and automatic tactile reflexes to govern volitional actions in situations that do not allow for constant visual attention. We therefore propose a myoelectric prosthesis system that reflects these concepts to aid manipulation performance without direct vision. To implement this design, we constructed two fabric-based tactile sensors that measure contact location along the palmar and dorsal sides of the prosthetic fingers and grasp pressure at the tip of the prosthetic thumb. Inspired by the natural sensorimotor system, we use the measurements from these sensors to provide vibrotactile feedback of contact location and implement a tactile grasp controller with reflexes that prevent over-grasping and object slip. We compare this tactile system to a standard myoelectric prosthesis in a challenging reach-to-pick-and-place task conducted without direct vision; 17 non-amputee adults took part in this single-session between-subjects study. Participants in the tactile group achieved more consistent high performance compared to participants in the standard group. These results show that adding contact-location feedback and reflex control increases the consistency with which objects can be grasped and moved without direct vision in upper-limb prosthetics.

ICRA Conference 2020 Conference Paper

Calibrating a Soft ERT-Based Tactile Sensor with a Multiphysics Model and Sim-to-real Transfer Learning

Hyosang Lee
Hyunkyu Park 0001
Gokhan Serhat
Huanbo Sun
Katherine J. Kuchenbecker

Tactile sensors based on electrical resistance tomography (ERT) have shown many advantages for implementing a soft and scalable whole-body robotic skin; however, calibration is challenging because pressure reconstruction is an ill-posed inverse problem. This paper introduces a method for calibrating soft ERT-based tactile sensors using sim-to-real transfer learning with a finite element multiphysics model. The model is composed of three simple models that together map contact pressure distributions to voltage measurements. We optimized the model parameters to reduce the gap between the simulation and reality. As a preliminary study, we discretized the sensing points into a 6 by 6 grid and synthesized single- and two-point contact datasets from the multiphysics model. We obtained another single-point dataset using the real sensor with the same contact location and force used in the simulation. Our new deep neural network architecture uses a de-noising network to capture the simulation-to-real gap and a reconstruction network to estimate contact force from voltage measurements. The proposed approach showed 82% hit rate for localization and 0. 51 N of force estimation error performance in singlecontact tests and 78. 5% hit rate for localization and 5. 0 N of force estimation error in two-point contact tests. We believe this new calibration method has the possibility to improve the sensing performance of ERT-based tactile sensors.

ICRA Conference 2019 Conference Paper

Improving Haptic Adjective Recognition with Unsupervised Feature Learning

Benjamin A. Richardson
Katherine J. Kuchenbecker

Humans can form an impression of how a new object feels simply by touching its surfaces with the densely innervated skin of the fingertips. Many haptics researchers have recently been working to endow robots with similar levels of haptic intelligence, but these efforts almost always employ hand-crafted features, which are brittle, and concrete tasks, such as object recognition. We applied unsupervised feature learning methods, specifically K-SVD and Spatio-Temporal Hierarchical Matching Pursuit (ST-HMP), to rich multi-modal haptic data from a diverse dataset. We then tested the learned features on 19 more abstract binary classification tasks that center on haptic adjectives such as smooth and squishy. The learned features proved superior to traditional hand-crafted features by a large margin, almost doubling the average F 1 score across all adjectives. Additionally, particular exploratory procedures (EPs) and sensor channels were found to support perception of certain haptic adjectives, underlining the need for diverse interactions and multi-modal haptic data.

ICRA Conference 2019 Conference Paper

Internal Array Electrodes Improve the Spatial Resolution of Soft Tactile Sensors Based on Electrical Resistance Tomography

Hyosang Lee
Kyungseo Park
Jung Kim
Katherine J. Kuchenbecker

Robots operating in unstructured environments would benefit from soft whole-body tactile sensors, but implementing such systems typically requires complex electrical wiring to a large number of sensing elements. The reconstruction method called electrical resistance tomography (ERT) has shown promising results (good coverage, manufacturability, and robustness) using electrodes located only along the boundary of the sensing region. However, relatively poor spatial resolution in the sensor's central region is a major drawback of the ERT approach. This paper introduces a new scheme of internal array electrodes to improve spatial resolution. We also systematically derive the optimal pairwise current injection patterns from a mathematical formulation of the ERT system. By highlighting the importance of each electrode pair, this approach enabled us to reduce the number of current injection patterns. Simulation of the standard and proposed sensor designs revealed that the internal array electrodes greatly improve distinguishability in the central region. For validation, a fabric-based soft tactile sensor made of multiple conductive fabrics was developed, including electronics that enable sampling at 200 Hz. During a 225-point localization test conducted without sensor-specific calibration, the constructed sensor showed average localization errors of 2. 85 cm ± 1. 02 cm. This result is notable because only 16 point electrodes were used to achieve this performance.

ICRA Conference 2017 Conference Paper

Proton 2: Increasing the sensitivity and portability of a visuo-haptic surface interaction recorder

Alex Burka
Abhinav Rajvanshi
Sarah Allen
Katherine J. Kuchenbecker

The Portable Robotic Optical/Tactile ObservatioN PACKage (PROTONPACK, or Proton for short) is a new handheld visuo-haptic sensing system that records surface interactions. We previously demonstrated system calibration and a classification task using external motion tracking. This paper details improvements in surface classification performance and removal of the dependence on external motion tracking, necessary before embarking on our goal of gathering a vast surface interaction dataset. Two experiments were performed to refine data collection parameters. After adjusting the placement and filtering of the Proton's high-bandwidth accelerometers, we recorded interactions between two differently-sized steel tooling ball end-effectors (diameter 6. 35 and 9. 525 mm) and five surfaces. Using features based on normal force, tangential force, end-effector speed, and contact vibration, we trained multi-class SVMs to classify the surfaces using 50 ms chunks of data from each end-effector. Classification accuracies of 84. 5% and 91. 5% respectively were achieved on unseen test data, an improvement over prior results. In parallel, we pursued on-board motion tracking, using the Proton's camera and fiducial markers. Motion tracks from the external and onboard trackers agree within 2 mm and 0. 01 rad RMS, and the accuracy decreases only slightly to 87. 7% when using onboard tracking for the 9. 525 mm end-effector. These experiments indicate that the Proton 2 is ready for portable data collection.

ICRA Conference 2016 Conference Paper

Deep learning for tactile understanding from visual and haptic data

Yang Gao 0029
Lisa Anne Hendricks
Katherine J. Kuchenbecker
Trevor Darrell

Robots which interact with the physical world will benefit from a fine-grained tactile understanding of objects and surfaces. Additionally, for certain tasks, robots may need to know the haptic properties of an object before touching it. To enable better tactile understanding for robots, we propose a method of classifying surfaces with haptic adjectives (e. g. , compressible or smooth) from both visual and physical interaction data. Humans typically combine visual predictions and feedback from physical interactions to accurately predict haptic properties and interact with the world. Inspired by this cognitive pattern, we propose and explore a purely visual haptic prediction model. Purely visual models enable a robot to “feel” without physical interaction. Furthermore, we demonstrate that using both visual and physical interaction signals together yields more accurate haptic classification. Our models take advantage of recent advances in deep neural networks by employing a unified approach to learning features for physical interaction and visual observations. Even though we employ little domain specific knowledge, our model still achieves better results than methods based on hand-designed features.

IROS Conference 2016 Conference Paper

Using IMU data to demonstrate hand-clapping games to a robot

Naomi T. Fitter
Katherine J. Kuchenbecker

All over the world, people find joy and amusement in playing hand-clapping games such as “Pat-a-cake” and “Slide. ” Thus, as robots enter everyday human spaces and work together with people, we see potential for them to entertain, engage, and assist humans through cooperative clapping games. This paper explores how data recorded from a pair of commonly available inertial measurement units (IMUs) worn on a human's hands can contribute to the teaching of a hand-clapping robot. We identified representative hand-clapping activities, considered approaches to classify games, and conducted a study to record hand-clapping motion data. Analysis of data from fifteen participants indicates that support vector machines and Markov chain analysis can correctly classify 95. 5% of the demonstrated hand-clapping motions (from ten discrete actions) and 92. 3% of the hand-clapping game demonstrations recorded in the study. These results were calculated by withholding a participant's entire dataset for testing, so these results should represent general system behavior for new users. Overall, this research lays the groundwork for a simple and efficient method that people could use to demonstrate hand-clapping games to robots.

ICRA Conference 2013 Conference Paper

Using robotic exploratory procedures to learn the meaning of haptic adjectives

Vivian Chu
Ian McMahon
Lorenzo Riano
Craig G. McDonald
Qin He
Jorge Martinez Perez-Tejada
Michael Arrigo
Naomi T. Fitter

Delivering on the promise of real-world robotics will require robots that can communicate with humans through natural language by learning new words and concepts through their daily experiences. Our research strives to create a robot that can learn the meaning of haptic adjectives by directly touching objects. By equipping the PR2 humanoid robot with state-of-the-art biomimetic tactile sensors that measure temperature, pressure, and fingertip deformations, we created a platform uniquely capable of feeling the physical properties of everyday objects. The robot used five exploratory procedures to touch 51 objects that were annotated by human participants with 34 binary adjective labels. We present both static and dynamic learning methods to discover the meaning of these adjectives from the labeled objects, achieving average F1 scores of 0. 57 and 0. 79 on a set of eight previously unfelt items.

ICRA Conference 2010 Conference Paper

Automatic filter design for synthesis of haptic textures from recorded acceleration data

Joseph M. Romano
Takashi Yoshioka
Katherine J. Kuchenbecker

Sliding a probe over a textured surface generates a rich collection of vibrations that one can easily use to create a mental model of the surface. Haptic virtual environments attempt to mimic these real interactions, but common haptic rendering techniques typically fail to reproduce the sensations that are encountered during texture exploration. Past approaches have focused on building a representation of textures using a priori ideas about surface properties. Instead, this paper describes a process of synthesizing probe-surface interactions from data recorded from real interactions. We explain how to apply the mathematical principles of Linear Predictive Coding (LPC) to develop a discrete transfer function that represents the acceleration response under specific probe-surface interaction conditions. We then use this predictive transfer function to generate unique acceleration signals of arbitrary length. In order to move between transfer functions from different probe-surface interaction conditions, we develop a method for interpolating the variables involved in the texture synthesis process. Finally, we compare the results of this process with real recorded acceleration signals, and we show that the two correlate strongly in the frequency domain.

ICRA Conference 2010 Conference Paper

Control of a high fidelity ungrounded torque feedback device: The iTorqU 2. 1

Kyle N. Winfree
Joseph M. Romano
Jamie Gewirtz
Katherine J. Kuchenbecker

This paper outlines how a control moment gyroscope can be used to generate haptic torque feedback while minimizing the effects of a constrained gimbal workspace. We present the design of the iTorqU 2. 1 and discuss how it compares to previously developed systems. We then detail the control algorithms we have developed for both transparency and torque output modes. The prescribed transparency controller is typical in design, but the torque output algorithm is novel to this type of haptic device. It makes use of a series of position-p-at-time-t commands, which we call packets. Five packet designs were considered in this research, but we have included only the most important three in this paper. While this research deals with torque feedback, it ultimately presents a method for working with devices that are limited by the need for continuous reset to a home position before subsequent outputs.

IROS Conference 2009 Conference Paper

Haptic display of realistic tool contact via dynamically compensated control of a dedicated actuator

William McMahan
Katherine J. Kuchenbecker

High frequency contact accelerations convey important information that the vast majority of haptic interfaces cannot render. Building on prior work, we present an approach to haptic interface design that uses a dedicated linear voice coil actuator and a dynamic system model to allow the user to feel these signals. This approach was tested through use in a bilateral teleoperation experiment where a user explored three textured surfaces under three different acceleration control architectures: none, constant gain, and dynamic compensation. The controllers that use the dedicated actuator vastly outperform traditional position-position control at conveying realistic contact accelerations. Analysis of root mean square error, linear regression, and discrete Fourier transforms of the acceleration data also indicate a slight performance benefit for dynamic compensation over constant gain.

ICRA Conference 2009 Conference Paper

The AirWand: Design and characterization of a large-workspace haptic device

Joseph M. Romano
Katherine J. Kuchenbecker

Almost all commercially available haptic interfaces share a common pitfall, a small shoebox-sized workspace; these devices typically rely on rigid-link manipulator design concepts. In this paper we outline our design for a new kinesthetic haptic system that drastically increases the usable haptic workspace. We present a proof-of-concept prototype, along with our analysis of its capabilities. Our design uses optical tracking to sense the position of the device, and air jet actuation to generate forces. By combining these two technologies, we are able to detach our device from the ground, thus sidestepping many problems that have plagued traditional haptic devices including workspace size, friction, and inertia. We show that optical tracking and air jet actuation successfully enable kinesthetic haptic interaction with virtual environments. Given an appropriately large volume high-pressure air source, and a reasonably high speed tracking system, this design paradigm has many desirable qualities when compared to traditional haptic design schemes.

ICRA Conference 2006 Conference Paper

Improving Telerobotic Touch via High-frequency Acceleration Matching

Katherine J. Kuchenbecker
Günter Niemeyer

Humans rely on information-laden high-frequency accelerations in addition to quasi-static forces when interacting with objects via a handheld tool. Telerobotic systems have traditionally struggled to portray such contact transients due to closed-loop bandwidth and stability limitations, leaving remote objects feeling soft and undefined. This work seeks to maximize the user's feel for the environment through the approach of acceleration matching; high-frequency fingertip accelerations are combined with standard low-frequency position feedback without requiring a secondary actuator on the master device. In this method, the natural dynamics of the master are identified offline using frequency-domain techniques, estimating the relationship between commanded motor current and handle acceleration while a user holds the device. During subsequent telerobotic interactions, a high-bandwidth sensor measures accelerations at the slave's end effector, and the real-time controller re-creates these important signals at the master handle by inverting the identified model. The details of this approach are explored herein, and its ability to render hard and rough surfaces is demonstrated on a standard master-slave system. Combining high-frequency acceleration matching with position-error-based feedback of quasi-static forces creates a hybrid signal that closely corresponds to human sensing capabilities, instilling telerobotics with a more realistic sense of remote touch

ICRA Conference 2005 Conference Paper

Modeling Induced Master Motion in Force-Reflecting Teleoperation

Katherine J. Kuchenbecker
Günter Niemeyer

Providing the user with high-fidelity force feedback has persistently challenged the field of telerobotics. Interaction forces measured at the remote site and displayed to the user cause unintended master device motion. This movement is interpreted as a command for the slave robot and can drive the closed-loop system unstable. This paper builds on a recently proposed approach for achieving stable, high-gain force reflection via cancellation of the master mech anism’s induced motion. Such a strategy hinges on obtaining a good model of the master’s response to force feedback. Herein, we present a thorough modeling approach based on successive isolation of system components, demonstrated on a one-degree-of-freedom testbed. A sixth-order mechanical model, including viscous and Coulomb friction as well as a new method for modeling hysteretic stiffness, describes the testbed’s high-frequency resonant modes. This modeling method’s ability to predict induced master motion should lead to significant improvements in force-reflecting teleoperation via the cancellation approach.