Author name cluster

Shaoxiong Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

1 author row

IROS Conference 2024 Conference Paper

In-Hand Singulation and Scooping Manipulation with a 5 DOF Tactile Gripper

Yuhao Zhou
Pokuang Zhou
Shaoxiong Wang
Yu She

Manipulation tasks often require a high degree of dexterity, typically necessitating grippers with multiple degrees of freedom (DoF). While a robotic hand equipped with multiple fingers can execute precise and intricate manipulation tasks, the inherent redundancy stemming from its extensive DoF often adds unnecessary complexity. In this paper, we introduce the design of a tactile sensor-equipped gripper with two fingers and five DoF. We present a novel design integrating a GelSight tactile sensor, enhancing sensing capabilities and enabling finer control during specific manipulation tasks. To evaluate the gripper’s performance, we conduct experiments involving two challenging tasks: 1) retrieving, singularizing, and classification of various objects embedded in granular media, and 2) executing scooping manipulations of credit cards in confined environments to achieve precise insertion. Our results demonstrate the efficiency of the proposed approach, with a high success rate for singulation and classification tasks, particularly for spherical objects at high as 94. 3%, and a 100% success rate for scooping and inserting credit cards.

Details

ICRA Conference 2023 Conference Paper

DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness

Changyi Lin
Ziqi Lin
Shaoxiong Wang
Huazhe Xu

Vision-based tactile sensors that can measure 3D geometry of the contacting objects are crucial for robots to perform dexterous manipulation tasks. However, the existing sensors are usually complicated to fabricate and delicate to extend. In this work, we novelly take advantage of the reflection property of semitransparent elastomer to design a robust, low-cost, and easy-to-fabricate tactile sensor named DTact. DTact measures high-resolution 3D geometry accurately from the darkness shown in the captured tactile images with only a single image for calibration. In contrast to previous sensors, DTact is robust under various illumination conditions. Then, we build prototypes of DTact that have non-planar contact surfaces with minimal extra efforts and costs. Finally, we perform two intelligent robotic tasks including pose estimation and object recognition using DTact, in which DTact shows large potential in applications.

Details

IROS Conference 2022 Conference Paper

Towards Learning to Play Piano with Dexterous Hands and Touch

Huazhe Xu
Yuping Luo
Shaoxiong Wang
Trevor Darrell
Roberto Calandra

As Liszt once said “(a virtuoso) must call up scent and blossom, and breathe the breath of life”, a virtuoso plays the piano with passion, poetry, and extraordinary technical ability. Hence, piano playing, being a task that is quintessentially human, becomes a hallmark for roboticians and artificial intelligence researchers to pursue. In this paper, we advocate an end-to-end reinforcement learning (RL) paradigm to demonstrate how an agent can learn directly from machine-readable music score to play the piano with touch-augmented dexterous hands on a simulated piano. To achieve the desired tasks, we design useful touch- and audio-based reward functions and a series of tasks. Empirical results show that the RL agent can not only find the correct key position but also deal with the various rhythmic, volume, and fingering requirements. As a result, the agent demonstrates its effectiveness in playing simple pieces that have different musical requirements which show the potential of leveraging reinforcement learning approach for the piano playing tasks.

Details

ICRA Conference 2021 Conference Paper

GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger

Shaoxiong Wang
Yu She
Branden Romero
Edward H. Adelson

Vision-based tactile sensors have the potential to provide important contact geometry to localize the objective with visual occlusion. However, it is challenging to measure high-resolution 3D contact geometry for a compact robot finger, to simultaneously meet optical and mechanical constraints. In this work, we present the GelSight Wedge sensor, which is optimized to have a compact shape for robot fingers, while achieving high-resolution 3D reconstruction. We evaluate the 3D reconstruction under different lighting configurations, and extend the method from 3 lights to 1 or 2 lights. We demonstrate the flexibility of the design by shrinking the sensor to the size of a human finger for fine manipulation tasks. We also show the effectiveness and potential of the reconstructed 3D geometry for pose tracking in the 3D space.

Details

ICRA Conference 2021 Conference Paper

PyTouch: A Machine Learning Library for Touch Processing

Mike Lambeta
Huazhe Xu
Jingwei Xu 0005
Po-Wei Chou
Shaoxiong Wang
Trevor Darrell
Roberto Calandra

With the increased availability of rich tactile sensors, there is an an equally proportional need for open-source and integrated software capable of efficiently and effectively processing raw touch measurements into high-level signals that can be used for control and decision-making. In this paper, we present PyTouch – the first machine learning library dedicated to the processing of touch sensing signals. PyTouch, is designed to be modular, easy-to-use and provides state-of-the-art touch processing capabilities as a service with the goal of unifying the tactile sensing community by providing a library for building scalable, proven, and performance-validated modules over which applications and research can be built upon. We evaluate PyTouch on real-world data from several tactile sensors on touch processing tasks such as touch detection, slip and object pose estimations. PyTouch is open-sourced at https://github.com/facebookresearch/pytouch.

Details

IROS Conference 2020 Conference Paper

SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation

Chen Wang
Shaoxiong Wang
Branden Romero
Filipe Veiga
Edward H. Adelson

Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity or arm accelerations, increasing the importance of mass, center of mass, and friction information. We present SwingBot, a robot that is able to learn the physical features of an held object through tactile exploration. Two exploration actions (tilting and shaking) provide the tactile information used to create a physical feature embedding space. With this embedding, SwingBot is able to predict the swing angle achieved by a robot performing dynamic swing-up manipulations on a previously unseen object. Using these predictions, it is able to search for the optimal control parameters for a desired swing-up angle. We show that with the learned physical features our end-to-end self-supervised learning pipeline is able to substantially improve the accuracy of swinging up unseen objects. We also show that objects with similar dynamics are closer to each other on the embedding space and that the embedding can be disentangled into values of specific physical properties.

Details

IROS Conference 2018 Conference Paper

3D Shape Perception from Monocular Vision, Touch, and Shape Priors

Shaoxiong Wang
Jiajun Wu 0001
Xingyuan Sun
Wenzhen Yuan 0001
William T. Freeman
Joshua B. Tenenbaum
Edward H. Adelson

Perceiving accurate 3D object shape is important for robots to interact with the physical world. Current research along this direction has been primarily relying on visual observations. Vision, however useful, has inherent limitations due to occlusions and the 2D-3D ambiguities, especially for perception with a monocular camera. In contrast, touch gets precise local shape information, though its efficiency for reconstructing the entire shape could be low. In this paper, we propose a novel paradigm that efficiently perceives accurate 3D object shape by incorporating visual and tactile observations, as well as prior knowledge of common object shapes learned from large-scale shape repositories. We use vision first, applying neural networks with learned shape priors to predict an object's 3D shape from a single-view color image. We then use tactile sensing to refine the shape; the robot actively touches the object regions where the visual prediction has high uncertainty. Our method efficiently builds the 3D shape of common objects from a color image and a small number of tactile explorations (around 10). Our setup is easy to apply and has potentials to help robots better perform grasping or manipulation tasks on real-world objects.

Details

ICRA Conference 2018 Conference Paper

Active Clothing Material Perception Using Tactile Sensing and Deep Learning

Wenzhen Yuan 0001
Yuchen Mo
Shaoxiong Wang
Edward H. Adelson

Humans represent and discriminate the objects in the same category using their properties, and an intelligent robot should be able to do the same. In this paper, we build a robot system that can autonomously perceive the object properties through touch. We work on the common object category of clothing. The robot moves under the guidance of an external Kinect sensor, and squeezes the clothes with a GelSight tactile sensor, then it recognizes the 11 properties of the clothing according to the tactile data. Those properties include the physical properties, like thickness, fuzziness, softness and durability, and semantic properties, like wearing season and preferred washing methods. We collect a dataset of 153 varied pieces of clothes, and conduct 6616 robot exploring iterations on them. To extract the useful information from the high-dimensional sensory output, we applied Convolutional Neural Networks (CNN) on the tactile data for recognizing the clothing properties, and on the Kinect depth images for selecting exploration locations. Experiments show that using the trained neural networks, the robot can autonomously explore the unknown clothes and learn their properties. This work proposes a new framework for active tactile perception system with vision-touch system, and has potential to enable robots to help humans with varied clothing related housework.

Details