Author name cluster

Juan Aparicio Ojea

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

1 author row

IROS Conference 2020 Conference Paper

Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards

Gerrit Schoettler
Ashvin Nair
Jianlan Luo
Shikhar Bahl
Juan Aparicio Ojea
Eugen Solowjow
Sergey Levine

Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling, traditional control methods often result in brittle and inaccurate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be capable of learning controllers in such environments from autonomous interaction with the environment, but running RL algorithms in the real world poses sample efficiency and safety challenges. Moreover, in practical real-world settings, we cannot assume access to perfect state information or dense reward signals. In this paper, we consider a variety of difficult industrial insertion tasks with visual inputs and different natural reward specifications, namely sparse rewards and goal images. We show that methods that combine RL with prior information, such as classical controllers or demonstrations, can solve these tasks from a reasonable amount of real-world interaction.

Details

IROS Conference 2020 Conference Paper

Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks

Gerrit Schoettler
Ashvin Nair
Juan Aparicio Ojea
Sergey Levine
Eugen Solowjow

Robotic insertion tasks are characterized by contact and friction mechanics, making them challenging for conventional feedback control methods due to unmodeled physical effects. Reinforcement learning (RL) is a promising approach for learning control policies in such settings. However, RL can be unsafe during exploration and might require a large amount of real-world training data, which is expensive to collect. In this paper, we study how to use meta-reinforcement learning to solve the bulk of the problem in simulation by solving a family of simulated industrial insertion tasks and then adapt policies quickly in the real world. We demonstrate our approach by training an agent to successfully perform challenging real-world insertion tasks using less than 20 trials of real-world experience.

Details

ICRA Conference 2019 Conference Paper

Domain Randomization for Active Pose Estimation

Xinyi Ren
Jianlan Luo
Eugen Solowjow
Juan Aparicio Ojea
Abhishek Gupta 0004
Aviv Tamar
Pieter Abbeel

Accurate state estimation is a fundamental component of robotic control. In robotic manipulation tasks, as is our focus in this work, state estimation is essential for identifying the positions of objects in the scene, forming the basis of the manipulation plan. However, pose estimation typically requires expensive 3D cameras or additional instrumentation such as fiducial markers to perform accurately. Recently, Tobin et al. introduced an approach to pose estimation based on domain randomization, where a neural network is trained to predict pose directly from a 2D image of the scene. The network is trained on computer generated images with a high variation in textures and lighting, thereby generalizing to real world images. In this work, we investigate how to improve the accuracy of domain randomization based pose estimation. Our main idea is that active perception - moving the robot to get a better estimate of pose- can be trained in simulation and transferred to real using domain randomization. In our approach, the robot trains in a domain-randomized simulation how to estimate pose from a sequence of images. We show that our approach can significantly improve the accuracy of standard pose estimation in several scenarios: when the robot holding an object moves, when reference objects are moved in the scene, or when the camera is moved around the object.

Details

ICRA Conference 2019 Conference Paper

Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly

Jianlan Luo
Eugen Solowjow
Chengtao Wen
Juan Aparicio Ojea
Alice M. Agogino
Aviv Tamar
Pieter Abbeel

Precise robotic manipulation skills are desirable in many industrial settings, reinforcement learning (RL) methods hold the promise of acquiring these skills autonomously. In this paper, we explicitly consider incorporating operational space force/torque information into reinforcement learning; this is motivated by humans heuristically mapping perceived forces to control actions, which results in completing high-precision tasks in a fairly easy manner. Our approach combines RL with force/torque information by incorporating a proper operational space force controller; where we also exploit different ablations on processing this information. Moreover, we propose a neural network architecture that generalizes to reasonable variations of the environment. We evaluate our method on the open-source Siemens Robot Learning Challenge, which requires precise and delicate force-controlled behavior to assemble a tight-fit gear wheel set.

Details

ICRA Conference 2019 Conference Paper

Residual Reinforcement Learning for Robot Control

Tobias Johannink
Shikhar Bahl
Ashvin Nair
Jianlan Luo
Avinash Kumar 0005
Matthias Loskyll
Juan Aparicio Ojea
Eugen Solowjow

Conventional feedback control methods can solve various types of robot control problems very efficiently by capturing the structure with explicit models, such as rigid body equations of motion. However, many control problems in modern manufacturing deal with contacts and friction, which are difficult to capture with first-order physical modeling. Hence, applying control design methodologies to these kinds of problems often results in brittle and inaccurate controllers, which have to be manually tuned for deployment. Reinforcement learning (RL) methods have been demonstrated to be capable of learning continuous robot controllers from interactions with the environment, even for problems that include friction and contacts. In this paper, we study how we can solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL. The final control policy is a superposition of both control signals. We demonstrate our approach by training an agent to successfully perform a real-world block assembly task involving contacts and unstable objects.

Details

IROS Conference 2018 Conference Paper

Deep Reinforcement Learning for Robotic Assembly of Mixed Deformable and Rigid Objects

Jianlan Luo
Eugen Solowjow
Chengtao Wen
Juan Aparicio Ojea
Alice M. Agogino

Reinforcement learning for assembly tasks can yield powerful robot control algorithms for applications that are challenging or even impossible for “conventional” feedback control methods. Insertion of a rigid peg into a deformable hole of smaller diameter is such a task. In this contribution we solve this task with Deep Reinforcement Learning. Force-torque measurements from a robot arm wrist sensor are thereby incorporated two-fold; they are integrated into the policy learning process and they are exploited in an admittance controller that is coupled to the neural network. This enables robot learning of contact-rich assembly tasks without explicit joint torque control or passive mechanical compliance. We demonstrate our approach in experiments with an industrial robot.

Details

ICRA Conference 2018 Conference Paper

Learning Robotic Assembly from CAD

Garrett Thomas
Melissa Chien
Aviv Tamar
Juan Aparicio Ojea
Pieter Abbeel

In this work, motivated by recent manufacturing trends, we investigate autonomous robotic assembly. Industrial assembly tasks require contact-rich manipulation skills, which are challenging to acquire using classical control and motion planning approaches. Consequently, robot controllers for assembly domains are presently engineered to solve a particular task, and cannot easily handle variations in the product or environment. Reinforcement learning (RL) is a promising approach for autonomously acquiring robot skills that involve contact-rich dynamics. However, RL relies on random exploration for learning a control policy, which requires many robot executions, and often gets trapped in locally suboptimal solutions. Instead, we posit that prior knowledge, when available, can improve RL performance. We exploit the fact that in modern assembly domains, geometric information about the task is readily available via the CAD design files. We propose to leverage this prior knowledge by guiding RL along a geometric motion plan, calculated using the CAD data. We show that our approach effectively improves over traditional control approaches for tracking the motion plan, and can solve assembly tasks that require high precision, even without accurate state estimation. In addition, we propose a neural network architecture that can learn to track the motion plan, thereby generalizing the assembly controller to changes in the object positions.

Details

ICRA Conference 2017 Conference Paper

Design of parallel-jaw gripper tip surfaces for robust grasping

Menglong Guo
David V. Gealy
Jacky Liang
Jeffrey Mahler
Aimee Goncalves
Stephen McKinley
Juan Aparicio Ojea
Ken Goldberg

Parallel-jaw robot grippers can grasp almost any object and are ubiquitous in industry. Although the shape, texture, and compliance of gripper jaw surfaces affect grasp robustness, almost all commercially available grippers provide a pair of rectangular, planar, rigid jaw surfaces. Practitioners often modify these surfaces with a variety of ad-hoc methods such as adding rubber caps and/or wrapping with textured tape. This paper explores data-driven optimization of gripper jaw surfaces over a design space based on shape, texture, and compliance using rapid prototyping. In total, 37 jaw surface design variations were created using 3D printed casting molds and silicon rubber. The designs were evaluated with 1377 physical grasp experiments using a 4-axis robot (with automated reset). These tests evaluate grasp robustness as the probability that the jaws will acquire, lift, and hold a training set of objects at nominal grasp configurations computed by Dex-Net 1. 0. Hill-climbing in parameter space yielded a grid pattern of 0. 03 inch void depth and 0. 0375 inch void width on a silicone polymer with durometer of A30. We then evaluated performance of this design using an ABB YuMi robot grasping a set of eight difficult-to-grasp 3D printed objects in 80 grasps with four gripper surfaces. The factory-provided gripper tips succeeded in 28. 7% of the 80 trials, increasing to 68. 7% when the tips were wrapped with tape. Gripper tips with gecko-inspired surfaces succeeded in 80. 0% of trials, and gripper tips with the designed silicone surfaces succeeded in 93. 7% of trials.

Details