Author name cluster

Wenyan Yang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

1 author row

ICML Conference 2024 Conference Paper

Probabilistic Subgoal Representations for Hierarchical Reinforcement Learning

Vivienne Huiling Wang
Tinghuai Wang
Wenyan Yang
Joni-Kristian Kämäräinen
Joni Pajarinen

In goal-conditioned hierarchical reinforcement learning (HRL), a high-level policy specifies a subgoal for the low-level policy to reach. Effective HRL hinges on a suitable subgoal representation function, abstracting state space into latent subgoal space and inducing varied low-level behaviors. Existing methods adopt a subgoal representation that provides a deterministic mapping from state space to latent subgoal space. Instead, this paper utilizes Gaussian Processes (GPs) for the first probabilistic subgoal representation. Our method employs a GP prior on the latent subgoal space to learn a posterior distribution over the subgoal representation functions while exploiting the long-range correlation in the state space through learnable kernels. This enables an adaptive memory that integrates long-range subgoal information from prior planning steps allowing to cope with stochastic uncertainties. Furthermore, we propose a novel learning objective to facilitate the simultaneous learning of probabilistic subgoal representations and policies within a unified framework. In experiments, our approach outperforms state-of-the-art baselines in standard benchmarks but also in environments with stochastic elements and under diverse reward conditions. Additionally, our model shows promising capabilities in transferring low-level policies across different tasks.

Details

ICRA Conference 2023 Conference Paper

Seq2Seq Imitation Learning for Tactile Feedback-based Manipulation

Wenyan Yang
Alexandre Angleraud
Roel Pieters
Joni Pajarinen
Joni-Kristian Kämäräinen

Robot control for tactile feedback based manip-ulation can be difficult due to modeling of physical contacts, partial observability of the environment, and noise in perception and control. This work focuses on solving partial observability of contact-rich manipulation tasks as a Sequence-to-Sequence (Seq2Seq) Imitation Learning (IL) problem. The proposed Seq2Seq model first produces a robot-environment interaction sequence to estimate the partially observable environment state variables, and then, the observed interaction sequence is transformed to a control sequence for the task itself. The proposed Seq2Seq IL for tactile feedback based manipulation is experimentally validated on a door-open task in a simulated environment and a snap-on insertion task with a real robot. The model is able to learn both tasks from only 50 expert demonstrations while state-of-the-art reinforcement learning and imitation learning methods fail.

Details

IROS Conference 2021 Conference Paper

Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning

Atakan Dag
Alexandre Angleraud
Wenyan Yang
Nataliya Strokina
Roel Pieters
Minna Lanz
Joni-Kristian Kämäräinen

Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.

Details

ICRA Conference 2021 Conference Paper

Neural Network Controller for Autonomous Pile Loading Revised

Wenyan Yang
Nataliya Strokina
Nikolay Serbenyuk
Joni Pajarinen
Reza Ghabcheloo
Juho Vihonen
Mohammad M. Aref
Joni-Kristian Kämäräinen

We have recently proposed two pile loading controllers that learn from human demonstrations: a neural network (NNet) [1] and a random forest (RF) controller [2]. In the field experiments the RF controller obtained clearly better success rates. In this work, the previous findings are drastically revised by experimenting summer time trained controllers in winter conditions. The winter experiments revealed a need for additional sensors, more training data, and a controller that can take advantage of these. Therefore, we propose a revised neural controller (NNetV2) which has a more expressive structure and uses a neural attention mechanism to focus on important parts of the sensor and control signals. Using the same data and sensors to train and test the three controllers, NNetV2 achieves better robustness against drastically changing conditions and superior success rate. To the best of our knowledge, this is the first work testing a learning-based controller for a heavy-duty machine in drastically varying outdoor conditions and delivering high success rate in winter, being trained in summer.

Details

ICRA Conference 2020 Conference Paper

Learning a Pile Loading Controller from Demonstrations

Wenyan Yang
Nataliya Strokina
Nikolay Serbenyuk
Reza Ghabcheloo
Joni-Kristian Kämäräinen

This work introduces a learning-based pile loading controller for autonomous robotic wheel loaders. Controller parameters are learnt from a small number of demonstrations for which low level sensor (boom angle, bucket angle and hydrostatic driving pressure), egocentric video frames and control signals are recorded. Application specific deep visual features are learnt from demonstrations using a Siamese network architecture and a combination of cross-entropy and contrastive loss. The controller is based on a Random Forest (RF) regressor that provides robustness against changes in field conditions (loading distance, soil type, weather and illumination). The controller is deployed to a real autonomous robotic wheel loader and it outperforms prior art with a clear margin.

Details