Author name cluster

Zilin Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

RLC Conference 2025 Conference Paper

How Should We Meta-Learn Reinforcement Learning Algorithms?

Alexander David Goldie
Zilin Wang
Jaron Cohen
Jakob Nicolaus Foerster
Shimon Whiteson

The process of meta-learning algorithms from data, instead of relying on manual design, is growing in popularity as a paradigm for improving the performance of machine learning systems. Meta-learning shows particular promise for reinforcement learning (RL), where algorithms are often adapted from supervised or unsupervised learning despite their suboptimality for RL. However, until now there has been a severe lack of comparison between different meta-learning algorithms, such as using evolution to optimise over black-box functions or LLMs to propose code. In this paper, we carry out this empirical comparison of the different approaches when applied to a range of meta-learned algorithms, which each target different parts of the RL pipeline. In addition to meta-train and meta-test performance, we also investigate factors including the interpretability, sample cost and train time for each meta-learning algorithm. Based on these findings, we propose several guidelines for meta-learning new RL algorithms which will help ensure that future learned algorithms are as performant as possible.

PDF Details

RLJ Journal 2025 Journal Article

How Should We Meta-Learn Reinforcement Learning Algorithms?

Alexander David Goldie
Zilin Wang
Jaron Cohen
Jakob Nicolaus Foerster
Shimon Whiteson

PDF Details

IROS Conference 2024 Conference Paper

Absolute Pose Estimation for a Millimeter-Scale Vision System

Derin Ozturk
Zilin Wang
E. Farrell Helbling

Vision is an important component of robotic perception systems due to the rich information provided by high resolution image sensors, but computer vision algorithms can be computationally expensive and ill-suited to resource-constrained robotic systems. Here, we present a mm-scale vision system capable of performing absolute pose estimation at 16. 5 FPS. This novel vision system uses a commercial-off-the-shelf sensor and microcontroller unit, as well as planar light-based landmarks in the environment to simplify feature detection. We exploit the structure of the planar pose problem to reduce algorithmic complexity and improve latency and energy consumption through software-, processor-, and hardware-in-the-loop testing. The end-to-end system consumes 49 mA of current and computes absolute pose estimates within 15 mm over a number of reference trajectories.

Details

AAAI Conference 2024 Conference Paper

Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrations

Zilin Wang
Haolin Zhuang
Lu Li
Yinmin Zhang
Junjie Zhong
Jun Chen
Yu Yang
Boshi Tang

This paper presents an Exploratory 3D Dance generation framework, E3D2, designed to address the exploration capability deficiency in existing music-conditioned 3D dance generation models. Current models often generate monotonous and simplistic dance sequences that misalign with human preferences because they lack exploration capabilities.The E3D2 framework involves a reward model trained from automatically-ranked dance demonstrations, which then guides the reinforcement learning process. This approach encourages the agent to explore and generate high quality and diverse dance movement sequences. The soundness of the reward model is both theoretically and experimentally validated. Empirical experiments demonstrate the effectiveness of E3D2 on the AIST++ dataset.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Improving Generalization and Convergence by Enhancing Implicit Regularization

Mingze Wang
Jinbo Wang
Haotian He
Zilin Wang
Guanhua Huang
Feiyu Xiong
Zhiyu Li
Weinan E

In this work, we propose an Implicit Regularization Enhancement (IRE) framework to accelerate the discovery of flat solutions in deep learning, thereby improving generalization and convergence. Specifically, IRE decouples the dynamics of flat and sharp directions, which boosts the sharpness reduction along flat directions while maintaining the training stability in sharp directions. We show that IRE can be practically incorporated with *generic base optimizers* without introducing significant computational overload. Experiments show that IRE consistently improves the generalization performance for image classification tasks across a variety of benchmark datasets (CIFAR-10/100, ImageNet) and models (ResNets and ViTs). Surprisingly, IRE also achieves a $2\times$ *speed-up* compared to AdamW in the pre-training of Llama models (of sizes ranging from 60M to 229M) on datasets including Wikitext-103, Minipile, and Openwebtext. Moreover, we provide theoretical guarantees, showing that IRE can substantially accelerate the convergence towards flat minima in Sharpness-aware Minimization (SAM).

PDF Details DOI

AAMAS Conference 2024 Conference Paper

Normalization Enhances Generalization in Visual Reinforcement Learning

Lu Li
Jiafei Lyu
Guozheng Ma
Zilin Wang
Zhenjie Yang
Xiu Li
Zhiheng Li

Recent advances in visual reinforcement learning (RL) have led to impressive success in handling complex tasks. However, these methods have demonstrated limited generalization capability to visual disturbances, which poses a significant challenge to their real-world application and adaptability. Though normalization techniques have demonstrated huge success in supervised and unsupervised learning, their applications in visual RL are still scarce. In this paper, we explore the potential benefits of integrating normalization into visual RL methods with respect to generalization performance. We find that, perhaps surprisingly, incorporating suitable normalization techniques is sufficient to enhance the generalization capabilities, without any additional special design. We utilize the combination of two normalization techniques, CrossNorm and SelfNorm, for generalizable visual RL. Extensive experiments are conducted on DMControl Generalization Benchmark, CARLA, and ProcGen Benchmark to validate the effectiveness of our method. We show that our method significantly improves generalization capability while only marginally affecting sample efficiency. In particular, when integrated with DrQ-v2, our method enhances the test performance of DrQ-v2 on CARLA across various scenarios, from 14% of the training performance to 97%. Our project page: https: //sites. google. com/view/norm-generalization-vrl/home

PDF

NeurIPS Conference 2023 Conference Paper

Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

Guozheng Ma
Linrui Zhang
Haoyu Wang
Lu Li
Zilin Wang
Zhen Wang
Li Shen
Xueqian Wang

Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. Notably, employing simple observation transformations alone can yield outstanding performance without extra auxiliary representation tasks or pre-trained encoders. However, it remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. To investigate this issue and further explore the potential of DA, this work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy and provides the following insights and improvements: (1) For individual DA operations, we reveal that both ample spatial diversity and slight hardness are indispensable. Building on this finding, we introduce Random PadResize (Rand PR), a new DA operation that offers abundant spatial diversity with minimal hardness. (2) For multi-type DA fusion schemes, the increased DA hardness and unstable data distribution result in the current fusion schemes being unable to achieve higher sample efficiency than their corresponding individual operations. Taking the non-stationary nature of RL into account, we propose a RL-tailored multi-type DA fusion scheme called Cycling Augmentation (CycAug), which performs periodic cycles of different DA operations to increase type diversity while maintaining data distribution consistency. Extensive evaluations on the DeepMind Control suite and CARLA driving simulator demonstrate that our methods achieve superior sample efficiency compared with the prior state-of-the-art methods.

PDF Details

ICLR Conference 2021 Conference Paper

Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration

Xavier Puig
Tianmin Shu
Shuang Li 0013
Zilin Wang
Yuan-Hong Liao
Joshua B. Tenenbaum
Sanja Fidler
Antonio Torralba 0001

In this paper, we introduce Watch-And-Help (WAH), a challenge for testing social intelligence in agents. In WAH, an AI agent needs to help a human-like agent perform a complex household task efficiently. To succeed, the AI agent needs to i) understand the underlying goal of the task by watching a single demonstration of the human-like agent performing the same task (social perception), and ii) coordinate with the human-like agent to solve the task in an unseen environment as fast as possible (human-AI collaboration). For this challenge, we build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines. We evaluate the performance of AI agents with the human-like agent as well as and with real humans using objective metrics and subjective user ratings. Experimental results demonstrate that our challenge and virtual environment enable a systematic evaluation on the important aspects of machine social intelligence at scale.

Details

AAMAS Conference 2018 Conference Paper

A Decision Theoretic Framework for Emergency Responder Dispatch

Ayan Mukhopadhyay
Zilin Wang
Yevgeniy Vorobeychik

Efficient emergency response is a major concern in urban areas across the globe. The problem of predicting incidents and subsequently allocating responders spatially has been studied extensively. The problem of dynamically deploying responders, however, has received considerably less attention and has been noted as a difficult problem in prior literature due to inherent complexities in the environment in which such problems evolve. We formulate a decisiontheoretic framework for the emergency responder problem, which effectively leverages state-of-the-art methods for continuous-time spatio-temporal incident forecasting. We formulate the responder dispatch problem as a Semi-Markov Decision Process (SMDP) that evolves in continuous time, and efficiently engineer its representation leveraging structural insights of the problem space. We then propose a novel approach to solve the problem based on policy iteration. First, we transform the SMDP into a discrete-time MDP (DTMDP). Then, we simulate our system to estimate value of states as well as learn the state transition probabilities of the transformed DTMDP. We also design heuristic policies with which our algorithm can be seeded. We validate the efficacy of our approach on real traffic and assault data from Nashville, USA, as well as synthetic data, and highlight that our approach outperforms the state of the art emergency responder dispatch system.

PDF