Author name cluster

Hao Liu 0028

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

ICML Conference 2022 Conference Paper

Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint

Hao Liu 0028
Minshuo Chen
Siawpeng Er
Wenjing Liao
Tong Zhang 0001
Tuo Zhao

Overparameterized neural networks enjoy great representation power on complex data, and more importantly yield sufficiently smooth output, which is crucial to their generalization and robustness. Most existing function approximation theories suggest that with sufficiently many parameters, neural networks can well approximate certain classes of functions in terms of the function value. The neural network themselves, however, can be highly nonsmooth. To bridge this gap, we take convolutional residual networks (ConvResNets) as an example, and prove that large ConvResNets can not only approximate a target function in terms of function value, but also exhibit sufficient first-order smoothness. Moreover, we extend our theory to approximating functions supported on a low-dimensional manifold. Our theory partially justifies the benefits of using deep and wide networks in practice. Numerical experiments on adversarial robust image classification are provided to support our theory.

Details

ICML Conference 2021 Conference Paper

Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks

Hao Liu 0028
Minshuo Chen
Tuo Zhao
Wenjing Liao

Most of existing statistical theories on deep neural networks have sample complexities cursed by the data dimension and therefore cannot well explain the empirical success of deep learning on high-dimensional data. To bridge this gap, we propose to exploit the low-dimensional structures of the real world datasets and establish theoretical guarantees of convolutional residual networks (ConvResNet) in terms of function approximation and statistical recovery for binary classification problem. Specifically, given the data lying on a $d$-dimensional manifold isometrically embedded in $\mathbb{R}^D$, we prove that if the network architecture is properly chosen, ConvResNets can (1) approximate {\it Besov functions} on manifolds with arbitrary accuracy, and (2) learn a classifier by minimizing the empirical logistic risk, which gives an {\it excess risk} in the order of $n^{-\frac{s}{2s+2(s\vee d)}}$, where $s$ is a smoothness parameter. This implies that the sample complexity depends on the intrinsic dimension $d$, instead of the data dimension $D$. Our results demonstrate that ConvResNets are adaptive to low-dimensional structures of data sets.

Details

IROS Conference 2019 Conference Paper

Robotic Tracking Control with Kernel Trick-based Reinforcement Learning

Yazhou Hu
Wenxue Wang
Hao Liu 0028
Lianqing Liu

In recent years, reinforcement learning has been developed dramatically and is widely used to solve control problems, e. g. , playing games. However, there are still some problems for reinforcement learning to perform robotic control tasks. Fortunately, the kernel trick-based methods provide a chance to deal with those challenges. This work aims at developing a kernel trick-based learning control method to carry out robotic tracking control tasks. A reward system, in this work, is presented in order to speed up the learning processes. And then, a kernel trick-based reinforcement learning tracking controller is presented to perform tracking control tasks on a robotic manipulator system. To evaluate the policy and assist the reward system to accelerate the speed of finding the optimal control policy, a critic system is introduced. Finally, from the comparison with the benchmark, the simulation results illustrate that our algorithm has faster convergence rate and can execute tracking control tasks effectively, the reward function and the critic system proposed in this work is efficient.

Details