Author name cluster

Yunfeng Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

2 author rows

AAAI Conference 2021 Conference Paper

ACT: an Attentive Convolutional Transformer for Efficient Text Classification

Pengfei Li
Peixiang Zhong
Kezhi Mao
Dongzhe Wang
Xuefeng Yang
Yunfeng Liu
Jianxiong Yin
Simon See

Recently, Transformer has been demonstrating promising performance in many NLP tasks and showing a trend of replacing Recurrent Neural Network (RNN). Meanwhile, less attention is drawn to Convolutional Neural Network (CNN) due to its weak ability in capturing sequential and longdistance dependencies, although it has excellent local feature extraction capability. In this paper, we introduce an Attentive Convolutional Transformer (ACT) that takes the advantages of both Transformer and CNN for efficient text classification. Specifically, we propose a novel attentive convolution mechanism that utilizes the semantic meaning of convolutional filters attentively to transform text from complex word space to a more informative convolutional filter space where important n-grams are captured. ACT is able to capture both local and global dependencies effectively while preserving sequential information. Experiments on various text classification tasks and detailed analyses show that ACT is a lightweight, fast, and effective universal text classifier, outperforming CNNs, RNNs, and attentive models including Transformer.

PDF Details

ECAI Conference 2020 Conference Paper

Derivative-Free Optimization with Adaptive Experience for Efficient Hyper-Parameter Tuning

Yi-Qi Hu
Zelin Liu
Hua Yang
Yang Yu 0001
Yunfeng Liu

Hyper-parameter tuning is a core part of automatic machine learning (AutoML), which aims to automatically configure machine learning systems in deployed applications. Previously, hyper-parameter tuning is usually formulated as a black-box optimization problem, for which derivative-free optimization (DFO) solver is often employed. Such solvers often suffered from low-efficiency. Thus experienced DFO was proposed, which utilizes historical optimization process data to guide the optimization on new problems. However, the effectiveness of experienced DFO is sensitive to the relevance between the experienced tasks and the target tasks. Relevant experience can accelerate the convergence, while irrelevant experience could injure the convergence. This paper proposes an adaptation mechanism for the experienced DFO. It learns a set of experience models to guide the DFO processes, and exams these models on a few labeled samples from the target task. By comparing model predictions with the ground-truth labels, it adaptively learns the relevant experience by weighting those models. The experiments on synthetic tasks verify that the proposed method can effectively adopt the relevant experience for a range of target tasks. Furthermore, we apply the proposed method to the tasks of configuring LightGBM hyper-parameters. The empirical results show that the proposed method effectively selects the relevant experience and significantly improves the performance of hyper-parameter tuning in only a few iterations.

Details