Felix Wu

ICML Conference 2023 Conference Paper

On the Effectiveness of Offline RL for Dialogue Response Generation

• Paloma Sodhi
• Felix Wu
• Ethan R. Elenberg
• Kilian Q. Weinberger
• Ryan McDonald

A common training technique for language models is teacher forcing (TF). TF attempts to match human language exactly, even though identical meanings can be expressed in different ways. This motivates use of sequence-level objectives for dialogue response generation. In this paper, we study the efficacy of various offline reinforcement learning (RL) methods to maximize such objectives. We present a comprehensive evaluation across multiple datasets, models, and metrics. Offline RL shows a clear performance improvement over teacher forcing while not inducing training instability or sacrificing practical training budgets.

ICML Conference 2021 Conference Paper

Making Paper Reviewing Robust to Bid Manipulation Attacks

• Ruihan Wu
• Chuan Guo 0001
• Felix Wu
• Rahul Kidambi
• Laurens van der Maaten
• Kilian Q. Weinberger

Most computer science conferences rely on paper bidding to assign reviewers to papers. Although paper bidding enables high-quality assignments in days of unprecedented submission numbers, it also opens the door for dishonest reviewers to adversarially influence paper reviewing assignments. Anecdotal evidence suggests that some reviewers bid on papers by "friends" or colluding authors, even though these papers are outside their area of expertise, and recommend them for acceptance without considering the merit of the work. In this paper, we study the efficacy of such bid manipulation attacks and find that, indeed, they can jeopardize the integrity of the review process. We develop a novel approach for paper bidding and assignment that is much more robust against such attacks. We show empirically that our approach provides robustness even when dishonest reviewers collude, have full knowledge of the assignment system’s internal workings, and have access to the system’s inputs. In addition to being more robust, the quality of our paper review assignments is comparable to that of current, non-robust assignment approaches.

ICLR Conference 2021 Conference Paper

Revisiting Few-sample BERT Fine-tuning

• Tianyi Zhang 0007
• Felix Wu
• Arzoo Katiyar
• Kilian Q. Weinberger
• Yoav Artzi

This paper is a study of fine-tuning of BERT contextual representations, with focus on commonly observed instabilities in few-sample scenarios. We identify several factors that cause this instability: the common use of a non-standard optimization method with biased gradient estimation; the limited applicability of significant parts of the BERT network for down-stream tasks; and the prevalent practice of using a pre-determined, and small number of training iterations. We empirically test the impact of these factors, and identify alternative practices that resolve the commonly observed instability of the process. In light of these observations, we re-visit recently proposed methods to improve few-sample fine-tuning with BERT and re-evaluate their effectiveness. Generally, we observe the impact of these methods diminishes significantly with our modified process.

ICLR Conference 2020 Conference Paper

BERTScore: Evaluating Text Generation with BERT

• Tianyi Zhang 0007
• Varsha Kishore
• Felix Wu
• Kilian Q. Weinberger
• Yoav Artzi

We propose BERTScore, an automatic evaluation metric for text generation. Analogously to common metrics, BERTScore computes a similarity score for each token in the candidate sentence with each token in the reference sentence. However, instead of exact matches, we compute token similarity using contextual embeddings. We evaluate using the outputs of 363 machine translation and image captioning systems. BERTScore correlates better with human judgments and provides stronger model selection performance than existing metrics. Finally, we use an adversarial paraphrase detection task and show that BERTScore is more robust to challenging examples compared to existing metrics.

ICLR Conference 2019 Conference Paper

Pay Less Attention with Lightweight and Dynamic Convolutions

• Felix Wu
• Angela Fan
• Alexei Baevski
• Yann N. Dauphin
• Michael Auli

Self-attention is a useful mechanism to build generative models for language and images. It determines the importance of context elements by comparing each element to the current time step. In this paper, we show that a very lightweight convolution can perform competitively to the best reported self-attention results. Next, we introduce dynamic convolutions which are simpler and more efficient than self-attention. We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements. The number of operations required by this approach scales linearly in the input length, whereas self-attention is quadratic. Experiments on large-scale machine translation, language modeling and abstractive summarization show that dynamic convolutions improve over strong self-attention models. On the WMT'14 English-German test set dynamic convolutions achieve a new state of the art of 29.7 BLEU.

NeurIPS Conference 2019 Conference Paper

Positional Normalization

• Boyi Li
• Felix Wu
• Kilian Weinberger
• Serge Belongie

A widely deployed method for reducing the training time of deep neural networks is to normalize activations at each layer. Although various normalization schemes have been proposed, they all follow a common theme: normalize across spatial dimensions and discard the extracted statistics. In this paper, we propose a novel normalization method that deviates from this theme. Our approach, which we refer to as Positional Normalization (PONO), normalizes exclusively across channels, which allows us to capture structural information of the input image in the first and second moments. Instead of disregarding this information, we inject it into later layers to preserve or transfer structural information in generative networks. We show that PONO significantly improves the performance of deep networks across a wide range of model architectures and image generation tasks.

ICML Conference 2019 Conference Paper

Simplifying Graph Convolutional Networks

• Felix Wu
• Amauri H. Souza Jr.
• Tianyi Zhang 0007
• Christopher Fifty
• Tao Yu
• Kilian Q. Weinberger

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

ICLR Conference 2018 Conference Paper

Multi-Scale Dense Networks for Resource Efficient Image Classification

• Gao Huang 0001
• Danlu Chen
• Tianhong Li
• Felix Wu
• Laurens van der Maaten
• Kilian Q. Weinberger

In this paper we investigate image classification with computational resource limits at test time. Two such settings are: 1. anytime classification, where the network’s prediction for a test example is progressively updated, facilitating the output of a prediction at any time; and 2. budgeted batch classification, where a fixed amount of computation is available to classify a set of examples that can be spent unevenly across “easier” and “harder” inputs. In contrast to most prior work, such as the popular Viola and Jones algorithm, our approach is based on convolutional neural networks. We train multiple classifiers with varying resource demands, which we adaptively apply during test time. To maximally re-use computation between the classifiers, we incorporate them as early-exits into a single deep convolutional neural network and inter-connect them with dense connectivity. To facilitate high quality classification early on, we use a two-dimensional multi-scale network architecture that maintains coarse and fine level features all-throughout the network. Experiments on three image-classification tasks demonstrate that our framework substantially improves the existing state-of-the-art in both settings.

NeurIPS Conference 2017 Conference Paper

On Fairness and Calibration

• Geoff Pleiss
• Manish Raghavan
• Felix Wu
• Jon Kleinberg
• Kilian Weinberger

The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models. This has motivated a growing line of work on what it means for a classification procedure to be "fair. " In this paper, we investigate the tension between minimizing error disparity across different population groups while maintaining calibrated probability estimates. We show that calibration is compatible only with a single error constraint (i. e. equal false-negatives rates across groups), and show that any algorithm that satisfies this relaxation is no better than randomizing a percentage of predictions for an existing classifier. These unsettling findings, which extend and generalize existing results, are empirically confirmed on several datasets.

JMLR Journal 2015 Journal Article

Combination of Feature Engineering and Ranking Models for Paper-Author Identification in KDD Cup 2013

• Chun-Liang Li
• Yu-Chuan Su
• Ting-Wei Lin
• Cheng-Hao Tsai
• Wei-Cheng Chang
• Kuan-Hao Huang
• Tzu-Ming Kuo
• Shan-Wei Lin

This paper describes the winning solution of team National Taiwan University for track 1 of KDD Cup 2013. The track 1 in KDD Cup 2013 considers the paper-author identification problem, which is to identify whether a paper is truly written by an author. First, we conduct feature engineering to transform the various types of provided text information into 97 features. Second, we train classification and ranking models using these features. Last, we combine our individual models to boost the performance by using results on the internal validation set and the official Valid set. Some effective post-processing techniques have also been proposed. Our solution achieves 0.98259 MAP score and ranks the first place on the private leaderboard of the Test set. [abs] [ pdf ][ bib ] &copy JMLR 2015. ( edit, beta )

JMLR Journal 2014 Journal Article

Effective String Processing and Matching for Author Disambiguation

• Wei-Sheng Chin
• Yong Zhuang
• Yu-Chin Juan
• Felix Wu
• Hsiao-Yu Tung
• Tong Yu
• Jui-Pin Wang
• Cheng-Xia Chang

Track 2 of KDD Cup 2013 aims at determining duplicated authors in a data set from Microsoft Academic Search. This type of problems appears in many large-scale applications that compile information from different sources. This paper describes our solution developed at National Taiwan University to win the first prize of the competition. We propose an effective name matching framework and realize two implementations. An important strategy in our approach is to consider Chinese and non-Chinese names separately because of their different naming conventions. Post-processing including merging results of two predictions further boosts the performance. Our approach achieves F1-score 0.99202 on the private leader board, while 0.99195 on the public leader board. [abs] [ pdf ][ bib ] &copy JMLR 2014. ( edit, beta )

TCS Journal 2004 Journal Article

Online learning in online auctions

• Avrim Blum
• Vijay Kumar
• Atri Rudra
• Felix Wu

We consider the problem of revenue maximization in online auctions, that is, auctions in which bids are received and dealt with one-by-one. In this paper, we demonstrate that results from online learning can be usefully applied in this context, and we derive a new auction for digital goods that achieves a constant competitive ratio with respect to the optimal (offline) fixed price revenue. This substantially improves upon the best previously known competitive ratio for this problem of O(exp( loglog h )). We also apply our techniques to the related problem of designing online posted price mechanisms, in which the seller declares a price for each of a series of buyers, and each buyer either accepts or rejects the good at that price. Despite the relative lack of information in this setting, we show that online learning techniques can be used to obtain results for online posted price mechanisms which are similar to those obtained for online auctions.

SODA Conference 2003 Conference Paper

Online learning in online auctions

• Avrim Blum
• Vijay Kumar
• Atri Rudra
• Felix Wu

SODA Conference 2002 Conference Paper

Incentive-compatible online auctions for digital goods

• Ziv Bar-Yossef
• Kirsten Hildrum
• Felix Wu

STOC Conference 1999 Conference Paper

The Quantum Query Complexity of Approximating the Median and Related Statistics

• Ashwin Nayak 0001
• Felix Wu