Author name cluster

Qingyang Wu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

NeurIPS Conference 2023 Conference Paper

Visual Instruction Tuning

Haotian Liu
Chunyuan Li
Qingyang Wu
Yong Jae Lee

Instruction tuning large language models (LLMs) using machine-generated instruction-following data has been shown to improve zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field. We present the first attempt to use language-only GPT-4 to generate multimodal language-image instruction-following data. By instruction tuning on such generated data, we introduce LLaVA: Large Language and Vision Assistant, an end-to-end trained large multimodal model that connects a vision encoder and an LLM for general-purpose visual and language understanding. To facilitate future research on visual instruction following, we construct two evaluation benchmarks with diverse and challenging application-oriented tasks. Our experiments show that LLaVA demonstrates impressive multimodal chat abilities, sometimes exhibiting the behaviors of multimodal GPT-4 on unseen images/instructions, and yields a 85. 1% relative score compared with GPT-4 on a synthetic multimodal instruction-following dataset. When fine-tuned on Science QA, the synergy of LLaVA and GPT-4 achieves a new state-of-the-art accuracy of 92. 53%. We make GPT-4 generated visual instruction tuning data, our model, and code publicly available.

PDF Details

AAAI Conference 2021 Conference Paper

Perception Score: A Learned Metric for Open-ended Text Generation Evaluation

Jing Gu
Qingyang Wu
Zhou Yu

Automatic evaluation for open-ended natural language generation tasks remains a challenge. We propose a learned evaluation metric: Perception Score. It utilizes a pre-trained model and considers context information for conditional generation. Perception Score assigns a holistic score along with uncertainty measurement. We conduct experiments on three openended conditional generation tasks and two open-ended unconditional generation tasks. Perception Score achieves stateof-the-art results on all the tasks consistently in terms of correlation with human evaluation scores.

PDF Details

AAAI Conference 2021 Conference Paper

TextGAIL: Generative Adversarial Imitation Learning for Text Generation

Qingyang Wu
Lei Li
Zhou Yu

Generative Adversarial Networks (GANs) for text generation have recently received many criticisms, as they perform worse than their MLE counterparts (Caccia et al. 2020; Tevet et al. 2019; Semeniuta, Severyn, and Gelly 2018). We suspect previous text GANs’ inferior performance is due to the lack of a reliable guiding signal in their discriminators. To address this problem, we propose a generative adversarial imitation learning framework for text generation that uses large pre-trained language models to provide more reliable reward guidance. As previous text GANs suffer from high variance of gradients, we apply contrastive discriminator, and proximal policy optimization (PPO) to stabilize and improve text generation performance. For evaluation, we conduct experiments on a diverse set of unconditional and conditional text generation tasks. Experimental results show that TextGAIL achieves better performance in terms of both quality and diversity than the MLE baseline. We also validate our intuition that TextGAIL’s discriminator demonstrates the capability of providing reasonable rewards with an additional task.

PDF Details

AAAI Conference 2020 Conference Paper

Importance-Aware Learning for Neural Headline Editing

Qingyang Wu
Lei Li
Hao Zhou
Ying Zeng
Zhou Yu

Many social media news writers are not professionally trained. Therefore, social media platforms have to hire professional editors to adjust amateur headlines to attract more readers. We propose to automate this headline editing process through neural network models to provide more immediate writing support for these social media news writers. To train such a neural headline editing model, we collected a dataset which contains articles with original headlines and professionally edited headlines. However, it is expensive to collect a large number of professionally edited headlines. To solve this low-resource problem, we design an encoder-decoder model which leverages large scale pre-trained language models. We further improve the pre-trained model’s quality by introducing a headline generation task as an intermediate task before the headline editing task. Also, we propose Self Importance- Aware (SIA) loss to address the different levels of editing in the dataset by down-weighting the importance of easily classiﬁed tokens and sentences. With the help of Pre-training, Adaptation, and SIA, the model learns to generate headlines in the professional editor’s style. Experimental results show that our method signiﬁcantly improves the quality of headline editing comparing against previous methods.

PDF Details