Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification

Multi-label text classification (MLTC) involves tagging a document with its most relevant subset of labels from a label set. In real applications, labels usually follow a long-tailed distribution, where most labels (called as tail-label) only contain a small number of documents and limit the performance of MLTC. To facilitate this low-resource problem, researchers introduced a simple but effective strategy, data augmentation (DA). However, most existing DA approaches struggle in multi-label settings. The main reason is that the augmented documents for one label may inevitably influence the other co-occurring labels and further exaggerate the long-tailed problem. To mitigate this issue, we propose a new pair-level augmentation framework for MLTC, called Label-Specific Feature Augmentation (LSFA), which merely augments positive feature-label pairs for the tail-labels. LSFA contains two main parts. The first is for label-specific document representation learning in the high-level latent space, the second is for augmenting tail-label features in latent space by transferring the documents second-order statistics (intra-class semantic variations) from head labels to tail labels. At last, we design a new loss function for adjusting classifiers based on augmented datasets. The whole learning procedure can be effectively trained. Comprehensive experiments on benchmark datasets have shown that the proposed LSFA outperforms the state-of-the-art counterparts.

Authors

• Pengyu Xu Beijing Jiaotong University
• Lin Xiao Beijing Jiaotong University
• Bing Liu
• Sijin Lu Beijing Jiaotong University
• Liping Jing Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University
• Jian Yu Southeast University

Keywords

• ML: Classification and Regression
• ML: Deep Generative Models & Autoencoders
• ML: Deep Neural Network Algorithms
• ML: Multi-class/Multi-label Learning & Extreme Classification
• ML: Representation Learning
• ML: Transfer, Domain Adaptation, Multi-Task Learning
• ML: Unsupervised & Self-Supervised Learning
• SNLP: Sentiment Analysis and Stylistic Analysis
• SNLP: Text Classification