Shoushan Li

ECAI Conference 2023 Conference Paper

Comment-Aware Multi-Modal Heterogeneous Pre-Training for Humor Detection in Short-Form Videos

• Yang Liu 0358
• Huanqin Ping
• Dong Zhang 0013
• Qingying Sun
• Shoushan Li
• Guodong Zhou

Conventional humor analysis normally focuses on text, text-image pair, and even long video (e. g. , monologue) scenarios. However, with the recent rise of short-form video sharing, humor detection in this scenario has not yet gained much exploration. To the best of our knowledge, there are two primary issues associated with short-form video humor detection (SVHD): 1) At present, there are no ready-made humor annotation samples in this scenario, and it takes a lot of manpower and material resources to obtain a large number of annotation samples; 2) Unlike the more typical audio and visual modalities, the titles (as opposed to simultaneous transcription in the lengthy film) and associated interactive comments in short-form videos may convey apparent humorous clues. Therefore, in this paper, we first collect and annotate a video dataset from DouYin (aka. TikTok in the world), namely DY24h, with hierarchical comments. Then, we also design a novel approach with comment-aided multi-modal heterogeneous pre-training (CMHP) to introduce comment modality in SVHD. Extensive experiments and analysis demonstrate that our CMHP beats several existing video-based approaches on DY24h, and that the comments modality further aids a better comprehension of humor. Our dataset, code and pre-trained models are available at https: //github. com/yliu-cs/CMHP.

IJCAI Conference 2022 Conference Paper

Aspect-based Sentiment Analysis with Opinion Tree Generation

• Xiaoyi Bao
• Wang Zhongqing
• Xiaotong Jiang
• Rong Xiao
• Shoushan Li

Existing studies usually extract these sentiment elements by decomposing the complex structure prediction task into multiple subtasks. Despite their effectiveness, these methods ignore the semantic structure in ABSA problems and require extensive task-specific designs. In this study, we introduce an Opinion Tree Generation task, which aims to jointly detect all sentiment elements in a tree. We believe that the opinion tree can reveal a more comprehensive and complete aspect-level sentiment structure. Furthermore, we propose a pre-trained model to integrate both syntax and semantic features for opinion tree generation. On one hand, a pre-trained model with large-scale unlabeled data is important for the tree generation model. On the other hand, the syntax and semantic features are very effective for forming the opinion tree structure. Extensive experiments show the superiority of our proposed method. The results also validate the tree structure is effective to generate sentimental elements.

AAAI Conference 2022 Conference Paper

KATG: Keyword-Bias-Aware Adversarial Text Generation for Text Classification

• Lingfeng Shen
• Shoushan Li
• Ying Chen

Recent work has shown that current text classification models are vulnerable to a small adversarial perturbation on inputs, and adversarial training that re-trains the models with the support of adversarial examples is the most popular way to alleviate the impact of the perturbation. However, current adversarial training methods have two principal problems: a drop in model’s generalization and ineffective defending against other text attacks. In this paper, we propose a Keywordbias-aware Adversarial Text Generation model (KATG) that implicitly generates adversarial sentences using a generatordiscriminator structure. Instead of using a benign sentence to generate an adversarial sentence, the KATG model utilizes extra multiple benign sentences (namely prior sentences) to guide adversarial sentence generation. Furthermore, to cover more perturbations used in existing attacks, a keyword-biasbased sampling is proposed to select sentences containing biased words as prior sentences. Besides, to effectively utilize prior sentences, a generative flow mechanism is proposed to construct a latent semantic space for learning a latent representation of the prior sentences. Experiments demonstrate that adversarial sentences generated by our KATG model can strengthen the generalization and the robustness of text classification models. Benign Sentence Sixthreezero is good, I’ve used it for a long time, only changed because I got tired of the same old bike. (Pos) Prior Sentences S1: Blackberry may work on the systems, but I’m not willing to take that chance on a new expensive phone. (Neg) S2: Iphone4s is in ok previously used condition as stated. But I was disappointed I couldn’t activate the phone upon arrival. (Neg) Adv. Sentence Amazing Iphone4s, used it for so long, only changed because I got tired of the old expensive Blackberry. (Pos) Table 1: Benign sentence, prior sentences and adversarial sentence used in our KATG model. *the corresponding author Copyright © 2022, Association for the Advancement of Artificial Intelligence (www. aaai. org). All rights reserved.

AAAI Conference 2021 Conference Paper

Multi-modal Graph Fusion for Named Entity Recognition with Targeted Visual Guidance

• Dong Zhang
• Suzhong Wei
• Shoushan Li
• Hanqian Wu
• Qiaoming Zhu
• Guodong Zhou

Multi-modal named entity recognition (MNER) aims to discover named entities in free text and classify them into predefined types with images. However, dominant MNER models do not fully exploit fine-grained semantic correspondences between semantic units of different modalities, which have the potential to refine multi-modal representation learning. To deal with this issue, we propose a unified multi-modal graph fusion (UMGF) approach for MNER. Specifically, we first represent the input sentence and image using a unified multi-modal graph, which captures various semantic relationships between multi-modal semantic units (words and visual objects). Then, we stack multiple graph-based multi-modal fusion layers that iteratively perform semantic interactions to learn node representations. Finally, we achieve an attentionbased multi-modal representation for each word and perform entity labeling with a CRF decoder. Experimentation on the two benchmark datasets demonstrates the superiority of our MNER model.

AAAI Conference 2021 Conference Paper

Multi-modal Multi-label Emotion Recognition with Heterogeneous Hierarchical Message Passing

• Dong Zhang
• Xincheng Ju
• Wei Zhang
• Junhui Li
• Shoushan Li
• Qiaoming Zhu
• Guodong Zhou

As an important research issue in affective computing community, multi-modal emotion recognition has become a hot topic in the last few years. However, almost all existing studies perform multiple binary classification for each emotion with focus on complete time series data. In this paper, we focus on multi-modal emotion recognition in a multilabel scenario. In this scenario, we consider not only the label-to-label dependency, but also the feature-to-label and modality-to-label dependencies. Particularly, we propose a heterogeneous hierarchical message passing network to effectively model above dependencies. Furthermore, we propose a new multi-modal multi-label emotion dataset based on partial time-series content to show predominant generalization of our model. Detailed evaluation demonstrates the effectiveness of our approach.

AAAI Conference 2020 Conference Paper

Sentiment Classification in Customer Service Dialogue with Topic-Aware Multi-Task Learning

• Jiancheng Wang
• Jingjing Wang
• Changlong Sun
• Shoushan Li
• Xiaozhong Liu
• Luo Si
• Min Zhang
• Guodong Zhou

Sentiment analysis in dialogues plays a critical role in dialogue data analysis. However, previous studies on sentiment classification in dialogues largely ignore topic information, which is important for capturing overall information in some types of dialogues. In this study, we focus on the sentiment classification task in an important type of dialogue, namely customer service dialogue, and propose a novel approach which captures overall information to enhance the classification performance. Specifically, we propose a topic-aware multi-task learning (TML) approach which learns topicenriched utterance representations in customer service dialogue by capturing various kinds of topic information. In the experiment, we propose a large-scale and high-quality annotated corpus for the sentiment classification task in customer service dialogue and empirical studies on the proposed corpus show that our approach significantly outperforms several strong baselines.

IJCAI Conference 2019 Conference Paper

Modeling both Context- and Speaker-Sensitive Dependence for Emotion Detection in Multi-speaker Conversations

• Dong Zhang
• Liangqing Wu
• Changlong Sun
• Shoushan Li
• Qiaoming Zhu
• Guodong Zhou

Recently, emotion detection in conversations becomes a hot research topic in the Natural Language Processing community. In this paper, we focus on emotion detection in multi-speaker conversations instead of traditional two-speaker conversations in existing studies. Different from non-conversation text, emotion detection in conversation text has one specific challenge in modeling the context-sensitive dependence. Besides, emotion detection in multi-speaker conversations endorses another specific challenge in modeling the speaker-sensitive dependence. To address above two challenges, we propose a conversational graph-based convolutional neural network. On the one hand, our approach represents each utterance and each speaker as a node. On the other hand, the context-sensitive dependence is represented by an undirected edge between two utterances nodes from the same conversation and the speaker-sensitive dependence is represented by an undirected edge between an utterance node and its speaker node. In this way, the entire conversational corpus can be symbolized as a large heterogeneous graph and the emotion detection task can be recast as a classification problem of the utterance nodes in the graph. The experimental results on a multi-modal and multi-speaker conversation corpus demonstrate the great effectiveness of the proposed approach.

IJCAI Conference 2019 Conference Paper

Modeling Source Syntax and Semantics for Neural AMR Parsing

• DongLai Ge
• Junhui Li
• Muhua Zhu
• Shoushan Li

Sequence-to-sequence (seq2seq) approaches formalize Abstract Meaning Representation (AMR) parsing as a translation task from a source sentence to a target AMR graph. However, previous studies generally model a source sentence as a word sequence but ignore the inherent syntactic and semantic information in the sentence. In this paper, we propose two effective approaches to explicitly modeling source syntax and semantics into neural seq2seq AMR parsing. The first approach linearizes source syntactic and semantic structure into a mixed sequence of words, syntactic labels, and semantic labels, while in the second approach we propose a syntactic and semantic structure-aware encoding scheme through a self-attentive model to explicitly capture syntactic and semantic relations between words. Experimental results on an English benchmark dataset show that our two approaches achieve significant improvement of 3. 1% and 3. 4% F1 scores over a strong seq2seq baseline.

IJCAI Conference 2018 Conference Paper

Aspect Sentiment Classification with both Word-level and Clause-level Attention Networks

• Jingjing Wang
• Jie Li
• Shoushan Li
• Yangyang Kang
• Min Zhang
• Luo Si
• Guodong Zhou

Aspect sentiment classification, a challenging task in sentiment analysis, has been attracting more and more attention in recent years. In this paper, we highlight the need for incorporating the importance degrees of both words and clauses inside a sentence and propose a hierarchical network with both word-level and clause-level attentions to aspect sentiment classification. Specifically, we first adopt sentence-level discourse segmentation to segment a sentence into several clauses. Then, we leverage multiple Bi-directional LSTM layers to encode all clauses and propose a word-level attention layer to capture the importance degrees of words in each clause. Third and finally, we leverage another Bi-directional LSTM layer to encode the outputs from the former layers and propose a clause-level attention layer to capture the importance degrees of all the clauses inside a sentence. Experimental results on the laptop and restaurant datasets from SemEval-2015 demonstrate the effectiveness of our proposed approach to aspect sentiment classification.

IJCAI Conference 2017 Conference Paper

Joint Learning on Relevant User Attributes in Micro-blog

• Jingjing Wang
• Shoushan Li
• Guodong Zhou

User attribute classification aims to identify users’ attributes (e. g. , gender, age and profession) by leveraging user generated content. However, conventional approaches to user attribute classification focus on single attribute classification involving only one user attribute, which completely ignores the relationship among various user attributes. In this paper, we confront a novel scenario in user attribute classification where relevant user attributes are jointly learned, attempting to make the relevant attribute classification tasks help each other. Specifically, we propose a joint learning approach, namely Aux-LSTM, which first learns a proper auxiliary representation between the related tasks and then leverages the auxiliary representation to integrate the learning process in both tasks. Empirical studies demonstrate the effectiveness of our proposed approach to joint learning on relevant user attributes.

IJCAI Conference 2015 Conference Paper

Interactive Gender Inference with Integer Linear Programming

• Shoushan Li
• Jingjing Wang
• Guodong Zhou
• Hanxiao Shi

IJCAI Conference 2013 Conference Paper

Active Learning for Cross-Domain Sentiment Classification

• Shoushan Li
• Yunxia Xue
• Zhongqing Wang
• Guodong Zhou

In the literature, various approaches have been proposed to address the domain adaptation problem in sentiment classification (also called cross-domain sentiment classification). However, the adaptation performance normally much suffers when the data distributions in the source and target domains differ significantly. In this paper, we suggest to perform active learning for cross-domain sentiment classification by actively selecting a small amount of labeled data in the target domain. Accordingly, we propose an novel active learning approach for cross-domain sentiment classification. First, we train two individual classifiers, i. e. , the source and target classifiers with the labeled data from the source and target respectively. Then, the two classifiers are employed to select informative samples with the selection strategy of Query By Committee (QBC). Third, the two classifier is combined to make the classification decision. Importantly, the two classifiers are trained by fully exploiting the unlabeled data in the target domain with the label propagation (LP) algorithm. Empirical studies demonstrate the effectiveness of our active learning approach for cross-domain sentiment classification over some strong baselines. 1

AAAI Conference 2012 Conference Paper

Opinion Target Extraction Using a Shallow Semantic Parsing Framework

• Shoushan Li
• Rongyang Wang
• Guodong Zhou

IJCAI Conference 2011 Conference Paper

Semi-Supervised Learning for Imbalanced Sentiment Classification

• Shoushan Li
• Zhongqing Wang
• Guodong Zhou
• Sophia Yat Mei Lee

Various semi-supervised learning methods have been proposed recently to solve the long-standing shortage problem of manually labeled data in sentiment classification. However, most existing studies assume the balance between negative and positive samples in both the labeled and unlabeled data, which may not be true in reality. In this paper, we investigate a more common case of semi-supervised learning for imbalanced sentiment classification. In particular, various random subspaces are dynamically generated to deal with the imbalanced class distribution problem. Evaluation across four domains shows the effectiveness of our approach.