Author name cluster

Shaoping Ma

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

14 papers

1 author row

AAAI Conference 2026 Conference Paper

Auto-PRE: An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation

Junjie Chen
Weihang Su
Zhumin Chu
Haitao Li
Yujia Zhou
Dingbo Yuan
Xudong Wang
Jun Zhou

The rapid development of large language models (LLMs) has highlighted the need for efficient and reliable methods to evaluate their performance. Traditional evaluation methods often face challenges like high costs, limited task formats, dependence on human references, and systematic biases. To address these limitations, we propose Auto-PRE, an automatic LLM evaluation framework inspired by the peer review process. Unlike previous approaches that rely on human annotations, Auto-PRE automatically selects evaluator LLMs based on three core traits: consistency, pertinence, and self-confidence, which correspond to the instruction, content, and response stages, respectively, and collectively cover the entire evaluation process. Experiments on three representative tasks, including summarization, non-factoid QA, and dialogue generation, demonstrate that Auto-PRE achieves state-of-the-art performance while significantly reducing evaluation costs. Furthermore, the structured and scalable design of our automatic qualification exam framework provides valuable insights into automating the evaluation of LLMs-as-judges, paving the way for more advanced LLM-based evaluation frameworks.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrieval (Extended Abstract)

Jingtao Zhan
Jiaxin Mao
Yiqun Liu
Jiafeng Guo
Min Zhang
Shaoping Ma

Dense Retrieval~(DR) has achieved state-of-the-art first-stage ranking effectiveness. However, the efficiency of most existing DR models is limited by the large memory cost of storing dense vectors and the time-consuming nearest neighbor search~(NNS) in vector space. Therefore, we present RepCONC, a novel retrieval model that learns discrete Representations via CONstrained Clustering. RepCONC jointly trains dual-encoders and the Product Quantization~(PQ) method to learn discrete document representations and enables fast approximate NNS with compact indexes. It models quantization as a constrained clustering process, which requires the document embeddings to be uniformly clustered around the quantization centroids. We theoretically demonstrate that the uniform clustering constraint facilitates representation distinguishability. Extensive experiments show that RepCONC substantially outperforms a wide range of existing retrieval models in terms of retrieval effectiveness, memory efficiency, and time efficiency.

PDF Details DOI

IJCAI Conference 2022 Conference Paper

Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrieval (Extended Abstract)

Jingtao Zhan
Jiaxin Mao
Yiqun Liu
Jiafeng Guo
Min Zhang
Shaoping Ma

Dense Retrieval (DR) has achieved state-of-the-art first-stage ranking effectiveness. However, the efficiency of most existing DR models is limited by the large memory cost of storing dense vectors and the time-consuming nearest neighbor search (NNS) in vector space. Therefore, we present RepCONC, a novel retrieval model that learns discrete Representations via CONstrained Clustering. RepCONC jointly trains dual-encoders and the Product Quantization (PQ) method to learn discrete document representations and enables fast approximate NNS with compact indexes. It models quantization as a constrained clustering process, which requires the document embeddings to be uniformly clustered around the quantization centroids. We theoretically demonstrate the importance of the uniform clustering constraint and derive an efficient approximate solution for constrained clustering by reducing it to an instance of the optimal transport problem. Extensive experiments on two popular ad-hoc retrieval benchmarks show that RepCONC substantially outperforms a wide range of existing retrieval models in terms of retrieval effectiveness, memory efficiency, and time efficiency.

PDF Details DOI

AAAI Conference 2021 Conference Paper

Graph Heterogeneous Multi-Relational Recommendation

Chong Chen
Weizhi Ma
Min Zhang
Zhaowei Wang
Xiuqiang He
Chenyang Wang
Yiqun Liu
Shaoping Ma

Traditional studies on recommender systems usually leverage only one type of user behaviors (the optimization target, such as purchase), despite the fact that users also generate a large number of various types of interaction data (e. g. , view, click, add-to-cart, etc). Generally, these heterogeneous multirelational data provide well-structured information and can be used for high-quality recommendation. Early efforts towards leveraging these heterogeneous data fail to capture the high-hop structure of user-item interactions, which are unable to make full use of them and may only achieve constrained recommendation performance. In this work, we propose a new multi-relational recommendation model named Graph Heterogeneous Collaborative Filtering (GHCF). To explore the high-hop heterogeneous user-item interactions, we take the advantages of Graph Convolutional Network (GCN) and further improve it to jointly embed both representations of nodes (users and items) and relations for multi-relational prediction. Moreover, to fully utilize the whole heterogeneous data, we perform the advanced efficient non-sampling optimization under a multi-task learning framework. Experimental results on two public benchmarks show that GHCF significantly outperforms the state-of-the-art recommendation methods, especially for cold-start users who have few primary item interactions. Further analysis verifies the importance of the proposed embedding propagation for modelling high-hop heterogeneous user-item interactions, showing the rationality and effectiveness of GHCF. Our implementation has been released (https: //github. com/chenchongthu/GHCF).

PDF Details

IJCAI Conference 2020 Conference Paper

BERT-PLI: Modeling Paragraph-Level Interactions for Legal Case Retrieval

Yunqiu Shao
Jiaxin Mao
Yiqun Liu
Weizhi Ma
Ken Satoh
Min Zhang
Shaoping Ma

Legal case retrieval is a specialized IR task that involves retrieving supporting cases given a query case. Compared with traditional ad-hoc text retrieval, the legal case retrieval task is more challenging since the query case is much longer and more complex than common keyword queries. Besides that, the definition of relevance between a query case and a supporting case is beyond general topical relevance and it is therefore difficult to construct a large-scale case retrieval dataset, especially one with accurate relevance judgments. To address these challenges, we propose BERT-PLI, a novel model that utilizes BERT to capture the semantic relationships at the paragraph-level and then infers the relevance between two cases by aggregating paragraph-level interactions. We fine-tune the BERT model with a relatively small-scale case law entailment dataset to adapt it to the legal scenario and employ a cascade framework to reduce the computational cost. We conduct extensive experiments on the benchmark of the relevant case retrieval task in COLIEE 2019. Experimental results demonstrate that our proposed method outperforms existing solutions.

PDF Details DOI

AAAI Conference 2020 Conference Paper

Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation

Chong Chen
Min Zhang
Yongfeng Zhang
Weizhi Ma
Yiqun Liu
Shaoping Ma

Recent studies on recommendation have largely focused on exploring state-of-the-art neural networks to improve the expressiveness of models, while typically apply the Negative Sampling (NS) strategy for efﬁcient learning. Despite effectiveness, two important issues have not been well-considered in existing methods: 1) NS suffers from dramatic ﬂuctuation, making sampling-based methods difﬁcult to achieve the optimal ranking performance in practical applications; 2) although heterogeneous feedback (e. g. , view, click, and purchase) is widespread in many online systems, most existing methods leverage only one primary type of user feedback such as purchase. In this work, we propose a novel nonsampling transfer learning solution, named Efﬁcient Heterogeneous Collaborative Filtering (EHCF) for Top-N recommendation. It can not only model ﬁne-grained user-item relations, but also efﬁciently learn model parameters from the whole heterogeneous data (including all unlabeled data) with a rather low time complexity. Extensive experiments on three real-world datasets show that EHCF signiﬁcantly outperforms state-of-the-art recommendation methods in both traditional (single-behavior) and heterogeneous scenarios. Moreover, EHCF shows signiﬁcant improvements in training ef- ﬁciency, making it more applicable to real-world large-scale systems. Our implementation has been released 1 to facilitate further developments on efﬁcient whole-data based neural methods.

PDF Details

IJCAI Conference 2018 Conference Paper

Your Tweets Reveal What You Like: Introducing Cross-media Content Information into Multi-domain Recommendation

Weizhi Ma
Min Zhang
Chenyang Wang
Cheng Luo
Yiqun Liu
Shaoping Ma

Cold start is a challenging problem in recommender systems. Many previous studies attempt to utilize extra information from other platforms to alleviate the problem. Most of the leveraged information is on-topic, directly related to users' preferences in the target domain. Thought to be unrelated, users' off-topic content information (such as user tweets) is usually omitted. However, the off-topic content information also helps to indicate the similarity of users on their tastes, interests, and opinions, which matches the underlying assumption of Collaborative Filtering (CF) algorithms. In this paper, we propose a framework to capture the features from user's off-topic content information in social media and introduce them into Matrix Factorization (MF) based algorithms. The framework is easy to understand and flexible in different embedding approaches and MF based algorithms. To the best of our knowledge, there is no previous study in which user's off-topic content in other platforms is taken into consideration. By capturing the cross-platform content including both on-topic and off-topic information, multiple algorithms with several embedding learning approaches have achieved significant improvements in rating prediction on three datasets. Especially in cold start scenarios, we observe greater enhancement. The results confirm our suggestion that off-topic cross-media information also contributes to the recommendation.

PDF Details

IJCAI Conference 2016 Conference Paper

Multi-Grained Role Labeling Based on Multi-Modality Information for Real Customer Service Telephone Conversation

Weizhi Ma
Min Zhang
Yiqun Liu
Shaoping Ma

Large-scale customer service call records include lots of valuable information for business intelligence. However, the analysis of those records has not utilized in the big data era before. There are two fundamental problems before mining and analyses: 1) The telephone conversation is mixed with words of agents and users which have to be recognized before analysis; 2) The speakers in conversation are not in a pre-defined set. These problems are new challenges which have not been well studied in the previous work. In this paper, we propose a four-phase framework for role labeling in real customer service telephone conversation, with the benefit of integrating multi-modality features, i. e. , both low-level acoustic features and semantic-level textual features. Firstly, we conduct Δ Bayesian Information Criterion (Δ BIC) based speaker diarization to get two segments clusters from an audio stream. Secondly, the segments are transferred into text in an Automatic Speech Recognition (ASR) phase with a deep learning model DNN-HMM. Thirdly, by integrating acoustic and textual features, dialog level role labeling is proposed to map the two clusters into the agent and the user. Finally, sentence level role correction is designed in order to label results correctly in a fine-grained notion, which reduces the errors made in previous phases. The proposed framework is tested on two real datasets: mobile and bank customer service calls datasets. The precision of dialog level labeling is over 99. 0%. On the sentence level, the accuracy of labeling reaches 90. 4%, greatly outperforming traditional acoustic features based method which achieves only 78. 5% in accuracy.

PDF Details

IJCAI Conference 2016 Conference Paper

Pay Me and I'll Follow You: Detection of Crowdturfing Following Activities in Microblog Environment

Yuli Liu
Yiqun Liu
Min Zhang
Shaoping Ma

A number of existing works have focused on the problem of malicious following activity detection in microblog services. However, most of them make the assumption that the spamming following relationships are either from fraudulent accounts or compromised legitimate users. They therefore developed detection methodologies based on the features derived from this assumption. Recently, a new type of malicious crowdturfing following relationship is provided by the follower market, called voluntary following. Followers who provide voluntary following services (or named volowers) are normal users who are willing to trade their following activities for profit. Since most of their behaviors follow normal patterns, it is difficult for existing methods to detect volowers and their corresponding customers. In this work, we try to solve the voluntary following problem through a newly proposed detection method named DetectVC. This method incorporates both structure information in user following behavior graphs and prior knowledge collected from follower markets. Experimental results on large scale practical microblog data set show that DetectVC is able to detect volowers and their customers simultaneously and it also significantly outperforms existing solutions.

PDF Details

IJCAI Conference 2016 Conference Paper

Rating-Boosted Latent Topics: Understanding Users and Items with Ratings and Reviews

Yunzhi Tan
Min Zhang
Yiqun Liu
Shaoping Ma

The performance of a recommendation system relies heavily on the feedback of users. Most of the traditional recommendation algorithms based only on historical ratings will encounter several difficulties given the problem of data sparsity. Users' feedback usually contains rich textual reviews in addition to numerical ratings. In this paper, we exploit textual review information, as well as ratings, to model user preferences and item features in a shared topic space and subsequently introduce them into a matrix factorization model for recommendation. To this end, the data sparsity problem is alleviated and good interpretability of the recommendation results is gained. Another contribution of this work is that we model the item feature distributions with rating-boosted reviews which combine textual reviews with user sentiments. Experimental results on 26 real-world datasets from Amazon demonstrate that our approach significantly improves the rating prediction accuracy compared with various state-of-the-art models, such as LFM, HFT, CTR and RMR models. And much higher improvement is achieved for users who have few ratings, which verifies the effectiveness of the proposed approach for sparse data. Moreover, our method also benefits much from reviews on top-N recommendation tasks.

PDF Details

IJCAI Conference 2015 Conference Paper

Catch the Black Sheep: Unified Framework for Shilling Attack Detection Based on Fraudulent Action Propagation

Yongfeng Zhang
Yunzhi Tan
Min Zhang
Yiqun Liu
Tat-Seng Chua
Shaoping Ma

Many e-commerce systems allow users to express their opinions towards products through user reviews systems. The user generated reviews not only help other users to gain a more insightful view of the products, but also help online businesses to make targeted improvements on the products or services. Besides, they compose the key component of various personalized recommender systems. However, the existence of spam user accounts in the review systems introduce unfavourable disturbances into personalized recommendation by promoting or degrading targeted items intentionally through fraudulent reviews. Previous shilling attack detection algorithms usually deal with a specific kind of attacking strategy, and are exhausted to handle with the continuously emerging new cheating methods. In this work, we propose to conduct shilling attack detection for more informed recommendation by fraudulent action propagation on the reviews themselves, without caring about the specific underlying cheating strategy, which allows us a unified and flexible framework to detect the spam users.

PDF Details

IJCAI Conference 2015 Conference Paper

Detecting Promotion Campaigns in Community Question Answering

Xin Li
Yiqun Liu
Min Zhang
Shaoping Ma
Xuan Zhu
Jiashen Sun

With Community Question Answering (CQA) evolving into a quite popular method for information seeking and providing, it also becomes a target for spammers to disseminate promotion campaigns. Although there are a number of quality estimation efforts on the CQA platform, most of these works focus on identifying and reducing lowquality answers, which are mostly generated by impatient or inexperienced answerers. However, a large number of promotion answers appear to provide high-quality information to cheat CQA users in future interactions. Therefore, most existing quality estimation works in CQA may fail to detect these specially designed answers or question-answer pairs. In contrast to these works, we focus on the promotion channels of spammers, which include (shortened) URLs, telephone numbers and social media accounts. Spammers rely on these channels to connect to users to achieve promotion goals so they are irreplaceable for spamming activities. We propose a propagation algorithm to diffuse promotion intents on an “answerer-channel” bipartite graph and detect possible spamming activities. A supervised learning framework is also proposed to identify whether a QA pair is spam based on propagated promotion intents. Experimental results based on more than 6 million entries from a popular Chinese CQA portal show that our approach outperforms a number of existing quality estimation methods for detecting promotion campaigns on both the answer level and QA pair level.

PDF Details

AAAI Conference 2014 Conference Paper

Fraudulent Support Telephone Number Identification Based on Co-Occurrence Information on the Web

Xin Li
Yiqun Liu
Min Zhang
Shaoping Ma

“Fraudulent support phones” refers to the misleading telephone numbers placed on Web pages or other media that claim to provide services with which they are not associated. Most fraudulent support phone information is found on search engine result pages (SERPs), and such information substantially degrades the search engine user experience. In this paper, we propose an approach to identify fraudulent support telephone numbers on the Web based on the co-occurrence relations between telephone numbers that appear on SERPs. We start from a small set of seed official support phone numbers and seed fraudulent numbers. Then, we construct a co-occurrence graph according to the co-occurrence relationships of the telephone numbers that appear on Web pages. Additionally, we take the page layout information into consideration on the assumption that telephone numbers that appear in nearby page blocks should be regarded as more closely related. Finally, we develop a propagation algorithm to diffuse the trust scores of seed official support phone numbers and the distrust scores of the seed fraudulent numbers on the co-occurrence graph to detect additional fraudulent numbers. Experimental results based on over 1. 5 million SERPs produced by a popular Chinese commercial search engine indicate that our approach outperforms TrustRank, Anti-TrustRank and Good-Bad Rank algorithms by achieving an AUC value of over 0. 90.

PDF Details

IJCAI Conference 2011 Conference Paper

Predicting Epidemic Tendency through Search Behavior Analysis

Danqing Xu
Yiqun Liu
Min Zhang
Shaoping Ma
Anqi Cui
Liyun Ru

The possibility that influenza activity can be generally detected through search log analysis has been explored in recent years. However, previous studies have mainly focused on influenza, and little attention has been paid to other epidemics. With an analysis of web user behavior data, we consider the problem of predicting the tendency of hand-foot -and-mouth disease (HFMD), whose out-break in 2010 resulted in a great panic in China. In addi-tion to search queries, we consider users' interactions with search engines. Given the collected search logs, we cluster HFMD-related search queries, medical pages and news reports into the following sets: epidemic-related queries (ERQs), epidemic-related pages (ERPs) and ep-idemic-related news (ERNs). Furthermore, we count their own frequencies as different features, and we conduct a regression analysis with current HFMD occurrences. The experimental results show that these features exhibit good performances on both accuracy and timeliness.

PDF Details DOI