Author name cluster

Weike Pan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

26 papers

2 author rows

TIST Journal 2025 Journal Article

Cross-User Federated Recommendation Unlearning

Yang Li
Enyue Yang
Weike Pan
Qiang Yang
Zhong Ming

Cross-user federated recommendation (CUFR) is a promising solution for providing personalized services without collecting users’ raw data. However, most previous CUFR works mainly focus on providing accurate and privacy-preserving personalized recommendations, but overlook the fact that users can opt out at any time during the training process. In response, we study an emerging and new problem of efficiently training an unlearned model to forget the data of the clients who leave a federated system. It is challenging to simply apply or slightly modify existing machine unlearning or federated unlearning methods to CUFR because of the unique collaboration effect in recommender systems. Although a recent gradient calibration-based method (i.e., FRU) shows promising in training an unlearned model, there are still some limitations: (i) there is a potential possibility that some clients run out of the storage space, (ii) all the remaining clients need to participate in computing the new gradients, (iii) it masks the uniqueness of the local gradients, and (iv) the errors of the calibrated gradients will increase gradually with more iterations. In this article, we propose a novel CUFR unlearning (CUFRU) method. Specifically, we design a gradient transfer station (GTS) module for storing the historical gradients while enabling clients to dynamically participate in the computation of the calibrated gradients with the new gradients based on their online status. Moreover, we design a novel iteration-aware gradient calibration mechanism to strike a balance between the weights of the historical and new gradients at the different stages of the unlearning process, alleviating the calibration errors. Finally, we conduct extensive experiments on three real-world datasets to show that our CUFRU can more efficiently train an unlearned model with the competitive recommendation performance.

Details DOI

IJCAI Conference 2024 Conference Paper

A Survey on Cross-Domain Sequential Recommendation

shu chen
Zitao Xu
Weike Pan
Qiang Yang
Zhong Ming

Cross-domain sequential recommendation (CDSR) shifts the modeling of user preferences from flat to stereoscopic by integrating and learning interaction information from multiple domains at different granularities (ranging from inter-sequence to intra-sequence and from single-domain to cross-domain). In this survey, we initially define the CDSR problem using a four-dimensional tensor and then analyze its multi-type input representations under multidirectional dimensionality reductions. Following that, we provide a systematic overview from both macro and micro views. From a macro view, we abstract the multi-level fusion structures of various models across domains and discuss their bridges for fusion. From a micro view, focusing on the existing models, we specifically discuss the basic technologies and then explain the auxiliary learning technologies. Finally, we exhibit the available public datasets and the representative experimental results as well as provide some insights into future directions for research in CDSR.

PDF Details DOI

TIST Journal 2024 Journal Article

Decentralized Federated Recommendation with Privacy-aware Structured Client-level Graph

Zhitao Li
Zhaohao Lin
Feng Liang
Weike Pan
Qiang Yang
Zhong Ming

Recommendation models are deployed in a variety of commercial applications to provide personalized services for users. However, most of them rely on the users’ original rating records that are often collected by a centralized server for model training, which may cause privacy issues. Recently, some centralized federated recommendation models are proposed for the protection of users’ privacy, which however requires a server for coordination in the whole process of model training. As a response, we propose a novel privacy-aware decentralized federated recommendation (DFedRec) model, which is lossless compared with the traditional model in recommendation performance and is thus more accurate than other models in this line. Specifically, we design a privacy-aware structured client-level graph for the sharing of the model parameters in the process of model training, which is a one-stone-two-bird strategy, i.e., it protects users’ privacy via some randomly sampled fake entries and reduces the communication cost by sharing the model parameters only with the related neighboring users. With the help of the privacy-aware structured client-level graph, we propose two novel collaborative training mechanisms in the setting without a server, including a batch algorithm DFedRec(b) and a stochastic one DFedRec(s), where the former requires the anonymity mechanism while the latter does not. They are both equivalent to probabilistic matrix factorization trained in a centralized server and are thus lossless. We then provide formal analysis of privacy guarantee of our methods and conduct extensive empirical studies on three public datasets with explicit feedback, which show the effectiveness of our DFedRec, i.e., it is privacy aware, communication efficient, and lossless.

Details DOI

ICML Conference 2024 Conference Paper

Delving into Differentially Private Transformer

Youlong Ding
Xueyang Wu 0001
Yining Meng
Yonggang Luo
Hao Wang 0014
Weike Pan

Deep learning with differential privacy (DP) has garnered significant attention over the past years, leading to the development of numerous methods aimed at enhancing model accuracy and training efficiency. This paper delves into the problem of training Transformer models with differential privacy. Our treatment is modular: the logic is to ’reduce’ the problem of training DP Transformer to the more basic problem of training DP vanilla neural nets. The latter is better understood and amenable to many model-agnostic methods. Such ’reduction’ is done by first identifying the hardness unique to DP Transformer training: the attention distraction phenomenon and a lack of compatibility with existing techniques for efficient gradient clipping. To deal with these two issues, we propose the Re-Attention Mechanism and Phantom Clipping, respectively. We believe that our work not only casts new light on training DP Transformers but also promotes a modular treatment to advance research in the field of differentially private deep learning.

Details

NeurIPS Conference 2024 Conference Paper

Federated Graph Learning for Cross-Domain Recommendation

Ziqi Yang
Zhaopeng Peng
Zihui Wang
Jianzhong Qi
Chaochao Chen
Weike Pan
Chenglu Wen
Cheng Wang

Cross-domain recommendation (CDR) offers a promising solution to the data sparsity problem by enabling knowledge transfer across source and target domains. However, many recent CDR models overlook crucial issues such as privacy as well as the risk of negative transfer (which negatively impact model performance), especially in multi-domain settings. To address these challenges, we propose FedGCDR, a novel federated graph learning framework that securely and effectively leverages positive knowledge from multiple source domains. First, we design a positive knowledge transfer module that ensures privacy during inter-domain knowledge transmission. This module employs differential privacy-based knowledge extraction combined with a feature mapping mechanism, transforming source domain embeddings from federated graph attention networks into reliable domain knowledge. Second, we design a knowledge activation module to filter out potential harmful or conflicting knowledge from source domains, addressing the issues of negative transfer. This module enhances target domain training by expanding the graph of the target domain to generate reliable domain attentions and fine-tunes the target model for improved negative knowledge filtering and more accurate predictions. We conduct extensive experiments on 16 popular domains of the Amazon dataset, demonstrating that FedGCDR significantly outperforms state-of-the-art methods.

PDF Details DOI

AIJ Journal 2023 Journal Article

Privacy-preserving graph convolution network for federated item recommendation

Pengqing Hu
Zhaohao Lin
Weike Pan
Qiang Yang
Xiaogang Peng
Zhong Ming

In traditional recommender systems, we often build models based on a centralized storage of user data, which however will lead to user privacy concerns and risks. In this paper, we study an emerging and important recommendation problem called federated item recommendation (FIR), in which a recommendation model is built with decentralized data of user-item interactions in a privacy-aware manner, i. e. , the personal behavior data of each user does not leave the owner. Recently, graph neural network (GNN) has been widely recognized as a state-of-the-art solution for item recommendation with implicit feedback since it is able to model the high-order connectivity between users and items. However, it is very challenging to exploit the high-order connectivity information in a decentralized user-item interaction graph without compromising user privacy. To address that, we propose a GNN-based federated recommendation framework, i. e. , privacy-preserving graph convolution network (P-GCN), for the studied problem of FIR. Our P-GCN can leverage the high-order connectivity information like a centralized GCN model such as LightGCN, though it is built using a decentralized user-item graph. To achieve that, we design a novel privacy-preserving graph convolution approach based on secure aggregation and employ item-based user representation to compensate for the performance loss it causes due to the protection of user privacy. Moreover, we improve a group-wise concealing strategy for protecting user privacy. Empirical studies on three datasets show that our P-GCN can achieve similar or even better performance comparing with the non-federated (i. e. , centralized) counterpart, and outperforms all the existing federated methods for the studied problem.

Details DOI

AIJ Journal 2023 Journal Article

Transfer learning for collaborative recommendation with biased and unbiased data

Zinan Lin
Dugang Liu
Weike Pan
Qiang Yang
Zhong Ming

In a recommender system, a user's interaction is often biased by the items' displaying positions and popularity, as well as the user's self-selection. Most existing recommendation models are built using such a biased user-system interaction data alone. In this paper, we introduce an additional specially collected unbiased data, and then have a new problem called collaborative recommendation with biased and unbiased data. We first formalize the studied problem and list three challenges, including the bias challenge, the heterogeneity challenge and the unbalance challenge. Then we propose a novel transfer learning-based AI solution, i. e. , transfer via joint reconstruction (TJR), to achieve knowledge transfer and sharing between the biased data and unbiased data. Specifically, in our TJR, we use two different models to extract the users' preferences and bias information, and then refine the prediction via the latent features containing the bias information in order to obtain a more accurate and unbiased recommendation. We further integrate the two data by reconstructing their interaction in a joint learning manner. Moreover, in order to better address the unbalance challenge, we introduce a bias regularization term and integrate bidirectional knowledge distillation. Finally, we adopt four representative methods, i. e. , variational autoencoders, matrix factorization, neural collaborative filtering and graph convolution network, as the backbone models of our TJR and conduct extensive empirical studies on three public datasets, showcasing the effectiveness of our transfer learning solution over some very competitive baselines.

Details DOI

TIST Journal 2022 Journal Article

FLAG: A Feedback-aware Local and Global Model for Heterogeneous Sequential Recommendation

Mingkai He
Jing Lin
Jinwei Luo
Weike Pan
Zhong Ming

Heterogeneous sequential recommendation that models sequences of items associated with more than one type of feedback such as examinations and purchases is an emerging topic in the research community, which is also an important problem in many real-world applications. Though there are some methods proposed to exploit different types of feedback in item sequences such as RLBL, RIB, and BINN, they are based on RNN and may not be very competitive in capturing users’ complex and dynamic preferences. And most existing advanced sequential recommendation methods such as the CNN- and attention-based methods are often designed for making use of item sequences with one single type of feedback, which thus can not be applied to the studied problem directly. As a response, we propose a novel feedback-aware local and global (FLAG) preference learning model for heterogeneous sequential recommendation. Our FLAG contains four modules, including (i) a local preference learning module for capturing a user’s short-term interest, which adopts a novel feedback-aware self-attention block to distinguish different types of feedback; (ii) a global preference learning module for modeling a user’s global preference; (iii) a local intention learning module, which takes a user’s real feedback in the next step, i.e., the user’s intention at the current step, as the query vector in a self-attention block to figure out the items that match the user’s intention well; and (iv) a prediction module for preference integration and final prediction. We then conduct extensive experiments on three public datasets and find that our FLAG significantly outperforms 13 very competitive baselines in terms of two commonly used ranking-oriented metrics in most cases. We also include ablation studies and sensitivity analysis of our FLAG to have more in-depth insights.

Details DOI

IJCAI Conference 2022 Conference Paper

Multi-Constraint Deep Reinforcement Learning for Smooth Action Control

Guangyuan Zou
Ying He
F. Richard Yu
Longquan Chen
Weike Pan
Zhong Ming

Deep reinforcement learning (DRL) has been studied in a variety of challenging decision-making tasks, e. g. , autonomous driving. \textcolor{black}{However, DRL typically suffers from the action shaking problem, which means that agents can select actions with big difference even though states only slightly differ. } One of the crucial reasons for this issue is the inappropriate design of the reward in DRL. In this paper, to address this issue, we propose a novel way to incorporate the smoothness of actions in the reward. Specifically, we introduce sub-rewards and add multiple constraints related to these sub-rewards. In addition, we propose a multi-constraint proximal policy optimization (MCPPO) method to solve the multi-constraint DRL problem. Extensive simulation results show that the proposed MCPPO method has better action smoothness compared with the traditional proportional-integral-differential (PID) and mainstream DRL algorithms. The video is available at https: //youtu. be/F2jpaSm7YOg.

PDF Details DOI

AAAI Conference 2021 Conference Paper

FedRec++: Lossless Federated Recommendation with Explicit Feedback

Feng Liang
Weike Pan
Zhong Ming

With the marriage of federated machine learning and recommender systems for privacy-aware preference modeling and personalization, there comes a new research branch called federated recommender systems aiming to build a recommendation model in a distributed way, i. e. , each user is represented as a distributed client where his/her original rating data are not shared with the server or the other clients. Notice that, besides the sensitive information of a specific rating score assigned to a certain item by a user, the information of a user’s rated set of items shall also be well protected. Some very recent works propose to randomly sample some unrated items for each user and then assign some virtual ratings, so that the server can not identify the scores and the set of rated items easily during the server-client interactions. However, the virtual ratings assigned to the randomly sampled items will inevitably introduce some noise to the model training process, which will then cause loss in recommendation performance. In this paper, we propose a novel lossless federated recommendation method (FedRec++) by allocating some denoising clients (i. e. , users) to eliminate the noise in a privacy-aware manner. We further analyse our FedRec++ in terms of security and losslessness, and discuss its generality in the context of existing works. Extensive empirical studies clearly show the effectiveness of our FedRec++ in providing accurate and privacy-aware recommendation without much additional communication cost.

PDF Details

IS Journal 2021 Journal Article

FedRec: Federated Recommendation With Explicit Feedback

Guanyu Lin
Feng Liang
Weike Pan
Zhong Ming

Recommendation models have been widely embedded in various online services, while most of which are designed with the assumption that users’ original behaviors are available in a central server. This may cause the privacy issue. As a response, we follow a recent work called federated collaborative filtering (FCF) for item recommendation with implicit feedback and propose a novel and generic federated recommendation (FedRec) framework for rating prediction with explicit feedback. Specifically, we federate some basic and advanced factorization-based recommendation models both in batch style and in stochastic style. More importantly, in order to protect the private information of which items each user has rated, as well as not to significantly increase the computational and communication cost, we design two simple but effective strategies, i. e. , user averaging and hybrid filling, in which some (instead of all) unrated items are randomly sampled and assigned with some virtual ratings accordingly. Empirical studies on two public datasets show the effectiveness of our FedRec in terms of the closeness of a federated model and an unfederated one, and the usefulness of the two filling strategies.

Details DOI

TIST Journal 2020 Journal Article

CoFi-points

Lin Li
Weike Pan
Zhong Ming

With the explosive growth of web resources, an increasingly important task in recommender systems is to provide high-quality personalized services by learning users’ preferences from historically observed information. As an effective preference learning technology, collaborative filtering has been widely extended to model the one-class or implicit feedback data, which is known as one-class collaborative filtering (OCCF). For a long time, pairwise ranking-oriented learning scheme has been viewed as a superior solution than the pointwise scheme for OCCF due to its higher accuracy in most cases. However, we argue that with appropriate model design, pointwise preference learning can achieve comparable or even better performance than the counterpart, i.e., pairwise preference learning. In particular, we propose a new preference assumption, i.e., pointwise preference on user/item-set. Based on this new assumption, we develop a novel, simple, and flexible solution called collaborative filtering via pointwise preference learning on user/item-set (CoFi-points). Furthermore, we derive two specific algorithms of CoFi-points with respect to the involved user-set and item-set, i.e., CoFi-points(u) and CoFi-points(i), referring to preference assumptions defined on user-set and item-set, respectively. Finally, we conduct extensive empirical studies on four real-world datasets with the state-of-the-art methods, and find that our solution can achieve very promising performance with respect to several ranking-oriented evaluation metrics.

Details DOI

IS Journal 2017 Journal Article

Big Data

Weike Pan
Qiang Yang
Charu Aggarwal
Christoph Koch

Big data has been an enabler for innovation, reconstruction, and advancement of most sectors of our society, and it's receiving continuous and growing attention from researchers and practitioners in academia, industry, and government. There are, however, still lots of challenges spanning from theoretical foundations, systems, and technology to data policy and standards. This special issue focuses on how big data cuts across systems and applications arenas, and the guest editors' introduction describes the five articles they selected out of 30 submitted to cover a wide spectrum of interesting topics, including feature selection for big data analytics, astronomical image analysis, large-scale network prediction, online URL filtering, and massive transaction clustering.

Details DOI

IS Journal 2017 Journal Article

Collaborative Recommendation with Multiclass Preference Context

Weike Pan
Zhong Ming

Factorization- and neighborhood-based methods have been recognized as state-of-the-art approaches for collaborative recommendation tasks. In this article, the authors take user ratings as categorical multiclass preferences and propose a novel method called matrix factorization with multiclass preference context (MF-MPC), which integrates an enhanced neighborhood based on the assumption that users with similar past multiclass preferences (instead of one-class preferences in SVD++) will have similar tastes in the future. The main merit of MF-MPC is its ability to make use of the multiclass preference context in the factorization framework in a fine-grained manner and thus inherit the advantages of those two methods. Experimental results on three real-world datasets show that their solution can perform significantly better than factorization-based methods, neighborhood-based methods, and integrated methods with a one-class preference context.

Details DOI

TIST Journal 2017 Journal Article

Transfer Learning for Behavior Ranking

Weike Pan
Qiang Yang
Yuchao Duan
Ben Tan
Zhong Ming

Intelligent recommendation has been well recognized as one of the major approaches to address the information overload problem in the big data era. A typical intelligent recommendation engine usually consists of three major components, that is, data as the main input, algorithms for preference learning, and system for user interaction and high-performance computation. We observe that the data (e.g., users’ behavior) are usually in different forms, such as examinations (e.g., browse and collection) and ratings, where the former are often much more abundant than the latter. Although the data are in different representations, they are both related to users’ true preferences and are also deemed complementary to each other for preference learning. However, very few ranking or recommendation algorithms have been developed to exploit such two types of user behavior. In this article, we focus on jointly modeling the examination behavior and rating behavior and develop a novel and efficient ranking-oriented recommendation algorithm accordingly. First, we formally define a new recommendation problem termed behavior ranking, which aims to build a ranking-oriented model by exploiting both the examination behavior and rating behavior. Second, we develop a simple and generic transfer to rank (ToR) algorithm for behavior ranking, which transfers knowledge of candidate items from a global preference learning task to a local preference learning task. Compared with the previous work on integrating heterogeneous user behavior, our ToR algorithm is the first ranking-oriented solution, which can effectively generate recommendations in a more direct manner than those regression-oriented methods. Extensive empirical studies show that our ToR algorithm performs significantly more accurately than the state-of-the-art methods in most cases. Furthermore, our ToR algorithm is very efficient in terms of the time complexity, which is similar to those for homogeneous user behavior alone.

Details DOI

IS Journal 2016 Journal Article

Transfer Learning for Heterogeneous One-Class Collaborative Filtering

Weike Pan
Mengsi Liu
Zhong Ming

Various memory- and model-based collaborative filtering algorithms have been designed for multiclass feedback (such as grade scores) in the past two decades. Recently, one-class feedback (such as positive feedback and implicit examination) has been recognized as a more pervasive and important source of information in many real recommendation systems. Previous work along these lines mainly focus on homogenous one-class positive feedback, such as likes on Facebook or transactions on Amazon, which might not capture a user's true preferences due to the sparsity of such data. To alleviate this sparsity problem, the authors study positive feedback and implicit examinations simultaneously, coined as heterogeneous one-class collaborative filtering (HOCCF). Specifically, they designed a novel transfer learning algorithm for HOCCF, called transfer via joint similarity learning (TJSL), that jointly learns a similarity between a candidate item and a preferred item, and a similarity between a candidate item and an identified likely-to-prefer examined item. Joint similarity learning has the merit of being able to connect two seemingly unrelated items along sparse positive feedback only. Empirical studies on three real-world datasets show that TJSL can recommend items more accurately than other state-of-the-art methods.

Details DOI

IS Journal 2014 Journal Article

An Adaptive Fusion Algorithm for Spam Detection

Congfu Xu
Baojun Su
Yunbiao Cheng
Weike Pan
Li Chen

Spam detection has become a critical component in various online systems such as email services, advertising engines, social media sites, and so on. Here, the authors use email services as an example, and present an adaptive fusion algorithm for spam detection (AFSD), which is a general, content-based approach and can be applied to nonemail spam detection tasks with little additional effort. The proposed algorithm uses n-grams of nontokenized text strings to represent an email, introduces a link function to convert the prediction scores of online learners to become more comparable, trains the online learners in a mistake-driven manner via thick thresholding to obtain highly competitive online learners, and designs update rules to adaptively integrate the online learners to capture different aspects of spams. The prediction performance of AFSD is studied on five public competition datasets and on one industry dataset, with the algorithm achieving significantly better results than several state-of-the-art approaches, including the champion solutions of the corresponding competitions.

Details DOI

IS Journal 2014 Journal Article

Bird Flu Outbreak Prediction via Satellite Tracking

Yuanchun Zhou
Mingjie Tang
Weike Pan
Jinyan Li
Weihang Wang
Jing Shao
Liang Wu
Jianhui Li

Advanced satellite tracking technologies have collected huge amounts of wild bird migration data. Biologists use these data to understand dynamic migration patterns, study correlations between habitats, and predict global spreading trends of avian influenza. The research discussed here transforms the biological problem into a machine learning problem by converting wild bird migratory paths into graphs. H5N1 outbreak prediction is achieved by discovering weighted closed cliques from the graphs using the mining algorithm High-wEight cLosed cliquE miNing (HELEN). The learning algorithm HELEN-p then predicts potential H5N1 outbreaks at habitats. This prediction method is more accurate than traditional methods used on a migration dataset obtained through a real satellite bird-tracking system. Empirical analysis shows that H5N1 spreads in a manner of high-weight closed cliques and frequent cliques.

Details DOI

IS Journal 2014 Journal Article

Interaction-Rich Transfer Learning for Collaborative Filtering with Heterogeneous User Feedback

Weike Pan
Zhong Ming

A real recommender system can usually make use of more than one type of user feedback--for example, numerical ratings and binary ratings-to learn a user's true preferences. Recent work has proposed a transfer learning algorithm called transfer by collective factorization (TCF) to exploit such heterogeneous user feedback. TCF performs via sharing data-independent knowledge and modeling data-dependent effects simultaneously. However, TCF is a batch algorithm and updates the model parameters only once after scanning all data, which might not be efficient enough for real systems. This article proposes a novel and efficient transfer learning algorithm called interaction-rich transfer by collective factorization (iTCF), which extends the efficient collective matrix factorization (CMF) algorithm by providing more interactions between the user-specific latent features. The assumption under iTCF is that the predictability with regards to the same user's rating behaviors in two related data is likely to be similar. Considering the shared predictability, the authors derive novel update rules for iTCF in a stochastic algorithmic framework. The advantages of iTCF include its efficiency compared with TCF, and its higher prediction accuracy compared with CMF. Experimental results on three real-world datasets show the effectiveness of iTCF over the state-of-the-art methods.

Details DOI

IJCAI Conference 2013 Conference Paper

GBPR: Group Preference Based Bayesian Personalized Ranking for One-Class Collaborative Filtering

Weike Pan
Li Chen

One-class collaborative ﬁltering or collaborative ranking with implicit feedback has been steadily receiving more attention, mostly due to the “oneclass” characteristics of data in various services, e. g. , “like” in Facebook and “bought” in Amazon. Previous works for solving this problem include pointwise regression methods based on absolute rating assumptions and pairwise ranking methods with relative score assumptions, where the latter was empirically found performing much better because it models users’ ranking-related preferences more directly. However, the two fundamental assumptions made in the pairwise ranking methods, (1) individual pairwise preference over two items and (2) independence between two users, may not always hold. As a response, we propose a new and improved assumption, group Bayesian personalized ranking (GBPR), via introducing richer interactions among users. In particular, we introduce group preference, to relax the aforementioned individual and independence assumptions. We then design a novel algorithm correspondingly, which can recommend items more accurately as shown by various ranking-oriented evaluation metrics on four real-world datasets in our experiments.

PDF Details DOI

AIJ Journal 2013 Journal Article

Transfer learning in heterogeneous collaborative filtering domains

Weike Pan
Qiang Yang

A major challenge for collaborative filtering (CF) techniques in recommender systems is the data sparsity that is caused by missing and noisy ratings. This problem is even more serious for CF domains where the ratings are expressed numerically, e. g. as 5-star grades. We assume the 5-star ratings are unordered bins instead of ordinal relative preferences. We observe that, while we may lack the information in numerical ratings, we sometimes have additional auxiliary data in the form of binary ratings. This is especially true given that users can easily express themselves with their preferences expressed as likes or dislikes for items. In this paper, we explore how to use these binary auxiliary preference data to help reduce the impact of data sparsity for CF domains expressed in numerical ratings. We solve this problem by transferring the rating knowledge from some auxiliary data source in binary form (that is, likes or dislikes), to a target numerical rating matrix. In particular, our solution is to model both the numerical ratings and ratings expressed as like or dislike in a principled way. We present a novel framework of Transfer by Collective Factorization (TCF), in which we construct a shared latent space collectively and learn the data-dependent effect separately. A major advantage of the TCF approach over the previous bilinear method of collective matrix factorization is that we are able to capture the data-dependent effect when sharing the data-independent knowledge. This allows us to increase the overall quality of knowledge transfer. We present extensive experimental results to demonstrate the effectiveness of TCF at various sparsity levels, and show improvements of our approach as compared to several state-of-the-art methods.

Details DOI

AAAI Conference 2012 Conference Paper

Transfer Learning in Collaborative Filtering with Uncertain Ratings

Weike Pan
Evan Xiang
Qiang Yang

To solve the sparsity problem in collaborative filtering, researchers have introduced transfer learning as a viable approach to make use of auxiliary data. Most previous transfer learning works in collaborative filtering have focused on exploiting point-wise ratings such as numerical ratings, stars, or binary ratings of likes/dislikes. However, in many real-world recommender systems, many users may be unwilling or unlikely to rate items with precision. In contrast, practitioners can turn to various non-preference data to estimate a range or rating distribution of a user’s preference on an item. Such a range or rating distribution is called an uncertain rating since it represents a rating spectrum of uncertainty instead of an accurate point-wise score. In this paper, we propose an efficient transfer learning solution for collaborative filtering, known as transfer by integrative factorization (TIF), to leverage such auxiliary uncertain ratings to improve the performance of recommendation. In particular, we integrate auxiliary data of uncertain ratings as additional constraints in the target matrix factorization problem, and learn an expected rating value for each uncertain rating automatically. The advantages of our proposed approach include the efficiency and the improved effectiveness of collaborative filtering, showing that incorporating the auxiliary data of uncertain ratings can really bring a benefit. Experimental results on two movie recommendation tasks show that our TIF algorithm performs significantly better over a state-ofthe-art non-transfer learning method.

PDF Details

IJCAI Conference 2011 Conference Paper

Source-Selection-Free Transfer Learning

Evan Wei Xiang
Sinno Jialin Pan
Weike Pan
Jian Su
Qiang Yang

Transfer learning addresses the problems that labeled training data are insufficient to produce a high-performance model. Typically, given a target learning task, most transfer learning approaches require to select one or more auxiliary tasks as sources by the designers. However, how to select the right source data to enable effective knowledge transfer automatically is still an unsolved problem, which limits the applicability of transfer learning. In this paper, we take one step ahead and propose a novel transfer learning framework, known as source-selection-free transfer learning (SSFTL), to free users from the need to select source domains. Instead of asking the users for source and target data pairs, as traditional transfer learning does, SSFTL turns to some online information sources such as World Wide Web or the Wikipedia for help. The source data for transfer learning can be hidden somewhere within this large online information source, but the users do not know where they are. Based on the online information sources, we train a large number of classifiers. Then, given a target task, a bridge is built for labels of the potential source candidates and the target domain data in SSFTL via some large online social media with tag cloud as a label translator. An added advantage of SSFTL is that, unlike many previous transfer learning approaches, which are difficult to scale up to the Web scale, SSFTL is highly scalable and can offset much of the training work to offline stage. We demonstrate the effectiveness and efficiency of SSFTL through extensive experiments on several real-world datasets in text classification.

PDF Details DOI

IJCAI Conference 2011 Conference Paper

Transfer Learning to Predict Missing Ratings via Heterogeneous User Feedbacks

Weike Pan
Nathan N. Liu
Evan W. Xiang
Qiang Yang

Data sparsity due to missing ratings is a major challenge for collaborative filtering (CF) techniques in recommender systems. This is especially true for CF domains where the ratings are expressed numerically. We observe that, while we may lack the information in numerical ratings, we may have more data in the form of binary ratings. This is especially true when users can easily express themselves with their likes and dislikes for certain items. In this paper, we explore how to use the binary preference data expressed in the form of like/dislike to help reduce the impact of data sparsity of more expressive numerical ratings. We do this by transferring the rating knowledge from some auxiliary data source in binary form (that is, likes or dislikes), to a target numerical rating matrix. Our solution is to model both numerical ratings and like/dislike in a principled way, using a novel framework of Transfer by Collective Factorization (TCF). In particular, we construct the shared latent space collectively and learn the data-dependent effect separately. A major advantage of the TCF approach over previous collective matrix factorization (or bi-factorization) methods is that we are able to capture the data-dependent effect when sharing the data-independent knowledge, so as to increase the overall quality of knowledge transfer. Experimental results demonstrate the effectiveness of TCF at various sparsity levels as compared to several state-of-the-art methods.

PDF Details DOI

AAAI Conference 2010 Conference Paper

Transfer Learning in Collaborative Filtering for Sparsity Reduction

Weike Pan
Evan Xiang
Nathan Liu
Qiang Yang

Data sparsity is a major problem for collaborative filtering (CF) techniques in recommender systems, especially for new users and items. We observe that, while our target data are sparse for CF systems, related and relatively dense auxiliary data may already exist in some other more mature application domains. In this paper, we address the data sparsity problem in a target domain by transferring knowledge about both users and items from auxiliary data sources. We observe that in different domains the user feedbacks are often heterogeneous such as ratings vs. clicks. Our solution is to integrate both user and item knowledge in auxiliary data sources through a principled matrix-based transfer learning framework that takes into account the data heterogeneity. In particular, we discover the principle coordinates of both users and items in the auxiliary data matrices, and transfer them to the target domain in order to reduce the effect of data sparsity. We describe our method, which is known as coordinate system transfer or CST, and demonstrate its effectiveness in alleviating the data sparsity problem in collaborative filtering. We show that our proposed method can significantly outperform several state-of-the-art solutions for this problem.

PDF Details

NeurIPS Conference 2009 Conference Paper

Accelerated Gradient Methods for Stochastic Optimization and Online Learning

Chonghai Hu
Weike Pan
James Kwok

Regularized risk minimization often involves non-smooth optimization, either because of the loss function (e. g. , hinge loss) or the regularizer (e. g. , $\ell_1$-regularizer). Gradient descent methods, though highly scalable and easy to implement, are known to converge slowly on these problems. In this paper, we develop novel accelerated gradient methods for stochastic optimization while still preserving their computational simplicity and scalability. The proposed algorithm, called SAGE (Stochastic Accelerated GradiEnt), exhibits fast convergence rates on stochastic optimization with both convex and strongly convex objectives. Experimental results show that SAGE is faster than recent (sub)gradient methods including FOLOS, SMIDAS and SCD. Moreover, SAGE can also be extended for online learning, resulting in a simple but powerful algorithm.

PDF Details