Dual-View Whitening on Pre-trained Text Embeddings for Sequential Recommendation

Recent advances in sequential recommendation models have demonstrated the efficacy of integrating pre-trained text embeddings with item ID embeddings to achieve superior performance. However, our study takes a unique perspective by exclusively focusing on the untapped potential of text embeddings, obviating the need for ID embeddings. We begin by implementing a pre-processing strategy known as whitening, which effectively transforms the anisotropic semantic space of pre-trained text embeddings into an isotropic Gaussian distribution. Comprehensive experiments reveal that applying whitening to pre-trained text embeddings in sequential recommendation models significantly enhances performance. Yet, a full whitening operation might break the potential manifold of items with similar text semantics. To retain the original semantics while benefiting from the isotropy of the whitened text features, we propose a Dual-view Whitening method for Sequential Recommendation (DWSRec), which leverages both fully whitened and relaxed whitened item representations as dual views for effective recommendations. We further examine the advantages of our approach through both empirical and theoretical analyses. Experiments on three public benchmark datasets show that DWSRec outperforms state-of-the-art methods for sequential recommendation.

Authors

• Lingzi Zhang School of Computer Science and Engineering, Nanyang Technological University, Singapore Alibaba-NTU Singapore Joint Research Institute, Nanyang Technological University, Singapore
• Xin Zhou Sina Weibo Inc.
• Zhiwei Zeng School of Computer Science and Engineering, Nanyang Technological University, Singapore
• Zhiqi Shen Nanyang Technological University

Keywords

• DMKM: Recommender Systems