Probabilistic Non-Negative Matrix Factorization and Its Robust Extensions for Topic Modeling

Traditional topic model with maximum likelihood estimate inevitably suffers from the conditional independence of words given the documents topic distribution. In this paper, we follow the generative procedure of topic model and learn the topic-word distribution and topics distribution via directly approximating the word-document co-occurrence matrix with matrix decomposition technique. These methods include: (1) Approximating the normalized document-word conditional distribution with the documents probability matrix and words probability matrix based on probabilistic nonnegative matrix factorization (NMF); (2) Since the standard NMF is well known to be non-robust to noises and outliers, we extended the probabilistic NMF of the topic model to its robust versions using 2, 1-norm and capped 2, 1-norm based loss functions, respectively. The proposed framework inherits the explicit probabilistic meaning of factors in topic models and simultaneously makes the conditional independence assumption on words unnecessary. Straightforward and efficient algorithms are exploited to solve the corresponding nonsmooth and non-convex problems. Experimental results over several benchmark datasets illustrate the effectiveness and superiority of the proposed methods.

Authors

• Minnan Luo School of Computer Science and Technology, Xi’an Jiaotong University, Ministry of Education Key Laboratory of Intelligent Networks and Network Security, Xi’an Jiaotong University. Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi’an Jiaotong University.
• Feiping Nie Northwest Polytechnical University Xi'an
• Xiaojun Chang University of Science and Technology of China Mohamed Bin Zayed University of Artificial Intelligenc
• Yi Yang
• Alexander Hauptmann
• Qinghua Zheng School of Computer Science and Technology, Xi'an Jiaotong University, China Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering, Xi'an Jiaotong University, China

Keywords

No keywords are indexed for this paper.