A Novel Neural Topic Model and Its Supervised Extension

Topic modeling techniques have the benefits of modeling words and documents uniformly under a probabilistic framework. However, they also suffer from the limitations of sensitivity to initialization and unigram topic distribution, which can be remedied by deep learning techniques. To explore the combination of topic modeling and deep learning techniques, we first explain the standard topic model from the perspective of a neural network. Based on this, we propose a novel neural topic model (NTM) where the representation of words and documents are efficiently and naturally combined into a uniform framework. Extending from NTM, we can easily add a label layer and propose the supervised neural topic model (sNTM) to tackle supervised tasks. Experiments show that our models are competitive in both topic discovery and classification/regression tasks.

Authors

• Ziqiang Cao School of Computer Science and Technology, Soochow University, China
• Sujian Li
• Yang Liu College of Computer Science, Sichuan University Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education
• Wenjie Li Computation Department, The Hong Kong Polytechnic University, Hong Kong
• Heng Ji University of Illinois at Urbana-Champaign

Keywords

No keywords are indexed for this paper.