Industrial Federated Topic Modeling

Probabilistic topic modeling has been applied in a variety of industrial applications. Training a high-quality model usually requires a massive amount of data to provide comprehensive co-occurrence information for the model to learn. However, industrial data such as medical or financial records are often proprietary or sensitive, which precludes uploading to data centers. Hence, training topic models in industrial scenarios using conventional approaches faces a dilemma: A party (i.e., a company or institute) has to either tolerate data scarcity or sacrifice data privacy. In this article, we propose a framework named Industrial Federated Topic Modeling (iFTM), in which multiple parties collaboratively train a high-quality topic model by simultaneously alleviating data scarcity and maintaining immunity to privacy adversaries. iFTM is inspired by federated learning, supports two representative topic models (i.e., Latent Dirichlet Allocation and SentenceLDA) in industrial applications, and consists of novel techniques such as private Metropolis-Hastings, topic-wise normalization, and heterogeneous model integration. We conduct quantitative evaluations to verify the effectiveness of iFTM and deploy iFTM in two real-life applications to demonstrate its utility. Experimental results verify iFTM’s superiority over conventional topic modeling.

Authors

• Di Jiang
• Yongxin Tong State Key Laboratory of Complex & Critical Software Environment, Beihang University, China
• Yuanfeng Song
• Xueyang Wu The Hong Kong University of Science and Technology
• Weiwei Zhao Shanghai Jiao Tong University
• Jinhua Peng
• Rongzhong Lian
• Qian Xu HKUST
• Qiang Yang Hong Kong University of Science and Technology, Hong Kong SAR, China

Keywords

No keywords are indexed for this paper.