SecureBoost: A Lossless Federated Learning Framework

The protection of user privacy is an important concern in machine learning, as evidenced by the rolling out of the General Data Protection Regulation (GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give users more control over their personal data, which motivates us to explore machine learning frameworks for data sharing that do not violate user privacy. To meet this goal, in this article, we propose a novel lossless privacy-preserving tree-boosting system known as SecureBoost in the setting of federated learning. SecureBoost first conducts entity alignment under a privacy-preserving protocol and then constructs boosting trees across multiple parties with a carefully designed encryption strategy. This federated learning system allows the learning process to be jointly conducted over multiple parties with common user samples but different feature sets, which corresponds to a vertically partitioned dataset. An advantage of SecureBoost is that it provides the same level of accuracy as the non -privacy-preserving approach while at the same time, reveals no information of each private data provider. We show that the SecureBoost framework is as accurate as other nonfederated gradient tree-boosting algorithms that require centralized data, and thus, it is highly scalable and practical for industrial applications such as credit risk analysis. To this end, we discuss information leakage during the protocol execution and propose ways to provably reduce it.

Authors

• Kewei Cheng
• Tao Fan School of Public Administration, Nanjing University of Finance & Economics, China
• Yilun Jin
• Yang Liu College of Computer Science and Software Engineering, Shenzhen University
• Tianjian Chen
• Dimitrios Papadopoulos
• Qiang Yang Hong Kong University of Science and Technology, Hong Kong SAR, China

Keywords

• Data models
• Machine learning
• Collaborative work
• Protocols
• Data privacy
• Servers
• General Data Protection Regulation
• Learning Framework
• Lossless
• Federated Learning
• Federated Learning Framework
• Big Data
• Machine Learning Models
• Personal Data
• European Union
• Confidential Information
• Information Leakage
• Updated Model
• XGBoost
• Label Information
• User Privacy
• Gradient Boosting Decision Tree
• Credit Rating
• Machine Learning Problems
• Retail Business
• Maximum Tree Depth
• Tree Depth
• Bill Payments
• Party Identification
• Class Labels
• Regression Tree
• Area Under Receiver Operating Characteristic Curve
• Optimal Split
• Global Model
• Private Parties
• Privacy
• Security
• Decision Tree