Fast Factorization-free Kernel Learning for Unlabeled Chunk Data Streams

Data stream analysis aims at extracting discriminative information for classification from continuously incoming samples. It is extremely challenging to detect novel data while updating the model in an efficient and stable fashion, especially for the chunk data. This paper proposes a fast factorization-free kernel learning method to unify novelty detection and incremental learning for unlabeled chunk data streams in one framework. The proposed method constructs a joint reproducing kernel Hilbert space from known class centers by solving a linear system in kernel space. Naturally, unlabeled data can be detected and classified among multi-classes by a single decision model. And projecting samples into the discriminative feature space turns out to be the product of two small-sized kernel matrices without needing such time-consuming factorization like QR-decomposition or singular value decomposition. Moreover, the insertion of a novel class can be treated as the addition of a new orthogonal basis to the existing feature space, resulting in fast and stable updating schemes. Both theoretical analysis and experimental validation on real-world datasets demonstrate that the proposed methods learn chunk data streams with significantly lower computational costs and comparable or superior accuracy than the state of the art.

Authors

• Yi Wang
• Nan Xue Wuhan University Ant Group
• Xin Fan Dalian University of Technology
• Jiebo Luo University of Rochester
• Risheng Liu Dalian University of Technology
• Bin Chen Harbin Institute of Technology, Shenzhen Research Center of Artificial Intelligence, Peng Cheng Laboratory
• Haojie Li X-ERA.ai
• Zhongxuan Luo

Keywords

• Machine Learning: Classification
• Machine Learning: Kernel Methods
• Machine Learning: Online Learning
• Machine Learning: Semi-Supervised Learning