implicit Online Learning with Kernels

We present two new algorithms for online learning in reproducing kernel Hilbert spaces. Our first algorithm, ILK (implicit online learning with kernels), employs a new, implicit update technique that can be applied to a wide variety of convex loss functions. We then introduce a bounded memory version, SILK (sparse ILK), that maintains a compact representation of the predictor without compromising solution quality, even in non-stationary environments. We prove loss bounds and analyze the convergence rate of both. Experimental evidence shows that our proposed algorithms outperform current methods on synthetic and real data.

Authors

• Li Cheng University of Alberta
• Dale Schuurmans
• Shaojun Wang
• Terry Caelli
• S. V. N. Vishwanathan

Keywords

No keywords are indexed for this paper.