Multicategory large margin classification methods: Hinge losses vs. coherence functions

Generalization of large margin classification methods from the binary classification setting to the more general multicategory setting is often found to be non-trivial. In this paper, we study large margin classification methods that can be seamlessly applied to both settings, with the binary setting simply as a special case. In particular, we explore the Fisher consistency properties of multicategory majorization losses and present a construction framework of majorization losses of the 0–1 loss. Under this framework, we conduct an in-depth analysis about three widely used multicategory hinge losses. Corresponding to the three hinge losses, we propose three multicategory majorization losses based on a coherence function. The limits of the three coherence losses as the temperature approaches zero are the corresponding hinge losses, and the limits of the minimizers of their expected errors are the minimizers of the expected errors of the corresponding hinge losses. Finally, we develop multicategory large margin classification methods by using a so-called multiclass C -loss.

Authors

• Zhihua Zhang
• Cheng Chen
• Guang Dai SGIT AI Lab, State Grid Corporation of China
• Wu-Jun Li National Key Laboratory for Novel Software Technology, School of Computer Science, Nanjing University
• Dit-Yan Yeung Hong Kong University of Science and Technology

Keywords

• Multiclass margin classification
• Fisher consistency
• Multicategory hinge losses
• Coherence losses
• Multicategory boosting algorithm