Multi-Label Learning with Deep Forest

In multi-label learning, each instance is associated with multiple labels, and the crucial task is how to leverage label correlations in building models. The deep forest is a recent deep learning framework based on decision tree ensembles, which has a cascade structure that can do representation learning like deep neural models and does not rely on backpropagation. Though deep forests have been found useful in classification tasks, the potential of applying it into multi-label learning has not been studied. We consider that the layer-by-layer processing structure of the deep forest is appropriate for solving multi-label problems. Therefore we design the Multi-Label Deep Forest (MLDF) method, including two mechanisms: measure-aware feature reuse and measure-aware layer growth. The measure-aware feature reuse mechanism enables MLDF to reuse better representation in the previous layer. The measure-aware layer growth mechanism ensures MLDF gradually increase the model complexity guided by performance measure. MLDF handles two challenging problems at the same time: one is restricting the model complexity to ease the overfitting issue; another is optimizing the performance measure on user’s demand since there are many different measures in the multi-label evaluation. Experiments demonstrate that our proposal not only beats the compared methods over six measures on benchmark datasets but also enjoys label correlation discovery and other desired properties in multi-label learning.

Authors

• Liang Yang
• Xi-Zhu Wu
• Yuan Jiang 0001
• Zhi-Hua Zhou

Keywords

No keywords are indexed for this paper.